NVIDIA Nemotron 3.5 Bolt E4B: A 4GB Open Model for Local Voice and Device Agents, and What the Numbers Say

NVIDIA released Nemotron 3.5 Bolt E4B 4GB on October 23. It is an open small model for local conversational agents: voice assistants, game characters and robot or IoT interfaces. NVIDIA says it builds on Gemma 4 E4B with Nemotron post-training and needs about 4GB in its target workloads. All the numbers below are NVIDIA's own, and I have not run the model.

I read one primary source: NVIDIA's release page, NVIDIA Nemotron 3.5 Bolt E4B 4GB. I did not read the technical report, the model card or the training recipes it links.

What it is

Per the page, Bolt E4B offers multi-turn tool use, a conversational mode that encourages concise, speech-ready replies, configurable reasoning that can be switched off or capped for latency, a 64k context window, and support for 35 languages on a base pre-trained on more than 140.

NVIDIA says it gives about 40% higher average quality than its prior model, Nemotron 3 Nano 4B, on key conversational agent benchmarks, and up to about 80% fewer output tokens than peers when reasoning is enabled.

The benchmark table, read carefully

NVIDIA compares Bolt E4B with Nemotron 3 Nano 4B, Qwen 3.5 4B and Gemma 4 E4B. It says the results are BF16 with the same reasoning mode across models, and that reasoning mode varies by benchmark. A selection from its tables:

Benchmark	Bolt E4B	Nano 4B	Qwen 3.5 4B	Gemma 4 E4B
MultiChallenge	53.1	35.3	51.8	42.1
IHEval (instruction hierarchy)	84.8	51.3	60.6	67.0
LongMemEval	72.1	25.2	66.8	70.9
BFCL-v3	68.5	61.1	70.9	66.4
Tau2-Bench	55.1	29.2	75.1	38.5
LongBench v2 (48K)	50.5	31.4	59.9	49.3
RULER (64K)	87.0	91.5	95.1	86.5

NVIDIA's headline is that Bolt E4B leads its peers on average. The table does not show a clean win. Qwen 3.5 4B scores higher on Tau2-Bench (75.1 against 55.1), BFCL-v3, LongBench v2 and RULER, and Nemotron 3 Nano 4B beats Bolt on RULER too. Bolt leads on instruction hierarchy, role adherence and several grounding tests. Two of the benchmarks, Brevity Bench and When2Call-MT, are NVIDIA's own and "to be released at a later date", so nobody outside can rerun them yet.

NVIDIA also says that no single benchmark covers what these agents need, and that benchmark gains must translate to the target application. That is the right caveat.

Memory and speed

NVIDIA reports Q4_K_M results at batch size 1 on llama.cpp with CUDA:

Hardware	Workload (cache / input / output tokens)	VRAM	Time to first token	Tokens per second
RTX 5070 (12GB)	0 / 3K / 50	3.5 GB	548 ms	133.4
RTX 5070 (12GB)	32k / 2K / 256	4.3 GB	1,238 ms	106.2
Jetson Orin Nano (8GB)	0 / 3K / 50	4.2 GB	5,785 ms	15.5
Jetson Orin Nano (8GB)	32k / 2K / 256	4.9 GB	6,259 ms	13.0
Jetson AGX Orin (32GB)	0 / 3K / 50	4.2 GB	3,687 ms	24.0

The page also lists RTX Spark rows. On the Orin Nano a 5.8 second wait for the first token will not feel like a real-time voice assistant. NVIDIA's own table shows the gap between desktop RTX and small Jetson hardware. Its caveat says results vary with hardware, software and configuration.

The accuracy chart is scored on an RTX 5090. NVIDIA lists two cloud references on its quality-versus-latency chart: Gemini 3.8 Flash at 77.1 with 2,449.5 ms time to first token, and Claude Sonnet 5.5 at 76.2 with 3,002.2 ms. I could not read the chart itself, only its caption.

How it was trained

NVIDIA says it used reinforcement learning with verifiable rewards and from human feedback, with NeMo-RL and NeMo-Gym, across tool calling, instruction following, instruction hierarchy, structured outputs and free-form formatting. It kept a 2 to 1 ratio of non-reasoning to reasoning samples per batch, added length penalties to control verbosity, then quantized to Q4_K_M with llama.cpp. The release includes a DFlash parallel drafter that NVIDIA says gives up to 21% higher throughput than Gemma 4 E4B's original MTP drafter.

What I would test

These are my own untested suggestions.

1. Run it on your target device, with your real prompt and tool lengths. The Jetson numbers differ a lot from desktop RTX.

2. Compare it with Qwen 3.5 4B on your tool calls. NVIDIA's own table has Qwen ahead on several tool and long-context tests.

3. Test the reasoning switch. Measure time to first token and answer quality with reasoning on, off and capped.

4. Check refusals to call tools. NVIDIA highlights When2Call-MT, which tests when to call, clarify or abstain. Test that with your own cases.

5. Try interruptions and long chats. Check persona drift and memory over a long session, not only one turn.

6. Check the licence on the model card before shipping in a product. I did not read it.

What I would not conclude

• That it is the best 4B-class model. The table has it behind Qwen 3.5 4B on several tests.
• That it will feel real time on small Jetson boards. NVIDIA's own figures say otherwise for first-token time there.
• That the 80% fewer tokens figure applies without reasoning, or on your prompts.
• That cloud-model parity holds beyond the specific average NVIDIA charted.

Frequently asked questions

What is Nemotron 3.5 Bolt E4B?

An open small language model from NVIDIA for local conversational agents, built on Gemma 4 E4B with Nemotron post-training.

How much memory does it need?

NVIDIA reports 3.1 to 5.0 GB of memory in its Q4_K_M tests across the listed hardware.

What context length does it have?

64k tokens, per NVIDIA.

Which languages does it support?

NVIDIA says 35 languages.

Does it beat Qwen 3.5 4B?

On NVIDIA's average, yes by its claim, but Qwen scores higher on Tau2-Bench, BFCL-v3, LongBench v2 and RULER in the published table.

Can I turn reasoning off?

Yes. NVIDIA lists configurable reasoning that can be disabled or capped.

What is DFlash?

A parallel drafter shipped with the release. NVIDIA says it gives up to 21% higher throughput than Gemma 4 E4B's original drafter.

Sources

• NVIDIA Nemotron, Nemotron 3.5 Bolt E4B 4GB (October 23, 2026): features, benchmark tables, hardware figures and training notes. The charts, model card and technical report were not read.