NVIDIA Nemotron Bolt E4B: A 4GB AI Model That Runs Locally

Direct answer:

Direct answer: NVIDIA has released Nemotron 3.5 Bolt E4B 4GB, a small open language model designed to power voice assistants, game characters and robots that run entirely on local hardware. Built from Google’s Gemma 4 E4B and trained with reinforcement learning alone, it packs conversational and tool-using ability into roughly 4GB of video memory, so it can live on a graphics card, a Jetson module or a Spark device you already own — no cloud API required for every conversation.

What happened

NVIDIA announced Bolt E4B on 23 October 2026, via its NVIDIA AI account and a technical report on its research site. The launch post put the pitch in one sentence: “Our newest small open model is built for local assistants, interactive characters and robots that can carry on a conversation and use tools to get things done.” The accompanying demo showed a Reachy Mini robot chatting and moving, with Bolt handling conversation and NVIDIA Cosmos handling vision, both models running locally on a Jetson Orin Nano with 8GB of memory.

That demo is the story in miniature. For years, the most capable conversational AI lived in data centres: you sent audio or text to a cloud service, waited for a response, and paid per token. Bolt E4B is part of a deliberate push in the opposite direction — small, open models that are good enough for a specific job and cheap enough to run where the user actually is. NVIDIA calls the family Nemotron 3.5, and Bolt E4B is its smallest, most local member: the model you put in a kiosk, a robot, a shop assistant or an always-on voice device, rather than the model you put in a data centre.

The technical facts, from NVIDIA’s own research page and model card, are as follows. Bolt E4B builds on Gemma 4 E4B, Google’s open model, with Nemotron post-training applied on top. It has 8 billion total parameters and 4 billion effective parameters — the “E” in E4B stands for effective, a reference to the model’s mixture-of-experts style design in which only part of the model is active for any given token. It supports a 64,000-token context window, enough to hold a long conversation, instructions, tool definitions and retrieved information at the same time. It supports 35 languages in this release, building on a base model pre-trained on more than 140 languages.

Three capabilities define what Bolt E4B is for. The first is multi-turn tool use: the model can call application programming interfaces (APIs) to carry out tasks across a conversation, with integration guidance for asynchronous tool calls — the kind of behaviour you need when a voice assistant has to check stock, book an appointment or look up an order, not just answer a question. The second is a conversational mode that can be toggled to encourage concise, speech-ready responses, because a model that writes a five-paragraph essay is a terrible voice assistant. The third is configurable reasoning: you can enable reasoning for complex requests or disable it for faster responses, and cap thinking time to meet latency targets. That trade-off — think longer for hard questions, answer instantly for easy ones — is exactly what a real-time voice product needs.

The training method is also notable. NVIDIA says Bolt E4B was post-trained exclusively from Gemma 4 E4B using reinforcement learning, rather than supervised fine-tuning. In plain terms, instead of training the model mainly by showing it examples of good answers to copy, NVIDIA trained it by rewarding correct, useful behaviour and penalising poor behaviour across many tasks. Reinforcement learning has become the standard way to sharpen tool use, reasoning and instruction-following in modern models, and NVIDIA’s claim is that an RL-only recipe produced a stronger conversational agent than the usual fine-tuning route. Independent researchers will want to see more detail on the recipe before accepting that claim in full, but the direction of travel is clear: small models are no longer just distilled copies of big ones. They are trained, deliberately, for agent behaviour.

Performance claims need the usual care. NVIDIA reports that Bolt E4B offers roughly 40 percent higher average quality against its prior model on key conversational agent benchmarks, and outputs up to roughly 80 percent fewer tokens than peer models — fewer tokens means shorter, more direct answers, which matters enormously for voice, where every extra sentence costs time and patience. Across 21 evaluations, NVIDIA says Bolt outperformed Gemma 4 E4B, Qwen3.5-4B and Nemotron 3 Nano 4B on 15 tests. Its standout result is IHEval at 84.8, against 67.0 for Gemma 4 E4B and 60.6 for Qwen3.5-4B. On the other side of the ledger, Bolt trailed Qwen3.5-4B on Tau2-Bench, a tool-use benchmark, scoring in the mid-50s against Qwen3.5-4B’s 75.1 in NVIDIA’s reporting. Different sources report slightly different Tau2 figures depending on quantisation and test configuration, so treat the exact decimal as less important than the pattern: Bolt is very strong on conversational quality, and less dominant on some structured tool-use tasks. That pattern should shape how you use it — brilliant front-of-house conversationalist, with careful testing before you trust it with complex multi-step transactions.

Speed is the other headline. Using speculative decoding with a DFlash drafter on an RTX 5070, NVIDIA reports generation speeds reaching 272.6 tokens per second. Speculative decoding is a technique in which a small, fast draft model proposes text and the main model checks it, producing fluent output much faster than generating token by token. For a voice assistant, that speed is the difference between a conversation that feels alive and one that feels like waiting for a web page to load. NVIDIA’s own tests put the Q4_K_M build’s video memory use at 3.1 to 4.3GB on RTX hardware, which is the basis for the “4GB” name: the model is designed to leave headroom for speech recognition, vision and application code running alongside it on the same device.

Distribution is deliberately open. Bolt E4B is licensed under OpenMDW-1.1, which allows commercial use, and the weights, data and recipes are open. You can download it from Hugging Face, and run it with Ollama, LM Studio, Unsloth, llama.cpp or NVIDIA’s own NIM and In-Game Inferencing SDK, with Jetson AI Lab support for embedded deployment. That openness is a genuine strategic choice. A closed API gives NVIDIA control and recurring revenue; an open small model gives NVIDIA ubiquity on its own hardware. Every Bolt deployment on an RTX card or Jetson module is another reason to buy NVIDIA hardware next time, and another developer building on NVIDIA’s stack. Small businesses benefit from that strategy, because the model itself is free — but they should understand the bargain: free software, on hardware that NVIDIA sells.

How local conversational AI actually works

It is worth slowing down here, because “runs locally” is often misunderstood, and the misunderstanding leads to bad buying decisions. A local model does not mean a model that lives in your head, or that needs no computer at all. It means the model runs on hardware you control — a PC with a decent graphics card, a Jetson module inside a device, a Spark laptop — rather than on a server owned by a model provider.

A typical local voice assistant built on Bolt E4B has several pieces. A microphone captures speech. A speech-recognition model turns that speech into text. Bolt E4B reads the text, decides what to say and whether to call a tool — for example, a stock-check API or a booking system. A text-to-speech model turns its reply into audio. If the device has a camera, a vision model such as Cosmos interprets what it sees. All of that happens on the device, in milliseconds, without sending your customer’s voice to a cloud server.

That architecture has three real advantages. The first is privacy and data control: conversations do not leave your premises unless you choose to send them somewhere. For a solicitor’s reception, a clinic, a school or any business handling sensitive conversations, that is a meaningful difference, and it simplifies (though it does not eliminate) data protection obligations. The second is resilience: the assistant keeps working if the internet goes down, which matters for a shop floor, a warehouse or a rural business with unreliable connectivity. The third is cost predictability: once you own the hardware, there is no per-conversation API bill that scales with success. A popular voice assistant that answers ten thousand questions a day costs the same to run as one that answers ten.

There are also real costs, and they are the reason most small businesses still use cloud APIs. You own the hardware, so you own failures, updates and security patches. You own safety: if the model says something wrong, offensive or dangerous, there is no provider filter standing between it and your customer unless you build one. You own performance tuning: a local model that is brilliant on benchmarks can still be mediocre on your customers’ actual accents, vocabulary and questions until you test it and adapt it. And you own the limits: a 4GB model, however good, is not a frontier model. It will not match GPT-class or Claude-class systems on hard reasoning, obscure knowledge or long, complex tasks. The honest question is not “is Bolt better than the cloud?” but “is Bolt good enough for this specific job, on this specific hardware, for these specific users?”

The Reachy Mini demo illustrates both sides. A small robot that chats, looks at the person in front of it, moves its head and antennas, and responds in real time is a compelling product — and it works because Bolt handles conversation while Cosmos handles vision, both locally, on a modest module. But a demo is a controlled environment: good lighting, clear speech, a scripted set of interactions. A shop, a classroom or a care home is messier. Before committing, run the model where your customers actually are, with their noise, their accents and their awkward questions, and measure what matters: task completion, not benchmark scores.

Comparisons: where Bolt E4B sits

The small open model field is crowded, and Bolt E4B’s position becomes clearer when you compare it with its named rivals.

Against its own base, Gemma 4 E4B, Bolt is a clear step up on NVIDIA’s conversational benchmarks — IHEval 84.8 against 67.0 is a large gap, and it is the result NVIDIA most wants you to remember. That gap is the payoff from Nemotron post-training: same starting point, much stronger conversational agent after RL. If you already run Gemma-based tools, Bolt is a plausible drop-in upgrade worth testing.

Against Qwen3.5-4B, the picture is mixed, and more interesting. Qwen’s models have a strong reputation for tool use and structured tasks, and Bolt’s weaker Tau2-Bench result suggests Qwen remains the safer choice for complex, multi-step tool workflows where a wrong API call has real consequences — a payment, a booking, a stock adjustment. Bolt’s strength is conversation: natural, concise, speech-ready responses. So the choice is not “which model is better?” but “what does this product do most?” A voice receptionist that mostly answers questions and routes callers? Bolt looks strong. An agent that must execute a ten-step procurement workflow without error? Test Qwen, and test Bolt, and measure error rates on your own workflows before choosing.

Against Nemotron 3 Nano 4B, NVIDIA’s own prior small model, Bolt is positioned as the successor for conversational agents: roughly 40 percent higher average quality on key benchmarks, fewer tokens per answer, and a design tuned for local deployment rather than general use. If you evaluated Nano and found it close but not quite good enough for voice, Bolt is the model to re-test.

Against the large open-weight models in a different class — Moonshot’s Kimi K3.1 Pro, for example, a 3-trillion-parameter-class model with a million-token context — comparison is almost meaningless. Kimi is a frontier-class system for coding, research and complex multi-step work, running on serious infrastructure. Bolt is a local agent for a single device. They answer different questions. The fact that both are “open” does not make them substitutes, any more than a bicycle and a lorry are substitutes because both have wheels.

Against cloud APIs, the trade-off is the one described above: capability and convenience against privacy, resilience and predictable cost. Many small businesses will end up with a hybrid: Bolt-class local models for routine, sensitive or latency-critical conversations, and cloud frontier models for hard cases, escalations and tasks the local model cannot handle. That hybrid is probably the most realistic architecture for the next few years, and Bolt’s open licence makes it a cheap component to add to such a system.

What it means for UK small businesses

Here is where Bolt E4B stops being a chip-industry story and starts being a practical option for ordinary businesses. Work through the use cases in order of realism.

The most immediate use is a local voice assistant for customer-facing roles. A small hotel, dental practice, veterinary clinic, garage or trades firm could run a Bolt-powered reception assistant on a PC or Jetson device: answering opening hours, explaining services, taking messages, checking availability, routing urgent calls to a human. Because the model runs locally, the business is not paying per call, and customer voice data does not leave the building. The pilot for this is cheap — a mid-range RTX card, a microphone, and a week of testing on real calls — but the testing matters more than the hardware. Record (with consent) the questions customers actually ask, measure how often the assistant completes the task without human help, and set a clear escalation rule: anything involving money, medical advice, complaints or vulnerability goes to a human, every time.

The second use is in-store and on-site assistance: kiosks, information points, museum guides, showroom assistants, warehouse helpers. A Bolt device can answer product questions, explain a process, translate a phrase, or guide a visitor, without a cloud connection and without a monthly subscription per device. For a business with several sites, the economics are attractive: buy the hardware once, deploy the same model everywhere, update it centrally. The catch is maintenance. Somebody has to own updates, monitor failures and refresh content when products, prices or opening hours change. A kiosk with last year’s prices is worse than no kiosk, because it answers confidently and wrongly.

The third use is for developers and agencies serving small clients. If you build chatbots, voice products or automation for small businesses, Bolt gives you a new option to offer: a private, on-premise assistant with no per-token cost and no data leaving the client’s premises. That is a genuine selling point for clients in legal, health, education or finance, where data control is a procurement requirement rather than a preference. It also changes your business model: instead of reselling API usage, you sell hardware, setup, integration and support. Price that support honestly. The model is free; your expertise in making it work safely is not, and it should not be.

The fourth use is education, training and prototyping. A college, training provider or internal learning team can run Bolt on ordinary lab hardware to teach conversational AI, agent design and tool use without API keys, billing accounts or data leaving the classroom. Students can break things safely, inspect behaviour, and learn the difference between a benchmark score and a real conversation. For a small business training its own staff in AI, that is a low-risk way to build practical skills.

There are also use cases to approach with caution. Do not use a local model as a sole source of medical, legal or financial advice, any more than you would use a cloud chatbot that way. Do not deploy it for vulnerable users — children, elderly customers, people in distress — without human oversight and clear safeguarding. Do not assume “local” means “compliant”: you still need a data protection impact assessment where personal data is involved, you still need to tell customers they are talking to an AI, and you still need retention and deletion policies for any conversation logs you keep. Local processing helps with GDPR, because data minimisation and purpose limitation become easier to demonstrate, but it does not exempt you from the regulation.

Cost deserves a frank paragraph. The model is free, but the system is not. A suitable RTX card, a Jetson module or a Spark device costs real money, and a device that runs Bolt alongside speech and vision models needs more than the bare minimum. If you already own a gaming PC or a workstation with a modern RTX card, your marginal cost is near zero — try it this week. If you would need to buy hardware for every site, compare the total cost against a year of cloud API bills for your actual volume before deciding. For low-volume use, the cloud is often cheaper. For high-volume, always-on use, local wins. Do the arithmetic for your volume, not for a hypothetical.

Key facts

Honest limits / what to watch

Start with the evidence. Every performance number in this article comes from NVIDIA’s own benchmarks and model card, not from independent evaluation. That is normal for a launch — independent benchmarks take weeks — but it means the numbers are best-case, on NVIDIA’s chosen tests, in NVIDIA’s chosen configurations. Wait for independent results on conversational quality, tool use and multilingual performance before treating any figure as settled, and be especially sceptical of any single-number ranking that crowns a winner. The Tau2 result is a useful reminder: even NVIDIA’s own reporting shows Bolt behind a rival on at least one important test.

Next, the size confusion. A model called “4GB” whose download weighs 6.18GB will confuse buyers, and confusion leads to under-specced hardware. The honest reading is that 4GB is a design target for video memory in typical conversational workloads, not the file size and not a guarantee for every workload. If you add speech recognition, vision and application code, budget more memory than 4GB, and test on the exact hardware you intend to deploy. The Jetson Orin Nano 8GB in the demo is a better planning guide than the product name.

Third, safety is your job. An open model ships without the content filters, refusal behaviour and abuse monitoring that cloud providers operate at scale. That freedom is valuable — you can adapt the model to your domain — but it means you must build guardrails: prompt instructions, output checks, topic restrictions, logging and human escalation. Test adversarial inputs, not just friendly ones. Ask the model rude, misleading and edge-case questions before your customers do, and decide in advance what it must never do: give medical diagnoses, quote prices it cannot verify, promise bookings it cannot make, or discuss a customer’s personal data beyond what it is authorised to access.

Fourth, language and accent coverage. Support for 35 languages does not mean equal quality in all 35, and a model trained largely on standard speech may struggle with regional accents, dialects and noisy environments. If your customers speak with a strong regional accent, use domain jargon, or call from a warehouse floor, test with those voices, not with a clean studio recording. Speech quality is a system property — microphone, acoustics, recognition model and language model together — and Bolt is only one component.

Finally, watch the ecosystem, not just the model. Bolt’s value grows if Ollama, LM Studio, Jetson tooling and agent frameworks keep improving around it, and if developers publish recipes, guardrails and domain adaptations. Watch for independent benchmarks, for real deployments in retail, hospitality and healthcare, and for NVIDIA’s next small-model release: the pace here is fast, and today’s best local model is rarely still the best in six months. That is not a reason to wait forever — a pilot now teaches you more than a white paper in spring — but it is a reason to build your assistant so the model can be swapped. Keep your tools, prompts and escalation rules separate from the model itself, so upgrading is a configuration change, not a rebuild.

Bolt E4B matters because it lowers the floor for private, local conversational AI. It does not replace frontier models, and it does not remove the need for careful design, testing and human oversight. But for a well-defined job, on hardware you control, with conversations you would rather keep to yourself, it is now a credible, free option — and for many UK small businesses, that combination is new.

FAQs

What is NVIDIA Nemotron 3.5 Bolt E4B?

It is NVIDIA’s newest open small language model, built for conversational agents that run locally on a device rather than in the cloud. It targets voice assistants, interactive game characters, robot interfaces and other embedded applications.

How big is Bolt E4B, really?

It has 8 billion total parameters and 4 billion effective parameters, with a 64,000-token context window. NVIDIA says it is designed for about 4GB of VRAM in its target workloads, though the downloadable Q4_K_M GGUF file itself weighs 6.18GB.

What hardware does Bolt E4B run on?

NVIDIA lists GeForce RTX graphics cards, Jetson modules, RTX Spark and DGX Spark systems. Its launch demo ran Bolt plus a vision model locally on an 8GB Jetson Orin Nano inside a Reachy Mini robot.

Is Bolt E4B free to use commercially?

Yes, under the OpenMDW-1.1 licence, which allows commercial use. The weights are available on Hugging Face and run with tools including Ollama, LM Studio and llama.cpp.

How does Bolt E4B compare with Qwen3.5-4B?

On NVIDIA’s own benchmarks across 21 evaluations, Bolt beat Gemma 4 E4B, Qwen3.5-4B and Nemotron 3 Nano 4B on 15 tests, with a standout IHEval score of 84.8. It trailed Qwen3.5-4B on some tool-use tests such as Tau2-Bench, so it is not ahead on every measure.

Should a small business switch its chatbot to a local model?

Only after a pilot. Local models cut per-token API costs and keep conversations on your own hardware, but you take on hosting, updates, safety filtering and support. Test Bolt E4B against your current cloud chatbot on your own real conversations before deciding.

Sources: NVIDIA Research, “Introducing NVIDIA Nemotron 3.5 Bolt E4B 4GB”; NVIDIA AI launch post, 23 October 2026; MarkTechPost, “NVIDIA Releases Nemotron 3.5 Bolt E4B 4GB”; Digest AI, “NVIDIA releases Nemotron 3.5 Bolt E4B for local voice agents and robots”. Related WeblifyAI reading: Kimi K3.1 Pro, Moonshot’s open-weight model and NVIDIA and TCV back Higgsfield’s AI video funding.

CTA: Get a Free AI Consultation