Building a sandboxed AI agent that runs entirely on my laptop

I set myself a slightly daft challenge recently: could I build a small AI agent, and run the entire development lifecycle for it - model, sandbox, tests and all - on my own laptop, for free, without sending a single token to the cloud?

My laptop is not a particularly LLM-friendly machine. It has a 4GB GPU, which by today’s standards is small. That was sort of the point though. I wanted to see what it takes to build a working agent on ordinary hardware, when you can’t just throw a big frontier model at the problem.

The result is a little data-analysis agent. You give it a CSV and ask a question in plain English, and it answers you:

You: which region had the highest average order value?
Agent: South, with an average of about $198.88.

The interesting bit is what goes on behind the scenes. The agent doesn’t know the answer and it never does any arithmetic itself. It writes a few lines of pandas, runs that code inside a real, throwaway virtual machine, reads the actual output, and tells you what it found. In this post I’ll go through how I wired it all together, the design decision that made a small model good enough, and the rabbit hole that ate most of my time.

The idea

The thing that kicked this off was reading about Eve, Vercel’s agent framework. Eve plugs neatly into serverless functions, which means an Eve agent doesn’t have to be a long-running process - it can spin up just to handle a request and then disappear. Run it on Vercel and you get that ephemeral, just-in-time execution out of the box.

I wanted to see if I could imitate that locally. Could I get a low-footprint sandbox that spins up just in time to run a snippet of code and then tears straight back down again, all on my own machine? This led me to Firecracker, the microVM technology that powers AWS Lambda, and to wiring it in as the place my agent’s code actually runs.

Here’s how it all fits together. There are three moving parts, all running on the one machine:

┌──────────── My laptop (Linux + KVM) ───────────────┐
│  Ollama (local LLM)  ◀── inference                 │
│        ▲                                           │
│        │                                           │
│  Eve runtime  ── the agent loop, tools, evals      │
│        │  (my custom sandbox adapter)              │
│        ▼                                           │
│  ForgeVM ──▶ Firecracker microVM                   │
│             (runs the agent's pandas, then dies)   │
└────────────────────────────────────────────────────┘
  • Eve runs the agent loop - what it calls a “turn” - in a host process. It ships with built-in tools (bash, write_file, read_file and friends), and I added one of my own.
  • Ollama serves a local, OpenAI-compatible model, so inference never leaves the laptop.
  • ForgeVM is a small Go daemon that drives Firecracker microVMs over a local REST API. This is where the agent’s code executes.

The design choice that makes a small model good enough

Before getting in to any of the agent plumbing, I made one architectural decision which turned out to be what made the whole thing viable on modest hardware.

The model never does the calculating. Its job is only to write a few lines of pandas. The microVM does the actual number-crunching, and the model just reads back whatever the code printed. So the model never has to be good at arithmetic, or at holding a dataset in its head - it only has to be good enough at writing a short, correct snippet of pandas.

This has two big benefits. Firstly, a small, cheap, local model becomes good enough, because the hard part (the computation) is handled by a regular Python process, not by the LLM’s fuzzy reasoning. Second - and this turned out to be lovely for testing - pandas gives you an exact answer. “South” is “South”. So I could test the agent precisely, asserting on the actual answer, rather than fuzzily grading whether its prose sounded about right.

The centrepiece: a custom Firecracker sandbox

Eve’s sandbox is a pluggable backend. The default runs your tools somewhere sensible, but the framework lets you swap in your own. This was exactly what I needed, and to be fair it was the bit I was least sure Eve would support. Happily it does.

A sandbox backend in Eve is essentially a contract: implement a handful of methods - run a command, read a file, write a file, remove a path - and Eve routes its built-in tools through them. So I wrote an adapter that maps that contract onto ForgeVM’s SDK. The mapping turned out to be pretty direct, which suggests Eve’s abstraction is in the right place:

export function createForgevmSession(sandbox: ForgevmSandboxLike): SandboxSession {
  return {
    id: sandbox.id,
    resolvePath: (path) => resolveWorkspacePath(path),

    async run(options) {
      const result = await sandbox.exec(options.command, {
        workingDirectory: options.workingDirectory ?? WORKSPACE,
        env: options.env,
      });
      return { exitCode: result.exit_code, stdout: result.stdout, stderr: result.stderr };
    },

    async writeTextFile(options) {
      await sandbox.writeFile(resolveWorkspacePath(options.path), options.content);
    },

    async readTextFile(options) {
      const text = await readOrNull(sandbox, resolveWorkspacePath(options.path));
      return text === null ? null : sliceLines(text, options.startLine, options.endLine);
    },

    // ...remove, spawn-a-process, and a few other methods in the same vein
  };
}

Once that adapter is wired in, the agent’s built-in bash tool runs its commands inside a Firecracker microVM, without the agent loop knowing anything about it. Ask it to uname -r and you get the Firecracker guest kernel back. The model is writing code on my laptop; the code is executing in a hardware-isolated VM that spins up on demand and is destroyed when it’s done. This was the bit I’d set out to prove, so it was really satisfying to see it working.

Then I met the small model

This is the bit that ate most of my time.

This was the first time I’d really tried to build something agentic on top of a small, less-capable model, and I was surprised by how much work it took to make it behave. I’d assumed the hard part would be the Firecracker plumbing, but that turned out to be the easy bit by comparison. The hard part was getting a 7B model to reliably call the tool at all instead of cheerfully making up an answer.

Here are a few things I learned the hard way:

Framing beats wording. I spent ages fiddling with the exact phrasing of the system prompt - politer, firmer, more emphatic - and it was almost all noise. What actually mattered was the shape of the instruction. When I told the model to do a two-step dance (“first generate a query, then call the tool to run it”), it would skip the tool and answer from memory. When I reframed it as a single action (“make one tool call whose code computes the answer”), the same model went from ignoring the tool to using it every single time. Same model, same task, just framed differently.

Measure properly. The model’s behaviour was bimodal: it either nailed it every time or failed every time, depending on the prompt. A single test run tells you almost nothing. I started running batches of at least eight, and tracking two separate things: did it call the tool? and was the answer right? Those two move independently, and lumping them together hid what was actually going on.

Never show a tool-call envelope in the prompt, not even as an example. A small model will happily imitate it as literal text rather than understanding it as a thing to do. Show it what a tool call looks like and it’ll print one as a string instead of making one.

One more fix was a bit more pragmatic. Even once I’d got the model reliably making its first tool call, it wouldn’t reliably chain a second one. The natural flow - write a script to a file, then run it - is two tool calls, and the model would do the first and then hallucinate the result instead of running anything. So I collapsed the two steps into a single tool, run_python, that writes the script into the microVM and runs it in one call:

async execute({ code }, ctx) {
  const sandbox = await ctx.getSandbox();
  const script = `${PREAMBLE}\n${code}`;       // PREAMBLE loads every CSV into a DataFrame
  await sandbox.writeTextFile({ path: SCRIPT_PATH, content: script });
  const { exitCode, stdout, stderr } = await sandbox.run({
    command: `python3 ${SCRIPT_PATH}`,
  });
  return { exitCode, stdout, stderr };
}

The write-then-run still happens inside the VM exactly as before. I just stopped relying on the model to chain it, and orchestrated it deterministically instead. Taking away the step the model kept failing at made the whole thing much more reliable.

I tried a few different models along the way too. llama3.1:8b made a mess of its tool calls (one malformed call actually crashed the runtime). The smaller qwen2.5:3b was quick - it fits entirely on the 4GB GPU - but kept skipping the tool to answer from memory. I eventually settled on qwen2.5:7b-instruct, wrapped in a tiny Ollama Modelfile with two changes that mattered:

FROM qwen2.5:7b-instruct
PARAMETER temperature 0
PARAMETER num_ctx 16384

temperature 0 makes it deterministic and stops it randomly wandering off the tool. A 16k context window (above Ollama’s small default) noticeably helped its tool-calling, without the heavy memory cost of going all the way to 32k on a small GPU. That combination took my data-analysis eval from passing maybe three times in ten to around eight in ten. So the tuning helped a lot, but a 7B model on a 4GB GPU is never going to be rock solid.

My main takeaway: you can get a lot more reliability out of a small local model through how you frame things, and by measuring carefully, before you need to reach for your credit card and a bigger model.

A word on building this with a coding agent

I built most of this with Claude Code, and there’s a lesson in here for me which I think is worth sharing.

Because this was my first time authoring a system prompt and wiring up an agent, I leaned on the coding agent to know what good looked like. It frequently didn’t. The biggest time-sink was that it didn’t go and read the relevant docs on its own - not Eve’s, and not the guidance on getting a model to make multiple tool calls. So I lost hours to approaches which sounded plausible, and were stated very confidently, but were wrong. Eventually I went and read the docs myself and made a few common-sense calls that worked far better.

This isn’t a case of coding agents being bad, more that it’s still very easy to head off in the wrong direction if you don’t apply enough rigour to what your agent gives you. If you want it grounded in the docs then tell it so explicitly, up front, and don’t assume it will. Next time I’ll be trusting my own judgement a lot more.

Testing something that’s allowed to be wrong

I care a lot about testing, and working with AI only makes me care more. A conversational product that says something untoward has a real-world effect on a real person, so building confidence that an agent behaves itself matters even more here than in ordinary software.

Eve has evals built in, and they’re nicely idiomatic: you write *.eval.ts files, send the agent a message, and assert on which tools it called and what came back. Because my design produces an exact answer, I could assert on it precisely. Multi-turn conversations are just as easy - you send follow-up messages in sequence. Here’s a two-turn eval where the second question only makes sense if the agent remembered its own first answer:

export default defineEval({
  description: "Find the biggest portfolio, then its highest-paid employee.",
  async test(t) {
    await t.send("Which portfolio has the highest number of employees?");
    t.calledTool("run_python");
    t.check(t.reply, includes("Nakatomi Growth"));

    await t.send("And which employee in that portfolio has the highest salary?");
    t.check(t.reply, includes("Aisha Adeyemi"));
  },
});

A couple of caveats though. With a flaky local model, an eval is more of a pass-rate probe than a hard pass/fail gate - so the real correctness guarantees in this project live in fast, deterministic integration tests that run pandas in a real microVM and assert on “South”. And as soon as you build anything non-trivial, you really do want to run all the evals all the time; a bit of manual poking at the chat window does not give you the same confidence. (There was also a fun moment where the eval runner never disposed of its sandbox sessions, so microVMs quietly piled up until I wrote an auto-reaper. Pre-1.0 frameworks, eh.)

Did it work?

Yes, and I’m really pleased with it. A few evenings’ work produced a small but real agent that runs end to end on a fairly ordinary laptop: a real local model, a real hardware-isolated sandbox, real code execution, and tests around it, all without needing a cloud account or an API key, or spending a penny.

The Firecracker side was my favourite bit. It was surprisingly easy to get microVMs spinning up on demand to run a snippet and then disappear, all locally, while imitating how this might work on an actual cloud provider. I should point out that I didn’t do any soak or performance testing, and I didn’t dig deeply into VM housekeeping - whether they all reliably self-terminate under load, that sort of thing. Still, it was really interesting, and it’s great that running a proper microVM sandbox is now this easy.

One caveat on my verdict overall: I’m a relative noob in this specific area. I haven’t built much with the other agent frameworks out there, so I can’t offer you a comparison, just my experience of one spike.

Next time, I’d love to push further into Eve’s “channels” and wire the whole thing into Slack, so you could just ask it a question from a chat message. Maybe that’s the next spike.

Try it yourself

The whole thing is open source, including the Firecracker sandbox backend and the tuned model setup:

github.com/eddgrant/vercel-eve-firecracker-agent

The README walks through getting it running locally on a Linux machine with KVM - clone, bootstrap the microVM runtime, start the local model, and chat to it. If you give it a go, or if you’ve been trying to make a small local model behave for your own agent, I’d love to hear how you got on. Please leave a comment or reaction below.

Cheers!

Edd

Edd Grant

Enjoyed this build log? If it nudged you to try Eve, Firecracker, or running a model locally, you can buy me a coffee. It’s about the price of, well, a coffee, takes 20 seconds, and needs no account. It genuinely makes my day.

A handful of lovely readers have already bought me a coffee — thank you, each one's a real treat. ☕