Local AI Models Are One Answer to Where Your Code Goes. Not the Only One.

On 7 October, Microsoft and GitHub announced that GitHub Copilot will run some coding work on the developer’s own laptop. Not every task. Some of them.

That is a real step. It is also easy to read more into it than it says. Microsoft’s own article is careful, and the most useful sentence in it is a short one: local inference does not make the session offline.

What was announced

There are three parts.

First, Copilot will choose for each task whether a model on the laptop or a model in the cloud should handle it. Microsoft calls this Auto. It is coming by the end of October, on Windows PCs with NVIDIA’s RTX Spark chip, such as the Surface Laptop Ultra. The local model is Microsoft’s own MAI Code 1.1 Flash, compressed to 53 GB, with a peak memory use of 75.5 GB at its longest context. The machine it runs on has up to 128 GB of shared memory.

Second, developers can pick a local model themselves instead of letting Copilot decide. Either Microsoft’s model, or any local server that speaks the common OpenAI-style interface.

Third, Copilot can run the commands an agent starts inside a sandbox, a boundary that limits which files, networks and credentials those commands can reach.

It is not a Windows story

The headline says Windows. Most of the substance works elsewhere.

The sandbox uses each operating system’s own controls: native containers on Windows, Seatbelt on macOS, bubblewrap on Linux. No virtual machine is needed on any of the three. The local-model route works anywhere a local server runs, which on a Mac or a Linux machine usually means Ollama or LM Studio. It covers the Copilot command line, the Copilot app and VS Code.

What is Windows-only, for now, is Microsoft’s own model and the automatic choice between local and cloud. So a team on Macs has most of this today. It just picks its own local model, and it decides by hand which work stays on the device.

Local is not offline

With Auto, Copilot decides where each step of a task runs. Some steps run on the laptop. Others go to the cloud. That is the point of the feature: the cloud model is larger, and some work needs it.

For many teams that is fine. For a client whose code must not leave the EU, “some of it stays local” is not an answer. The answer there is explicit selection: one named local model, for the whole session, and a way to check that it held. Ask any supplier which mode they use, and who decides.

The sandbox is the bigger news

Moving the model onto the laptop changes where the thinking happens. It does not change what the agent can do. Microsoft says it plainly: an agent’s shell commands normally inherit the access of the account running them, and moving inference onto the device does not change that.

The sandbox does. It applies one policy to the commands and local tool servers an agent starts, whatever model asked for them. That protects you with a local model and with a cloud model alike, which is why we think it matters more than the model.

Microsoft is also clear about its limits. Copilot’s own file tools are checked against the policy inside Copilot, not enforced by the operating system. Remote tool servers sit outside the local sandbox. Treat it as a seatbelt, not a vault.

Read the benchmarks as you would read any vendor’s

The article includes benchmark scores. They were measured by Microsoft on 5 October and published on 7 October. On one test of 89 tasks, the compressed local model scores higher than the full cloud model. The comparison model is a community-compressed build of another vendor’s open model.

None of that makes the numbers wrong. It makes them a vendor’s own first measurement. Before you build a workflow on a local model, run it on your own repositories, with your own tasks.

Where we stand

We made our own choice on this question earlier. Our AI-assisted engineering tooling runs on infrastructure in Germany, so client code stays in the EU when a developer asks a model for help. We explained why in our piece on where your data lives.

A local model is another valid answer. It removes the network from the picture. It also needs a high-memory laptop per developer, and it limits you to the models that fit on one. Regional hosting keeps the model choice open and needs no new hardware. Both are better than a default nobody has examined.

What we would do first

If you are deciding this for your own team, start with one question and one test. The question is which of your projects must never send code to a cloud model, and whether your tools let you force that for exactly those projects. The test is to switch the sandbox on for a week, on one real repository, and see what breaks. Both take less time than reading the benchmarks, and both tell you more. If you want to talk through the answers for your own setup, talk to us.