Artificial Intelligence
Small models on the device are quietly becoming the default
Models small enough to run on a laptop or phone now handle a large share of everyday tasks. The interesting consequence is where data stops travelling.
Updated 6 min readqikfox
The short answer
Models in the range that fits comfortably on consumer hardware now handle summarising, classifying, rewriting, extracting and routing well enough that sending those tasks to a datacentre is often unnecessary. The practical effect is cost and latency for developers and, more importantly, a class of work where personal data never leaves the machine.
Key points
- Most everyday tasks are not frontier tasks and do not need frontier models.
- Local execution removes an entire category of privacy question rather than mitigating it.
- Latency and offline behaviour improve in ways users notice immediately.
- Hard reasoning, long context and current knowledge still favour larger hosted models.
What small models are now good at
The gap between small and large models has narrowed most sharply on bounded language work. Turning a page into three sentences, deciding which of six categories a message belongs to, pulling structured fields out of a receipt, cleaning up a draft: these are now routine locally.
They remain weak where the task needs broad world knowledge, multi step reasoning across a long document, or precision on facts they were never trained on. That division is stable enough to design around.
The privacy argument is structural, not promissory
When a task runs locally, there is no retention policy to read, no processor to vet, no jurisdiction to consider and no breach to disclose, because no copy was created off the device. That is a different kind of assurance from a promise about how a copy will be handled.
It also changes what is reasonable to ask a user to trust. A claim that data stayed on the machine can be checked with ordinary network inspection. A claim about what a remote service does with a copy cannot.
The shape most products are converging on
Hybrid routing has become the standard pattern. A local model handles the common path, and the request escalates to a hosted model only when the task exceeds what the local one does reliably.
- Classify and triage locally, always.
- Escalate on length, ambiguity or explicit user request.
- Tell the user when a request left the device, in plain words.
- Make escalation refusable without breaking the feature entirely.
The honest costs
Local inference spends battery, memory and disk. Model files are large, they need updating, and older hardware will fall back to slower paths. Quality varies more across devices than a hosted endpoint does, which makes support harder.
None of that argues against the approach. It argues for stating which work is local, which is not, and what the user gives up either way.
Questions and answers
Related qikfox pages
Technology
How the products are built.
Privacy at qikfox
Our approach to personal data.
Provenance and synthetic media
The other side of generative systems.
Our mission
Principles behind product decisions.
Back to The Perspective · Written and reviewed by the qikfox editorial team. Published .
Put the advice into practice
qikfox Safe Browser applies most of what this article describes by default, and works alongside qikfox Search, Antivirus and VPN.