Can the Best Open Source LLM Run on a Normal Computer? Our Test Protocol, Published Before the Results

What this article is, and what it is not

This is the published test protocol for NewsForge's local large language model comparison — the plan, the rigs, the exact prompts, the metrics, and the rules for reporting. It is not the results.

The test runs have not been logged, so this article contains no throughput figures, no memory peaks, no setup times, no quality scores, and no model recommendations. Nothing here should be read as a finding. Where a measurement belongs, this piece states what will be measured, how, and under what conditions, so that a reader can run the same procedure independently and so that the results article can be checked against a protocol that was published before the numbers existed.

If you arrived looking for a ranked answer to "which model should I install on my laptop," that answer is not on this page yet. The results article will carry a dated "last tested" line and will name specific model builds, quantization levels, and machines.

Status: protocol published; measurements not yet taken.
Disclosure: no hardware or software has been provided free of charge for this test. If that changes, the disclosure changes with it.
Update policy for the results article: re-run the full protocol when a major open-weights model ships or when a tested runner release materially changes inference performance, and date every result.

Why the best open source LLM on a leaderboard may not be the best one on your machine

What leaderboards measure, and what they leave out

Public leaderboards differ from one another in evaluation design, in the prompts and sampling settings used, in who hosts the model during evaluation, and in which exact build is evaluated. Some publish their methodology in full; some do not. Treating "leaderboard rank" as a single, uniform quantity is a mistake, and this test does not do that.

What most of them have in common is that they are not designed to answer a hardware question. A rank tells you something about output quality under the evaluator's conditions. It does not tell you how much memory a given quantized build occupies on your machine, how many tokens per second it produces there, or whether the runner you installed is using your GPU at all.

Those outcomes depend on more than the model. They depend on the quantization method and level, the runner and its version, the backend it selects, the context length and key-value cache size you configure, memory bandwidth, and the machine's thermal and power limits. Two people running "the same model" can have very different experiences.

The results article will not use vague phrasing such as "top-ranked." It will name the specific leaderboard or leaderboards consulted, give the date the standings were captured, identify the exact evaluated build, and state the eligibility cutoff used to select contenders — for example, a publication-date window and a parameter-count ceiling. Selection criteria will be published alongside the list of models actually tested, including any model that met the criteria but was excluded, with the reason.

What "normal computer" means in this test

For this test, a normal computer is one of four machines a general reader plausibly already owns:

An 8GB laptop, included as a floor case because a large share of consumer laptops still ship with 8GB of memory.
A 16GB Apple silicon MacBook, where CPU and GPU share unified memory.
A mid-range Windows laptop with integrated graphics and no discrete VRAM, running inference on system RAM.
A desktop with a consumer GPU in the 8–12GB VRAM class.

Data-center accelerators, multi-GPU workstations, and cards with more than 12GB of VRAM are outside the scope. That is a scope choice about the target reader, not a claim about what larger systems can do. High-VRAM cards are not a universal solution either: many current builds, especially at long context lengths with a large key-value cache, exceed 24GB. This test simply is not about those machines.

The hypotheses this test is designed to check

The following are stated as hypotheses, not findings. They are written down in advance so that the results can confirm, qualify, or refute them, and so that a reader can see whether we adjusted our expectations after the fact.

Memory capacity is frequently, but not always, the binding constraint on these machines. Compute throughput, memory bandwidth, backend support, context length, cache size, and sustained power limits can each bind instead. The test records enough telemetry to say which one bound in each case.
Exceeding available VRAM does not usually mean disk swapping. In most runners it means part of the model runs on the CPU using system RAM. Operating-system swap to disk is a separate and more severe regime. The results article reports full GPU offload, partial offload, CPU-only execution, and observed disk swap as four distinct regimes, with telemetry for each, rather than collapsing them into "fits" and "does not fit."
Quantization's effect on quality is an empirical question, not a given. Lower precision reduces memory use; whether it measurably reduces output quality depends on the quantization method, the model, the task, and how much of the difference is within sampling variance. The test compares matched builds and reports observed differences with their spread, rather than assuming a lower-precision build scores worse.
A smaller model that fits may outperform a larger model that does not. This is the central practical trade in local inference. The test puts both on the same machine with the same prompts and reports both, including the case where the larger model runs acceptably in a partial-offload configuration.

How the test will be run

Test hardware

Each machine will be published with CPU model, GPU and VRAM (or "integrated graphics, no dedicated VRAM"), system RAM and configuration, operating system build number, and storage type and free space at test time. Specifications will be taken from the manufacturer's published documentation and linked, rather than from secondhand summaries. Memory bandwidth will be recorded where the vendor publishes it, because on unified-memory systems it is often a better predictor of generation throughput than core count.

Machines will be tested on AC power with the operating system's standard performance profile, with the power mode named in the results table. Battery-power behavior is measured separately, as described under metrics. Ambient temperature at the start of each session will be recorded, and machines will be allowed to return to idle temperature between model swaps.

Runners and versions

Two categories of software will be tested, because readers use both: a desktop model runner with a graphical installer, and a command-line inference tool. Each will be recorded with the exact version installed on the test date, with a link to the project's own documentation and release history. Runner version is not a footnote — inference performance changes between releases, and a throughput figure without a runner version cannot be reproduced.

Model builds and quantization levels

Every contender will be identified by its exact repository or catalog identifier, not by a marketing name. "The 8B model" is not an identifier; the specific quantized build — in GGUF or the runner's native format — with its full file name and size, is.

Parameter count, context window, and any stated hardware requirements will come from the publisher's own model card, cited by direct link, and will be attributed as the publisher's claim rather than presented as our measurement.

Where a model is tested at more than one quantization level, each level appears as a separate row. Comparing a higher-precision build on one machine against a lower-precision build on another is a common way local-model comparisons mislead people, and this test avoids it by construction.

The five everyday tasks

All five prompts will be reproduced verbatim in the results article, in a copyable code block, so readers can rerun them unchanged. The task set is fixed now:

Summarize a long document. A fixed public-domain text of a stated word count, with a fixed instruction, so summarization accuracy can be checked against a source both we and the reader can read.
Draft a professional email. A fixed scenario with fixed constraints on tone and length.
Answer a factual question. A question with a verifiable answer, chosen to sit outside heavily memorized trivia. The answer key and its source will be published with the prompt.
Write and explain a short script. A small, self-contained programming task whose output can be executed and checked, plus a request for an explanation of the code. The execution environment — language, interpreter version, operating system, and the exact command used to run the output — will be published, and the same environment will be used for every model's output.
Rewrite text in a different tone. A fixed passage, rewritten to a fixed target register.

Run conditions and sampling settings

Sampling parameters — temperature, top-p, top-k, repetition penalty, and seed where the runner supports one — are held constant across models wherever the runner allows, and any model where a setting could not be matched is flagged in its row.
Context settings — configured context length and key-value cache settings — are recorded per run, because they change memory use substantially.
Warm-up policy — one discarded warm-up generation after each model load, so that first-load effects do not contaminate throughput numbers. Model load time is measured separately and reported.
Fresh context — every prompt runs from a fresh context on every model, with no conversation carried over.
Run count and aggregation — each measurement is repeated across multiple runs; the number of runs, the median, and the observed spread (minimum and maximum) are published. Any figure from a single run is labeled as a single run in the table itself.
Failed runs — a run that errors, is interrupted, or produces no output is recorded as a failure with its error text, and is not silently replaced. If a measurement required discarding runs, the count of discarded runs and the reason appear in a footnote.
Monitoring tools — the specific tools used to read memory, VRAM, swap activity, and power draw are named per platform in the results article, with versions.

Metrics captured

Total install-to-first-token time is useful experientially but mixes several things together, so it is reported both as a total and decomposed:

Download time, against a stated connection speed, so readers can scale it to their own.
Installation and configuration time, which does not depend on connection speed.
Model load time, from launching the model to readiness.
Time to first token, measured separately from throughput because the two fail differently and readers feel them differently.
Prompt-processing speed, reported separately from generation speed where the runner exposes it — this matters most on the long-document summarization task.
Sustained generation speed in tokens per second.
Peak system RAM and peak VRAM during generation, plus observed operating-system swap activity.
Thermal, power, and battery behavior on laptops: observed throttling, sustained versus initial throughput over a fixed workload, and battery drain across that workload. Subjective observations such as fan noise are reported as observations, not as measurements.

Scoring rubric for output quality

Quality is scored against a written numeric rubric applied to the fixed prompts. Each output is scored on four criteria, 0–5 each, for a 20-point total:

Instruction compliance — did the output meet the stated length, format, register, and constraints.
Factual accuracy — for the summarization task, checked against the supplied source; for the factual question, checked against the published answer key.
Absence of fabricated detail — names, figures, or events not present in the source or not verifiable, counted and deducted.
Formatting integrity — did requested structure survive, including lists, headings, and code blocks.

For the coding task, correctness is reported first as a binary: the script either runs in the published environment and produces the correct output, or it does not. Rubric scoring applies to the accompanying explanation and to code readability, and is reported separately from the binary result.

Reviewer procedure will be disclosed: how many reviewers scored each output, whether they scored independently, and the agreement between them where more than one scored the same item. Outputs will be presented to reviewers with model identifiers stripped where the output format allows it; some models produce recognizable formatting habits, so blinding is imperfect and the results article will say so rather than claim clean blinding. Scores are editorial judgments and will be labeled as such. Raw outputs will be published alongside the scores so readers can disagree with a specific score rather than with an aggregate. A second editor will review scoring before the results article is finalized.

What is out of scope, and why

Fine-tuning, multi-GPU rigs, agentic tool use, and long-context stress tests are excluded. Each is a legitimate subject, and each would change hardware requirements enough to make results incomparable to the everyday-use question this test asks. Also out of scope: image and audio modalities, and any deployment serving multiple concurrent users. The results article will restate these limits and acknowledge that findings reflect the model and runner versions available on the stated test dates.

Setup difficulty and how it will be recorded

Setup friction is part of what determines whether local inference is usable, so it is measured rather than described anecdotally. We make no claim here about how many people abandon local inference at the setup stage, or about how other outlets cover it; we have no data for either.

Graphical installers versus command-line setup

Both paths will be walked on each machine, and the difference in time and failure rate between them recorded. This protocol does not predict where each path will break. The results article reports only failures actually encountered during the test, with the exact error text, the platform, the runner version, and the fix that resolved it. Known issues documented by runner maintainers will be cited as such and clearly separated from incidents we observed ourselves. Failures we could not resolve will be reported as unresolved rather than dropped.

Download sizes and disk footprint

Per model build, the results article publishes the download size, the on-disk footprint after installation, and the free space required to complete the install without the runner failing partway.

How results will be reported

Each contender will be reported in an identical structure so rows can be compared directly.

Contender profiles

For each model: publisher, release date, parameter count, context window, and license — all taken from the publisher's model card and license file, cited by direct link. Where a publisher states minimum hardware requirements, those are quoted as the publisher's claim, with our measurements reported separately.

The results table

Results will be published as a real HTML table, machine-readable rather than an image, with a downloadable data file alongside it. Planned columns: machine, model build identifier, quantization level, offload configuration, context length, peak RAM, peak VRAM, observed swap, model load time, time to first token, prompt-processing speed, sustained tokens per second, runs averaged, observed spread, and quality score. Empty cells mean "did not load," with the failure mode stated in a footnote.

Regimes, not verdicts

Each model-and-machine pairing will be categorized by the execution regime observed — full GPU offload, partial offload with a stated layer count, CPU-only with system RAM, or a configuration that triggered operating-system swap — with the throughput and latency for each. This distinction matters more than a single "fits or doesn't" label, because partial offload can still be considerably faster than CPU-only execution while being slower than full offload.

Quantization comparisons

Where the same model is tested at more than one precision, the results article reports memory saved, throughput change, and quality-score change side by side, with the spread across runs, so readers can see whether an observed quality difference is larger than the run-to-run variance. Where it is not, the article will say the difference was within variance rather than reporting it as a quality loss.

Output quality: what will be examined

Summarization accuracy and fabricated detail

Because the source document is fixed and public, every claim in a summary can be checked against it. Fabricated details — names, figures, or events not present in the source — are counted separately from omissions, because a hallucination is a different failure from an incomplete summary.

Writing and tone rewriting

Judged on instruction compliance first: length, register, and stated constraints. Style preference is recorded as preference, not scored as accuracy.

Coding help

The script is executed in the published environment. It runs and produces the correct output, or it does not, and that binary result is reported before any judgment about style or explanation quality.

Refusals, repetition loops, and formatting failures

Every refusal of a benign prompt, every repetition loop, and every case where requested formatting collapsed will be logged with the model build and quantization level attached, so any relationship between these behaviors and quantization level can be examined rather than assumed.

Side-by-side excerpts

The results article reproduces excerpts from the same prompt across models, so readers can judge differences themselves rather than taking a score on faith.

The hosted-assistant reference point

A hosted commercial assistant will be run on the same five prompts as a dated reference, not as a contender. Hosted systems change without notice and behave differently by product mode, account tier, system prompt, and enabled tools, so the results article will name the service, the plan, the mode or model if the provider discloses it, the settings and tool state used, and the date of the transcript. It is not scored on speed, and it is labeled a reference throughout, because it is not running on the reader's machine and is not subject to the constraint this test is about.

What each class of hardware will be evaluated for

This section describes the questions each hardware tier is meant to answer. The answers, with specific model builds and quantization levels attached, belong to the results article.

8GB machines

The memory available to a model after the operating system and a browser take their share is well under 8GB, which pushes users toward smaller builds at more aggressive quantization. The test reports what loaded, in what regime, and what it was like to use, including whether the machine entered disk swap.

16GB Apple silicon

Unified memory means the GPU can address a large share of system RAM. The test measures how much memory the system actually leaves available to a model in ordinary use, and what throughput results. Any price-performance comparison against Windows or Linux machines would require dated, price-matched configurations, which this test does not attempt; the results article will therefore report measurements for these machines without ranking them against other platforms on value.

Consumer GPUs with 8–12GB VRAM

A model does not have to fit entirely in VRAM for the GPU to help. Partial layer offload can still accelerate inference relative to CPU-only execution, though generally less than full offload. The test measures full offload, partial offload at stated layer counts, and CPU-only execution as separate configurations on the same machine, so readers can see the cost of each step down.

Upgrade paths

Different upgrades affect different workloads. More memory changes which builds fit and at what context length. A GPU with more VRAM changes how much of a model can be offloaded. Faster storage primarily affects download handling, model load time, and behavior when a machine is already swapping — it is not expected to change steady-state generation speed when a model fits in memory, and the test will report whether that holds. Many laptops, including Apple silicon machines, do not permit memory or GPU upgrades after purchase, so upgrade guidance will state which tiers can be upgraded at all. Any performance-per-dollar figure will be calculated only from measured, comparable before-and-after configurations with component prices dated at test time; where we cannot measure a before-and-after, no figure will be given.

Licensing and privacy

"Open source" versus "open weights"

These are not synonyms, and the distinction is often blurred in coverage of this field. Some models are released under licenses widely recognized as open source; others release weights under custom licenses that impose conditions on commercial use, on scale of deployment, on permitted applications, or on downstream naming. There is also a live debate about whether weights released without training data and training code should be called open source at all.

The results article will state explicitly which classification standard it applies, link the exact license version that ships with the exact build tested, and quote only the operative clauses needed to support the classification. No model's license is characterized here, because doing so accurately requires reading the specific license version attached to the specific build, and because ambiguous cases warrant review before publication. Readers planning business use should read the license before installing. Nothing in this article is legal advice.

The privacy case for local inference, and its limits

Local inference has a real privacy property: a model running on your own machine can process prompts without sending them to a provider. That is the main reason privacy-minded professionals look at local models, and it is a genuine difference from a hosted assistant.

The limits deserve equal billing, and they are broader than generation-time network traffic. A runner may check for updates, fetch model catalogs, send crash reports, or collect telemetry, and whether it does is a property of that application and its settings. Operating-system services, cloud-synced folders, and backup software can move conversation logs off the device without the runner doing anything. Plugins, search integrations, and sync features that send context to an external service reintroduce exactly the exposure you were avoiding. And an unencrypted disk holding a full conversation history is a risk of its own.

The results article will document, per runner, the privacy-relevant settings available, the vendor's stated telemetry behavior with a link to its documentation, where conversation data is stored on disk, and — where feasible — observations from network monitoring during a test session, with the monitoring method described. Absence of observed traffic during one monitored session is evidence about that configuration on that date, not a general guarantee, and it will be reported that way.

Security hygiene for model files and runners

The risks here are not all the same, and treating them as one undifferentiated "supply chain" problem is unhelpful. Separating them:

Model provenance. Prefer the publisher's official repository or the runner's official catalog over reuploads and repackaged builds. Where the publisher provides checksums or signatures, verify the downloaded file against them.
File format matters. GGUF is a data format for weights and metadata; some older serialization formats can execute code when loaded, which is why several ecosystems moved away from them. Which format a build ships in is worth checking before loading it.
The runner is software. Graphical installers, install scripts, and command-line tools are executable code and carry the ordinary risks of running executables from the internet. Install from the project's official distribution channel and follow its published security guidance.
Plugins and integrations expand the surface. Extensions that fetch web pages, read local files, or call external APIs change both the security and the privacy picture, and should be evaluated separately from the model itself.

The results article will cite each runner's own security documentation for these points rather than generalizing. For broader context on how AI is changing defensive security work, see our analysis of whether AI will replace cybersecurity roles.

What the results article will have to deliver

A recommendation that a reader can act on needs four things attached to it: a named model build, a quantization level, a named machine, and the measured numbers. The results article will name a best overall fit for mainstream hardware, a best option for a low-spec machine, and a best option for readers with a discrete GPU, each with those four things. Until the runs are logged, this article names none of them, and no recommendation should be inferred from the ordering or framing of anything above.

One thing can be said without measurements, because it is a matter of what a reader is optimizing for rather than a performance claim: some readers should not run a model locally at all. If you need the strongest available reasoning, work with very long documents, or simply do not want to manage model files and runner updates, a hosted service may serve you better. Privacy and cost control point toward local inference; peak capability and zero maintenance point toward hosted.

Results in this field also age quickly. Open-weights releases arrive frequently — see our coverage of Moonshot AI's Kimi K3 launch and its wider competitive implications — and on-device arrangements shift with commercial deals, as in our report on Apple Intelligence and Alibaba's Qwen in China. Corporate disputes can also affect what ships where, as in our report on Apple's trade secret lawsuit against OpenAI. And user reaction shapes which AI features survive at all, as our coverage of Meta's withdrawal of an Instagram AI feature shows. Every result we publish will be dated, and the protocol above is the thing we intend to keep stable across re-runs so that successive tests can be compared to one another.

Run the same five prompts and send us your numbers

Reader submissions from hardware we do not own are the fastest way to make this test more useful than any set of rigs we can buy. When the five prompts are published verbatim in the results article, you will be able to run them unchanged and send us what you get.

To be comparable, a submission needs:

Hardware: CPU model, GPU model and VRAM (or "integrated"), total system RAM, storage type, and free space.
Software: operating system and build number, runner name and exact version, and power mode during the test.
Model: the full build identifier and file name, quantization level, and configured context length.
Settings: temperature, top-p, top-k, repetition penalty, and seed if used.
Measurements: model load time, time to first token, and sustained tokens per second, each from at least three runs after one discarded warm-up generation, reported as median plus minimum and maximum.
Which of the five prompts each measurement came from.

Submission terms, in plain language: send only the measurements and the model's output to our published prompts. Do not send personal data, work documents, confidential material, or anything you would not want quoted. By sending results you agree that we may publish them, in whole or in part, with your hardware and software details and a name or handle of your choosing. We cannot independently verify reader-submitted numbers, and any that we publish will be clearly labeled as reader-reported and unverified, kept separate from our own logged runs. The submission address and any form will be published with the results article so that submissions arrive against a fixed, comparable prompt set.

Subscribe if you want the completed test when it lands, and the refreshed run after the next major open-weights release.