How Fast Is Offline Local Document Search? Real Benchmarks

Most "local search" demos run against a handful of curated files. That tells you almost nothing about how a tool behaves on the drive you actually have — the one with years of PDFs, version-stamped spreadsheets, and a Downloads folder no one has opened in months. So instead of a synthetic benchmark, we measured LocalSynapse on a real, messy working drive, and on a controlled extraction run, on the same mid-range PC. Here are the numbers, and — just as important — what we didn't measure.

The test machine

These are not server numbers. The hardware is a normal office-grade desktop, deliberately, because that is where LocalSynapse is meant to run.

Every figure below was produced on this machine. There is no GPU acceleration in any of these results — so treat the CPU numbers as a floor, not a ceiling.

Dataset 1 — a real 93 GiB working drive

This is the live index: a genuine working drive scanned in normal day-to-day use, not a hand-picked test set. The mix of formats is what makes it credible.

• Total indexed: 68,877 files / 93.0 GiB
• Content-searchable: 15,349 files
• Semantically embedded: 12,407 files → 520,610 chunks → 520,518 vectors (1024-dimension BGE-M3)

The drive also holds thousands of images, archives, and code files — exactly the kind of noise a real machine accumulates. LocalSynapse scans all of it for filename search, and reads inside the document formats for content and semantic search.

Search latency

Keyword (BM25) search is the everyday path, and it is fast even at this scale. Measured across 4,367 real searches against the live index:

The p95 of 425 ms covers the heaviest queries — broad terms that match across tens of thousands of chunks. The typical query is effectively instant.

Semantic (AI) reranking is a separate, heavier path, and we are deliberately not publishing a headline latency for it yet — see the limitations section for why the existing sample is too small to be fair.

Index footprint

A common worry about "AI search" is that the index will balloon to several times the size of your files. It doesn't. Against the 93 GiB corpus:

The entire searchable index — keyword plus half a million AI vectors — costs under 7% of the space your documents already occupy.

Dataset 2 — controlled extraction run

To get clean, repeatable speed and memory numbers, we ran a controlled pass over a fixed 5.11 GiB document set (3,468 files) from empty, twice. This isolates the read-and-extract pipeline — opening each file, pulling its text, and splitting it into chunks.

Memory peaks came from parsing the largest PDFs and slide decks; the process otherwise sat well under 4 GB. Once extraction finishes, keyword content search is immediately available.

How semantic search builds

Semantic embeddings — the part that lets you search by meaning rather than exact words — are generated by the BGE-M3 model. On a CPU, individual inference is quick (a 50-chunk batch runs in roughly 1–2.5 seconds), but LocalSynapse intentionally meters embedding work as a low-priority background task so it never competes with what you're actively doing on the machine.

The practical consequence: keyword search works within minutes of indexing, and semantic search deepens in the background over time. There is no single "time to fully embed" figure, because embedding is paced to stay out of your way rather than run flat-out — which is also why the half-million vectors on the live drive accumulated gradually rather than in one sitting.

Method & limitations

We would rather tell you what we didn't measure than imply we measured it.

• Two datasets, one machine. Footprint and search-latency numbers come from the live 93 GiB index (read directly from its database and diagnostic logs). Extraction speed, throughput, and peak RAM come from the controlled 5.11 GiB run. Both ran on the same i5-12400 described above.
• All figures are measured, not modeled. Latency and throughput were read from the application's own diagnostic log; counts and sizes were read from the live index database; RAM was sampled externally during the controlled run.
• No discrete-GPU results. The test machine has only an integrated GPU, so every number here is CPU-only. A machine with a discrete GPU should do better on embedding; we have not measured that and make no claim about the size of the gain.
• Semantic latency is not yet a headline figure. In the logged usage, AI reranking ran on only a small subset of queries (39 cases), and those samples include cold-start warm-up, which inflates the tail. A clean, controlled semantic-latency measurement is on our list before we publish a number we'd stand behind. For transparency: in those 39 cases the median was around 770 ms on CPU.
• There is no "full-build wall-clock." Because embedding is paced as a background task, the time to fully embed a corpus depends on how much you use the machine. We report extraction time (a clean, bounded number) and embedding throughput separately, rather than a single figure that would mislead in either direction.

Real drives are messy. We think a benchmark should be too — measured on the kind of machine and the kind of files you actually have, with the gaps stated plainly.

Related Posts

• Give Your AI Agent a Local Memory: MCP Server for File Search
• Everything Search Is Great — But It Can't Search Inside Files