Search engine comparison

SPECIFICATION.md’s “Integrate search” section lists 5 in-browser search library candidates. Pagefind is already the main integrated engine for /search/ and the homepage’s free-text box (see EVALUATION.md). This page adds a working demo of each of the other four — Lunr.js, FlexSearch, MiniSearch, Orama — each on its own page, each searching the same 13 HDC entries with the full same filter set as the homepage (Category, Readiness tier, License, Intended users, Primary purpose, Cost model, Support level, Number-adopting range, Go/Reset), so this is a genuine like-for-like comparison of how each library performs as a search engine under identical filters, not just a bare text box. Each is tested against the same bar as Pagefind: searching “Sangya” must return Cohort Discovery Service, because that word only exists in contactTeamPerson, a field no page displays.

All 4 passed that test, filters and all. Try each one:

The structural difference from Pagefind, upfront

Pagefind indexes the built HTML — it crawls dist/ after astro build and reads whatever text and data-pagefind-* attributes are on the page. None of these four libraries do that. All four index JS objects you hand them — there’s no equivalent of Pagefind’s “crawl the site” step. That single difference shapes almost everything else about integrating them:

A single shared corpus (/search-data.json, a build-time Astro endpoint reading the entries content collection) feeds all four demo pages. It deliberately includes every textual field per entry — contactTeamPerson, originatorOrgs, adoptionEvidence, and so on — not just what’s shown on any page, which is what makes the “Sangya” test meaningful for these libraries too.
None of the four need a postbuild CLI step the way Pagefind does (pagefind --site dist) — it’s npm install + import in a client <script>, and the index builds in the browser on page load. That’s a genuine ease-of-integration win over Pagefind’s build-step requirement.
The flip side: rebuilding the index from scratch in every visitor’s browser is fine at 13 entries but won’t stay fine at “100s” (the spec’s own scale target) without moving to a prebuilt, serialized index fetched once and deserialized client-side — all four support exporting/importing a serialized index, but none of these demos do that yet. That’s the natural next step before treating any of them as production-ready, not a blocker for this comparison.
The filters (and the “Go”/“Reset” buttons) aren’t reimplemented per engine — one shared component (FilterForm.astro) renders identical markup on all 5 pages from real entry data, and one shared filter-matching function (matchesFilters() in src/lib/searchDemo.ts) applies the checkbox/range state to whatever candidate set each engine’s text search produces. Only the “which document IDs match this text query” step differs page to page — which is exactly the part being compared.

Lunr.js

Verdict: easy, if you accept its search-syntax quirks.

const idx = lunr(function () {
  this.ref('id');
  this.field('name');
  this.field('description');
  this.field('body');
  for (const doc of docs) this.add(doc);
});
idx.search('Sangya*'); // -> [{ ref: 'cohort-discovery-service', ... }]

No separate types package ships with lunr itself — needed @types/lunr as a dev dependency to get TypeScript support.
idx.search('Sangya') (no wildcard) also happened to match here, but Lunr’s default query parser does exact-term matching against its (stemmed, lowercased) tokens — it does not do substring/“contains” matching by default. A trailing * (Sangya*) gets prefix matching, which is what most users expect from a search box; without it, a partial word typed mid-search returns nothing until the word is complete. Worth knowing before shipping — it reads as “broken” search to a user typing incrementally, when it’s actually working as documented.
Fully synchronous, zero runtime dependencies, smallest of the four to reason about conceptually — one function, one method.

FlexSearch

Verdict: most powerful, most configuration surface to learn.

const index = new Document({
  document: { id: 'id', index: ['name', 'description', 'body'] },
  tokenize: 'forward',
});
for (const doc of docs) index.add(doc);
index.search('Sangya', { merge: true }); // -> [{ id: 'cohort-discovery-service', field: ['body'] }]

Two distinct top-level APIs (Index for single-field, Document for multi-field) — picking the right one for “search across several fields of a structured record” takes a moment even with docs open.
Default tokenizer only matches whole tokens; needed tokenize: 'forward' to get the kind of partial-word matching a free-text box implies (similar caveat to Lunr’s wildcard, different mechanism).
Result shape without merge: true is grouped per matched field ([{ field: 'body', result: [...ids] }]) — natural if you want to know why something matched, but an extra step to flatten into “the list of matching documents” a UI actually wants. merge: true does that, but isn’t the default and isn’t obvious from the top-level docs.
Ships its own type declarations; no extra install needed there.
The library’s real strength (worker-thread/persistent-index modes for very large corpora) is irrelevant at 13 documents — most of its configuration surface is aimed at a scale this spike doesn’t test.

MiniSearch

Verdict: easiest of the four. If picking one today, this is the one.

const mini = new MiniSearch({ fields: ['name', 'description', 'body'], idField: 'id' });
mini.addAll(docs);
mini.search('Sangya', { prefix: true, fuzzy: 0.2 }); // -> [{ id: 'cohort-discovery-service', score: ... }]

One class, four methods (addAll, search, plus add/remove for incremental updates). prefix and fuzzy are named options right on the search call, not a separate tokenizer concept to learn — this is the only one of the four where “partial word, slightly misspelled” worked without reading past the README’s first example.
Ships its own types.
No API decision this spike hit that felt like it needed a second look at the docs.

Orama

Verdict: easy once you accept it’s async-first and schema-typed; the most “modern TypeScript library” feel of the four.

const db = await create({ schema: { id: 'string', name: 'string', description: 'string', body: 'string' } });
for (const doc of docs) await insert(db, doc);
const found = await search(db, { term: 'Sangya', properties: ['name', 'description', 'body'] });
// found.hits -> [{ document: { id: 'cohort-discovery-service', ... }, score }]

Every operation (create, insert, search) is async, including inserting a single document — reasonable for a library designed to support pluggable/persistent storage backends, but it means even a 13-document insert loop is for (...) await insert(...), more ceremony than the other three’s synchronous .add().
Requires an explicit schema up front (field name → type), unlike the other three which happily index whatever properties an object has. This is a plus for catching typos in field names early, a minor extra step for a quick spike.
Written in TypeScript natively — no separate types package, and the schema-typed API means autocomplete on insert()/search() calls matches the declared schema exactly.

Net read for the next SSG iteration

If the goal is “smallest amount of code to get free text + filters working over a small-to-medium entry set,” MiniSearch had the least friction. If the goal is “prepare for a much larger corpus with incremental updates and possibly a persistent index,” FlexSearch’s extra configuration surface and Orama’s async/schema-typed design are the two worth a longer look — both are built with that scale in mind in a way Lunr and MiniSearch aren’t. None of the four required anything Pagefind didn’t already prove necessary for this project (a corpus that includes every field, not just what’s displayed) — the difference is entirely in how each library wants that corpus handed to it.