Skip to content
Development documentation for unreleased code on main. Behavior can change before a release. Latest release docs

Luxir Architecturedev

Luxir is a hybrid search engine: full-text relevance, vector similarity, and faceted analytics in one native-code core, queried through one composable request tree. For a class-by-class map of the source tree, see ../dev/codebase-map.md.

Luxir starts from the economics of cloud compute. Inefficiency is a recurring cost rather than a one-time hardware purchase, and instance pricing is roughly linear up to very large machines, so one big node is now as economical a unit of capacity as a cluster of small ones. Both facts favor a single efficient process that can use an entire modern machine.

  • Built for modern hardware. Many cores, large memories, fast NVMe, and SIMD are baseline assumptions: work-stealing parallelism throughout, memory-mapped immutable data, vectorized codecs, and allocation discipline on every hot path.
  • Scales up before it scales out. One process is designed around large machines, many cores, and a memory-mapped address space, with no managed-heap ceiling or per-node coordination tax. Replication and scale-to-zero are separate future layers rather than reasons to shard first.
  • One request tree. Lexical queries, vector similarity, filters, facets, statistics, and fusion are nodes in one request tree. A result page with ranking and analytics is one request and one round trip.
  • Frugal with memory. Native code with no garbage collector, arenas and pools instead of general-purpose allocation on hot paths, and an index served from memory-mapped files.

Documents live in collections. A collection is stored as a shard holding one index; an index is a set of immutable segments plus a small metadata file naming the current commit point.

Most of what follows depends on segments being immutable:

  • Writers never modify what readers are using. A search pins a consistent snapshot for its whole lifetime; new commits swap in atomically without pausing queries.
  • Crash safety comes from the write protocol: Directory::sync makes both file bytes and names durable before publishFile atomically publishes a durable commit point. deleteDurably persists removals. After a crash the index reopens at a complete commit.
  • Compaction is concurrent merging: merges produce new segments in the background and never stall ingest or search.

All parallelism runs on a work-stealing task scheduler, which keeps every core busy even when work is skewed.

  • Search parallelizes across segments (and within segments) and across the independent operations in a request; cores that finish small segments steal work from large ones.
  • Indexing is a flow-graph pipeline: document processing, inversion, segment flushing, merging, and commit sequencing are independent stages. Ingest never waits behind a merge, and a commit does not stop the world. Independent updates are automatically parallelized. A single HTTP NDJSON connection submits multiple internal batches for concurrent indexing, with a bounded number in flight and backpressure when that limit is reached.
  • Blocking control work runs on dedicated threads. Administration, schema changes, replication transfer setup and expiry, and detailed stats run on a node-owned pool (--server.control-threads, default 2). Fsync, catalog writes, and writer drains can occupy those threads while search workers remain available. Writer drains enter the index arena through reserved seats. Bare node stats run on the connection or completion-queue thread. The default /metrics includes collection and system stats and uses the control pool.
  • Merging parallelizes inside a single merge, not just across merges: every field merges as its own task, because the segment format does not tie index structures to specific files - concurrent tasks write their own output streams with no coordination. Tasks are admitted against a node-wide indexing-RAM budget: merge memory is predictable up front from segment statistics, so heavy fields are priced exactly, scheduled largest-first, and light column merges pack around them. A merge waiting for that budget claims at most half of it from in-progress indexing, so ingest keeps building large segments instead of flushing a small one per batch to make room.

Both API surfaces multiplex connections with event loops. Search and streaming response paths keep connection handling separate from long-running engine work.

  • HTTP uses one single-runner io_context per configured connection shard, plus a dedicated accept context and thread. Successful accepts go to the shard with the fewest live connections and the connection stays pinned to that shard; an idle shard parks independently and does no per-request work for active shards. The gRPC server runs on completion queues.
  • HTTP searches with max_parallel: 0 (the default) start inline on the connection thread and adaptively publish work when credit and idle cores permit. The task group and arena entry are lazy. 1 runs serially on an arena worker to keep an expensive query off its connection thread. -1 publishes every sibling regardless of credit and load; -2 runs serially inline. The adaptive grain defaults to 50 us, with a 1000 us minimum share when only SMT siblings are idle. --search.fanout-grain-us=-1 pins 0 to serial execution. gRPC and continuations after min_commit waits start on arena workers for every value.
  • Ingest is streaming and incremental: NDJSON bodies are framed as bytes arrive and each document enters the indexing pipeline immediately - a document can be getting inverted while the request that carried it is still on the wire. The connection thread only splits records, recognizes control records, and cuts batches; JSON decoding of documents happens on the indexing workers, so one connection can keep several cores busy. Buffered JSON update bodies are likewise decoded on the indexing workers. Responses stream back the same way, a chunk at a time. For HTTP/JSON clients, there is no maximum stream size and no required user-visible bulk-request boundary.

Segments are read via mmap, and the on-disk format is the in-memory format: postings, columns, and vector index data are consumed in place, with no deserialization step and no buffer copies between the page cache and query execution. The OS page cache is the caching layer - a hot index is served from memory, a cold one faults in on demand.

  • Postings and numeric data are compressed with SIMD codecs (FastPFOR-based bit-packing, StreamVByte).
  • Document sets use Screaming bitsets, a roaring-style two-level bitmap tuned for the engine’s access patterns.
  • Hot lookup paths use branchless binary search where measurement shows it wins.
  • Relevance scoring is written so the hot loops auto-vectorize (BM25 over blocks of postings).
  • Indexing runs on rollback-capable memory pools; per-document work stays off the general-purpose allocator.
  • Requests decode into arena-backed, non-owning message objects: no per-field heap allocations on the way in, trivially destructible on the way out.

A request is a tree of named ops - top-docs, facets, statistics, and fusion - over a query tree. Top-docs can own facets and statistics over their complete match domain; facets can own per-bucket metrics, nested facets, and a ranked document list per bucket; fusion consumes ranked top-docs sources and can carry facets over the fused set. One request describes the whole page you want to render; the engine executes its named operations in parallel over the same index view.

Execution profiling follows that tree rather than wrapping the engine in a black-box timer. Instrumented operations emit per-segment strategy, selection inputs, timing, and human-readable decisions on request. Profiling is opt-in; string facets are the first instrumented operation, and other nodes can adopt the same hook without changing dispatch or response assembly.

Execution adapts to what the request actually consumes. A top-k request that does not ask for an exact total runs with block-max pruning: postings carry per-block score upper bounds, and a MaxScore-based scorer skips blocks and documents that cannot reach the current top k. An exact count requires exhaustive counting unless it is already known. Ranking can still prune when completeness is supplied separately, including field-sorted ranking whose exact total comes from metadata or a materialized domain. The choice is made per request from what the request asks for; there is no separate accuracy setting.

Vector search composes the same way. A kNN query is just a query, with a scorer that participates in the boolean tree like any term. Filters apply inside the vector search rather than as a post-pass over a fixed unfiltered top-k, and hybrid ranking combines lexical and vector evidence in a single request.

Filters are cached per collection as document sets. Eviction ranks recent charged and credited work per effective residency byte. Slot ledgers retain that work across production and eviction; whole-reader values are ranked and evicted as one unit. A lazy refresh under pressure rolls a common activity interval of at least 4,096 cache epochs. Epochs count cache activity, including uses constructed during planning, rather than requests or wall time. Ties evict older publications. Payload sweeps free only enough to fit the budget; metadata has a separate recency policy and low watermark. The two bounded admission histories still track sightings. Cached sets are consumed in bulk windows, avoiding repeated postings decoding.

Vectors are stored as ordinary columns. Exact kNN scans the column; approximate search uses per-segment ANN indexes (IVF with PQ codes by default, or opt-in RaBitQ, chosen per field) built as segment overlays, which follow the segment lifecycle and are served zero-copy from memory-mapped files like everything else. Approximate candidates are rescored at full precision. Because the ANN index is part of the segment, there is no separate vector store to keep in sync with your documents: deletes, updates, and merges apply to vectors the same way they apply to text.

See vector-search.md for the overlay internals and ../guide/vector-search.md for the user contract.

The engine speaks gRPC and JSON over HTTP. Both share the protobuf message vocabulary and semantic model, while HTTP applies its own public JSON dialect designed to be written by hand. Requests are structured; wherever a request takes a query, the query can also be written in the Luxir query language, and both forms build the same tree. Unknown keys are reported with positions, so a typo is an error rather than silently ignored, and ?explain=request echoes the canonical structured form of whatever was sent. Ingest streams over gRPC or HTTP (NDJSON).