Skip to content
Development documentation for unreleased code on main. Behavior can change before a release. Latest release docs

Documents and valuesdev

A Luxir document is a flat JSON object with field names and values. The schema defines each field’s type and how it is indexed and stored:

{
"id": "b1",
"title_t": "The Way of Kings",
"year_i": 2010,
"tags_ss": ["fantasy","epic"]
}

This page covers field templates, variants, IDs, and the values you can send and retrieve.

Field templates apply a type and other settings to fields with a matching name suffix. Luxir’s built-in templates let you start indexing without defining your own schema:

  • title_t is analyzed text;
  • category_s is an indexed exact string;
  • year_i is an integer column;
  • tags_ss is a multi-valued exact string;
  • published_dt is a date column;
  • embedding_v is a vector column.

Templates also handle fields you cannot enumerate ahead of time. An ecommerce catalog might define an _attr template for product attributes. New product classes can then introduce fields such as shoe_width_attr or socket_attr without a schema change for each new name. Define the rule once; new matching fields work automatically. See the product attributes example.

You can combine templates with explicit field definitions, choosing which fields to define individually and which to cover with a naming rule. Explicit definitions take precedence over templates. The schema guide lists the built-in suffixes and explains how to customize fields and templates.

A field can index the same input in several ways. For example, you might want to search an author’s name by individual words, but facet and sort by the whole name. Field variants let one field do both.

The built-in _name template already sets this up. Index these books with an author_name field:

POST /collections/authors/_update
{
"docs": [
{
"id": "b1",
"title_t": "Gridlinked",
"author_name": "Neal Asher"
},
{
"id": "b2",
"title_t": "The Skinner",
"author_name": "Neal Asher"
},
{
"id": "b3",
"title_t": "Snow Crash",
"author_name": "Neal Stephenson"
},
{
"id": "b4",
"title_t": "Prador Moon",
"author_name": "Neal Asher"
},
{
"id": "b5",
"title_t": "Anathem",
"author_name": "Neal Stephenson"
},
{
"id": "b6",
"title_t": "Scythe",
"author_name": "Neal Shusterman"
}
],
"commit": {}
}

Search for neal, return the first three matches by ID, and facet on the same author field:

POST /collections/authors/_search
{
"query": "author_name:neal",
"fields": ["title_t", "author_name"],
"sort": "id",
"limit": 3,
"count_limit": -1,
"ops": {
"authors": {
"field_facet": {
"field": "author_name"
}
}
}
}
{
"found": 6,
"docs": [
{
"title_t": "Gridlinked",
"author_name": "Neal Asher"
},
{
"title_t": "The Skinner",
"author_name": "Neal Asher"
},
{
"title_t": "Snow Crash",
"author_name": "Neal Stephenson"
}
],
"ops": {
"authors": {
"buckets": [
{
"val": "Neal Asher",
"count": 3
},
{
"val": "Neal Stephenson",
"count": 2
},
{
"val": "Neal Shusterman",
"count": 1
}
]
}
}
}

The word neal matches all six books. The facet groups them by the full author name: three by Neal Asher, two by Neal Stephenson, and one by Neal Shusterman. Facet counts include every match, so Shusterman appears in the facet even though his book is beyond the first three results.

The template creates an analyzed text field for word search and a string variant, author_name__s, for whole-name facets and sorting. Both operations use author_name in the request; the template chooses the appropriate representation. You only supply the name once when indexing.

Use _names for a multi-valued field, such as author_names for books with several authors. You can also define your own variants on explicitly named fields.

id is the reserved unique-ID field. String and numeric values are accepted; numeric IDs use their canonical string rendering, so 123 and "123" are the same ID.

With the default allow_dups: false, another document with the same ID replaces the old document. Replacement is whole-document replacement: fields omitted by the new version disappear. For faster indexing when incoming IDs are known to be new and unique, see allow_dups.

An ID is not required. A document with an absent or null ID can be searched, but it cannot be overwritten or deleted by ID and has no useful external identity.

_version_ is the update version of the request that indexed the document, including allow_dups and documents without an ID. It is assigned by the engine and is retrieval-only: name it explicitly in the returned fields (fl) to read it. Sorting, filtering, faceting, functions, collapse and value statistics on _version_ are not supported. Default and wildcard projections exclude it. Documents that supply it, directly or through field mapping, are rejected with a document error. Optimistic concurrency is not supported.

The outer JSON shape follows the schema:

{
"tags_ss": ["fiction","classic"],
"years_is": [1965,1969],
"locations": [[-74.0060,40.7128],[-0.1278,51.5074]],
"embedding_vs": [[0.8,0.1,0.1],[0.2,0.7,0.1]]
}

String, text, numeric, and date multi-fields use an array of scalar values. A multi-geo field uses an array of points in any point form; an array of numbers alone is one point. A multi-vector field uses an array of number arrays; a bare number array is also accepted as a one-vector list. Enable a multi-valued mode; the _ss, _is, _fs, _ds, _dts, _ps, and _vs suffixes already do.

Columns and stored fields solve different problems:

  • A column retains a typed per-document value for sorting, faceting, numeric and geo queries, vector search, and retrieval where that type is supported.
  • Stored text, string, and ID fields keep source text before analysis or normalization. Text fields are stored by default, so searching individual words still lets you retrieve the original title or author name.

Use fields to choose which values to return, as in the example above. You can also use * wildcard patterns such as "attr_*". Omitting fields returns every retrievable logical field except vectors and engine fields. See field retrieval for the full projection rules.

Index-time and query-time coercion use the same rules: a literal accepted for a field at ingest can be used to match or bound that field later.

  • String, text, and ID fields accept strings. Numeric and boolean scalars are converted to their canonical text rendering.
  • Integer fields accept JSON integers, integral floating-point numbers, and strings that parse to an integer within the signed 64-bit range.
  • Float and double fields accept JSON numbers and numeric strings.
  • Decimal fields accept numbers and numeric strings, rounding half away from zero to their declared scale; see Decimals.
  • Boolean fields accept JSON true/false or the strings "true"/"false". Numbers, including 0 and 1, are rejected.
  • Date fields accept epoch milliseconds, ISO-8601 strings, partial dates, and date math. Update messages interpret offset-less date text in UTC; search requests may supply a query time zone. See Dates.
  • Geo points take [lon, lat] arrays, {"lat", "lon"} objects, GeoJSON points, "lat,lon" and WKT strings, and geohashes; see Point formats. Vectors have array shapes described in their guide; neither is a generic numeric multi-value field. A vector is a number array; integer elements are accepted, and every element must be a finite float32 with magnitude at most 2^40.

Numeric primaries return their typed column; stored is ignored for them. Each variant converts the submitted value independently. For example, an integer field edition with a string variant label converts "0042" to numeric 42, while edition__label keeps "0042". Reindexing needs the producer’s input, not just the primary’s retrieved value.

Stored text, string, and ID fields retain canonical source text. Numeric and boolean inputs become text: a numeric 42 stored as TEXT returns "42".

Bare retrieval and __self use the primary’s stored source when enabled, otherwise its own column. A normalized STRING primary without storage returns its normalized column value. Variants have no stored copy: an exact selector returns that representation’s column under the selector key, and a TEXT variant has no retrievable value.

Patterns without __ never expand variants; author__* or *__s discovers variants that have a column. HTTP row format omits missing fields; HTTP column format includes the requested key with null.

Column retrieval follows each representation’s multi-valued mode. Stored-source retrieval keeps source order and repeats independently of that mode.

A multi-valued text field analyzes each input value separately and inserts a position gap of 100 between values, so a phrase does not cross values unless its slop reaches that gap.

  • Nested objects and nested documents are not a field type. Model relationships with IDs or flatten the fields in the producer.
  • Binary is reserved in the wire/schema enum but is not a usable engine field type in the current release.

IP fields accept address strings and return canonical text. IPv4 and its v4-mapped IPv6 spelling are equal; CIDR is a match-query literal, not an ingest value. See IP addresses.