The record
An application that searches its own content usually keeps three things in three places: the data in a database, the files in object storage, and the vectors in a vector store. Each update then means three writes, and sooner or later a deleted article still shows up in search, or a file points to a row that is gone. Barrel avoids that drift by keeping all three under one id.
A record is everything Barrel keeps under that id: a JSON document, the files attached to it, and in record mode the vector that represents it. They are read, written, replicated and deleted by that id. Read this to know what a document looks like on the way in and on the way out, which fields Barrel adds, and what moves with the record.
Take the support article kb-142, "Reset a locked account". Its fields
are the document, the file steps.md is an attachment, and the vector
computed from its text is what lets a search by meaning find it. This
page follows that one article.
The three parts
Each part of a record behaves a little differently when it changes or when it travels to another copy of the database. The table sums it up; the sections below explain each row.
| Part | What it is | Versioned | Replicated |
|---|---|---|---|
| Document | A JSON object with an id | yes, every write is a version | yes |
| Attachments | Named binary blobs, by document id and name | no, last write wins on its timestamp | yes, on their own feed |
| Vector | The embedding of the fields you choose | follows the document | no, each copy rebuilds it |
The document
A document is the part you query. You
write a map with an id. Every path in it is indexed, so you can query
any field without declaring an index.
barrel:put_doc(Db, #{<<"id">> => <<"kb-142">>,
<<"title">> => <<"Reset a locked account">>,
<<"_note">> => <<"dropped">>}),
barrel:get_doc(Db, <<"kb-142">>). {ok,#{<<"_rev">> =>
<<"000001a1ba45b14e00000000@wddc4e1c028733ad0">>,
<<"id">> => <<"kb-142">>,
<<"title">> => <<"Reset a locked account">>}} The read returns your id and title, plus a _rev that Barrel added.
The _note field you sent is gone: Barrel dropped it because it starts
with _.
_revis the revision: the version token Barrel adds on read. Send it back to update the document; a missing or stale_revreturns{error, conflict}. This stops you from overwriting a change you never read. See Versions, conflicts, convergence.- Top-level fields that start with
_are reserved. Barrel drops the ones you send (_noteabove) and never indexes them. - A delete writes a version that marks the document deleted. A read then
returns
{error, not_found}, and the vector goes too.
Fields Barrel adds on request
Besides _rev, Barrel adds a few fields to a read when you ask for them.
You need them when a document has
conflicts to resolve, or when you
want to see the vector behind a search result.
| Field | Read option | What it holds |
|---|---|---|
_rev | always | The winning version token |
_conflicts | conflicts => true | The other concurrent versions |
_observed | conflicts => true | Every version the read showed, to pass back when you resolve |
_embedding | include_embedding => true | vector and source (computed when Barrel embedded it) |
Attachments
An attachment is a named file that
belongs to a document id, such as the steps.md file of kb-142. Use
attachments for content that is not JSON. Writing one does not change the
document's _rev, so updating a file never conflicts with an edit of the
document.
barrel:put_attachment(Db, <<"kb-142">>, <<"steps.md">>,
<<"1. Open the console">>),
barrel:list_attachments(Db, <<"kb-142">>). {ok,#{name => <<"steps.md">>,length => 19,
digest => <<"sha256-dd490eb840644ed69f89e41c5...">>,
content_type => <<"application/octet-stream">>,
chunked => false}}
[<<"steps.md">>] The write returns the attachment's metadata: its name, its size in bytes
(19), a SHA-256 digest of the content, and a generic content type since
none was given. The listing then shows steps.md as the only attachment
of kb-142.
See Attachments.
The vector
A vector is a list of numbers that
stands for the meaning of a text. It is what lets a search find kb-142
when the query uses other words than the article.
In record mode you give the database an embedding policy: which fields to embed, with which provider, at which dimension. Barrel computes the embedding of those fields on every write and drops the vector on delete. Search results read their text and metadata from the current document, so they never show a stale copy.
In a plain database you manage vectors yourself with vector_add, under
the same id or another one. The document and its vector are then two
separate writes, and keeping them in step is up to you.
What else a write can carry
A write can carry options beyond the document itself. They say who made the change, how long the document lives, and whether to wait for the disk.
| Write option | Effect |
|---|---|
provenance => #{actor, session, source} | Stored with the version, shown in history |
expires_at => UnixMs | The document disappears after that time |
sync => true | Waits for the write to reach disk |
Here an agent, agent-7, records itself and its session as the author
of a write. The version history then shows who changed the article.
barrel:put_doc(Db, Doc,
#{provenance => #{actor => <<"agent-7">>,
session => <<"ses_42">>}}),
barrel:doc_versions(Db, <<"kb-142">>). {ok,[#{status => current,
version =>
<<"000001a1ba45b14e00000000@wddc4e1c028733ad0">>,
deleted => false,
provenance => #{session => <<"ses_42">>,
actor => <<"agent-7">>},
vv => #{<<"wddc4e1c028733ad0">> =>
{timestamp,1794126491982,0}}}]} The history holds one version, marked current and not deleted. Its
provenance holds the actor and session you passed. vv is the version
vector: for each writer, the latest clock this version had seen.
Provenance is not replicated. See Audit and provenance.