Barrel
FR GitHub

Configuration

This page lists the options you pass when you open a database, and the keys each Barrel application reads from its environment (sys.config), with their defaults. Use it as a lookup; the guides explain when you would change each one.

Open a database

barrel:open/2 takes one map, with a sub-map per layer:

{ok, Db} = barrel:open(<<"notes">>, #{
    docdb => #{retention_period => 2592000},
    vectordb => #{db_path => "/var/lib/app/vec/notes",
                  dimensions => 768,
                  bm25_backend => disk},
    encryption => default
}).
KeyDefaultMeaning
docdb#{}Options for the document store, below
vectordb#{}The vector store config, below, or none for documents only
embeddingnoneAn embedding policy: the database runs in record mode. stored reopens with the policy the database saved.
encryptiondisableddefault (key from BARREL_ENCRYPTION_KEY) or #{provider => Mod}. Pass it on every open. See Encryption.
read_onlyfalseOpen both stores read only
store_supervisedfalseSupervise the vector store instead of linking it to the caller

Document store (docdb)

These are the options of barrel_docdb:create_db/2. encryption, read_only, channels, conflict_resolver and the retention settings are not stored: pass them again on every open.

KeyDefaultMeaning
data_dirthe app env data_dirParent folder of the database
att_optsblob in RocksDBAttachment backend: #{backend => none}, or #{backend => s3, s3 => #{bucket => ...}} with barrel_att_s3
retention_period2592000 (30 days, in seconds)History kept for audit and point-in-time branches; 0 keeps all and turns the sweeper off
retention_check_interval600000 msHow often the retention sweep runs
ttl_sweep_interval0 (off)How often expired documents are removed
ttl_sweep_batch512Documents removed per sweep step
tombstone_purge_after0 (keep)Purge delete stubs after this many seconds
channelsnone#{Name => [Pattern]}, fixed at creation
conflict_resolvernoneA resolver for concurrent versions
max_conflict_versions8Concurrent versions kept per document
max_group, write_chunk256, 16Group commit sizes
stats_interval60000 msHow often the size and compaction gauges refresh; 0 turns it off
att_upload_timeout3600 sLongest an attachment upload may stay open
store_opts#{}RocksDB options, see below

The main store_opts keys. Each applies to every column family of the document store, so memtable memory per database can reach write_buffer_size × max_write_buffer_number × 4.

KeyDefaultMeaning
compression, bottommost_compressionzstdrocksdb always builds zstd. snappy and lz4 work only if rocksdb was built with the system library.
bloom_bits10Bloom filter bits per key: a read of a missing key skips the files that cannot hold it
block_size4096Table block size, in bytes
write_buffer_size64 MiBMemtable size before a flush
max_write_buffer_number2Memtables per column family before writes wait for a flush
min_tombstones_for_range_conversion64A run of this many deletes becomes one range delete at flush, so reads skip it at once. 0 turns it off.
use_direct_io_for_compaction_readsfalseCompactions read with direct I/O and leave the page cache to your reads. Also accepted in att_opts. Needs a file system with direct I/O (not tmpfs).

prune_depth, compaction_interval and compaction_size_threshold have no effect since 2.2.0; a database given them logs a warning.

Vector store (vectordb)

KeyDefaultMeaning
db_path (or path)<data_dir>/<name>.vectordb, next to the documentsWhere the store lives. A store created before barrel 1.13.0 in priv/barrel_vectordb_<name> under the working directory keeps being used there.
dimensions (or dimension)768Vector size
embeddernoneA barrel_embed provider, such as {ollama, #{model => <<"nomic-embed-text">>}}. Without one, text calls return embedder_not_configured.
backendhnswhnsw, diskann, or faiss (needs barrel_faiss)
hnsw#{m => 16, ef_construction => 200}Also distance_fn (cosine or euclidean) and quantization (scalar by default, turboquant, subspace_turboquant, none)
diskann#{r, l, base_path}
faiss#{index_type => <<"HNSW32">>}Also nprobe
bm25_backendnone; disk in record modememory or disk turns keyword search on
read_onlyfalseRefuse writes

Embedding policy (embedding)

KeyDefaultMeaning
fieldsthe whole documentPaths whose text is embedded
join<<"\n">>Separator between fields
modeasyncsync embeds before the write returns
embedderThe barrel_embed provider
dimensions768Must match the model
metadata_fieldsthe document minus id and _ keysFields returned as search metadata

Other keys are refused with {error, {unknown_options, Keys}}.

Application environment

Set these in sys.config. Paths are relative to the working directory of the node unless you give absolute ones.

barrel_server

KeyDefaultMeaning
http_port8080Port of the cleartext HTTP listener, when listeners is not set
listenersone HTTP/1.1 listener#{http => ..., https => ..., http3 => ...}
tlsnone#{certfile, keyfile, cacertfile, verify, ssl_opts}
data_dirbarrel_docdb'sCopied into the barrel_docdb data_dir at start. Vector stores follow it, as <data_dir>/<name>.vectordb.
open_opts#{}barrel:open/2 options for every database the server opens
authnone (open server)#{tokens => [...]}, or #{accept => [bearer, signed, mtls], signers => ..., skew_ms => 300000, require_nonce => ...}. See Authentication.
corsnone#{origins, expose, max_age}; needed by barrel-lite
mcp#{}#{enabled => true, allowed_origins, allow_missing_origin, resources => full | live_only, live}. See the MCP guide.
max_body1 GiBLargest request body
sig_replay_sweep_ms60000How often seen signature nonces are swept
[{barrel_server, [
    {http_port, 8080},
    {data_dir, "/var/lib/barrel"},
    {auth, #{tokens => [<<"change-me">>]}},
    {cors, #{origins => [<<"https://app.example">>]}}
 ]}].

The server also sets barrel_docdb's task_db_opener, so replication tasks restored after a restart open their databases the way the server does.

barrel_docdb

KeyDefaultMeaning
data_dir"data/barrel_docdb"Where databases live
default_storebarrel_store_rocksdbStorage module
block_cache_size512 MiBRocksDB block cache shared by every database
block_cache_typeclockCache kind
query_timeout_ms30000Query deadline
hlc_max_offset0 (off)Largest clock offset accepted from a peer, in ms
body_async_iofalseExperimental
metrics[][{exporter, none | console | otlp | {Mod, Map}}, {otlp_endpoint, Url}, {otlp_headers, #{}}, {otlp_compression, none | gzip}]
tracing[][{enabled, true}, {exporter, ...}, {otlp_endpoint, Url}]
logging[][{exporter, none | console | file | otlp | {Mod, Map}}, {file_path, ...}, {file_format, text | json}, ...]
sync_auth, sync_signing, sync_ssl#{}Credentials for HTTP replication, keyed by origin or database URL
task_db_openernone{Mod, Fun} that opens a database for a restored task

See Monitoring for the exporters.

barrel

KeyDefaultMeaning
dbs_idle_timeout300000 msbarrel_dbs closes an unused database after this; 0 never
dbs_max_open0 (no limit)Most databases barrel_dbs keeps open
dbs_evict_guard5000 msAt the dbs_max_open cap, only a database idle this long is evicted; otherwise the open fails with too_many_open_dbs
ctx_max_contexts8Contexts per query
ctx_deadline_ms5000 (at most 60000)Context query deadline
ctx_member_timeout_ms4000Time per member
ctx_max_parallel8 (at most 16)Members queried at once
ctx_offlinefalseStart in offline mode
ctx_catalog_db_barrel_catalogDatabase holding the context cards
ctx_dir<docdb data_dir>/_ctxSlices and imports
ctx_credentials#{}Credentials for remote contexts

barrel_embed

KeyDefaultMeaning
managed_venvtrueCreate a Python venv and install provider packages on first use; false uses your own python
venv_dir<priv>/.venvWhere the managed venv lives

barrel_crypto

KeyDefaultMeaning
encryption_keynoneMaster secret when BARREL_ENCRYPTION_KEY is not set: 32 bytes, 64 hex characters or a passphrase
encryption_saltbarrel_default_salt_v1Salt when BARREL_ENCRYPTION_SALT is not set

barrel_spaces

KeyDefaultMeaning
registry_db_barrel_spacesThe registry database
embeddernoneEmbedder used to reopen record-mode spaces
janitor_interval300000 msHow often leftovers of expired sessions are removed

barrel_ngram

KeyDefaultMeaning
data_dir"data/barrel_ngram"Where indexes live; everything else is per corpus

barrel_att_s3

KeyDefaultMeaning
multipart_gc_interval1 hour, in ms; 0 offSweep of abandoned multipart uploads
multipart_gc_max_age24 hours, in secondsAge after which an upload is abandoned

barrel_rerank

KeyDefaultMeaning
venv_pathundefined, which means priv/.venv of the appWhere the virtualenv lives. Set it after application:load/1 and before the app starts.

barrel_vectordb

Its .app.src declares default_dimensions, default_embedder, hnsw_m, hnsw_ef_construction, hnsw_ef_search and embed_cache_size, but none is read. Set these per store in vectordb.

Written for the site Edit this page View as Markdown

Up and down to moveEnter to open