barrel_crypto
Encryption at rest for Barrel: the ciphers, key derivation and key providers behind the encryption option of a database.
What it solves
A database per tenant or per agent should be unreadable without its key, on disk and in backups. barrel_crypto derives a key per database from a master secret (or asks your key provider), and the document store, vector store, disk BM25 files and blob attachments are encrypted with it.
When to use it
- Each tenant, agent or space needs its own key.
- Processes in one Erlang node must not read each other's spaces: encryption is that boundary.
- You keep keys in a KMS: write a key provider module.
When not to
- You store attachments in S3 through barrel_att_s3: those bytes are not encrypted.
- You need key rotation or to encrypt an existing database in place: neither exists.
How it fits into Barrel
- Builds on
- No other Barrel library
- Used by
barrel,barrel_docdb,barrel_vectordb,barrel_spaces- Other dependencies
- OTP crypto
- On its own
- Yes, it has no dependencies beyond OTP crypto.
Install
{barrel_crypto, "~> 1.0"} Configuration
Keys
| Key | Default | What it does |
|---|---|---|
BARREL_ENCRYPTION_KEY | none | Master secret: 32 raw bytes, 64 hex characters, or a passphrase |
{barrel_crypto, encryption_key} | none | Used when the environment variable is not set |
BARREL_ENCRYPTION_SALT / encryption_salt | barrel_default_salt_v1 | HKDF salt |
encryption option
| Key | Default | What it does |
|---|---|---|
disabled | default | Plaintext |
default | Per-database key derived from the master secret | |
#{provider => Mod} | Your module returns the 32-byte key for a keyspace |
The most used keys only. For every option, see the configuration reference and the API reference.
The API by task
Keys
| Task | Call | Returns |
|---|---|---|
| Key for a database | barrel_keyprovider:key_for_db/2 | {ok, Key}, {ok, plaintext}, or {error, _} |
| Your own provider | callback key_for_db(Keyspace) | {ok, <<_:256>>} |
| Derive | barrel_crypto:derive_key/1,2 | a key |
Ciphers
| Task | Call | Returns |
|---|---|---|
| Seal, open (AES-256-GCM) | barrel_crypto:encrypt/2, decrypt/2 | envelope, plaintext or {error, decryption_failed} |
| Terms | barrel_crypto:encrypt_term/2, decrypt_term/2 | envelope, term |
| File slices (AES-256-CTR) | barrel_crypto:ctr_crypt/3,4, new_nonce/1 | bytes |
| Check a key | barrel_crypto:key_check_new/1, key_check_verify/2 | token, boolean |
An encrypted database, checked on disk
From a verified run: documents, a vector, disk BM25 and a blob attachment, then a search of the files for the plaintext.
os:putenv("BARREL_ENCRYPTION_KEY", "a long passphrase"),
{ok, _} = application:ensure_all_started(barrel),
Opts = fun(Enc) -> #{encryption => Enc,
docdb => #{data_dir => Dir},
vectordb => #{dimension => 3, bm25_backend => disk,
db_path => Dir ++ "/secret_vec"}} end,
{ok, Db} = barrel:open(secret, Opts(default)),
{ok, _} = barrel:put_doc(Db, #{<<"id">> => <<"a">>,
<<"note">> => <<"TOPSECRETMARKER">>}),
ok = barrel:vector_add(Db, <<"a">>, <<"TOPSECRETVECTEXT">>,
#{}, [0.1, 0.2, 0.3]),
{ok, _} = barrel:put_attachment(Db, <<"a">>, <<"f.txt">>,
<<"TOPSECRETBLOB">>),
ok = barrel:close(Db),
io:format("files containing plaintext marker: ~s",
[os:cmd("grep -rl -a 'TOPSECRET' " ++ Dir
++ " | wc -l")]),
io:format("plaintext open: ~p~n",
[barrel:open(secret, Opts(disabled))]),
os:putenv("BARREL_ENCRYPTION_KEY", "another passphrase"),
io:format("wrong key open: ~p~n",
[barrel:open(secret, Opts(default))]). files containing plaintext marker: 0
plaintext open: {error,db_is_encrypted}
wrong key open: {error,wrong_encryption_key} Limits to know
- No key rotation and no in-place encryption of an existing database (
{error, cannot_encrypt_existing_db}). - Keys are never written to disk, but a cleartext key-check file
CRYPTOsits in the database directory. - CTR mode is unauthenticated, and keys are not zeroized in memory (both documented).
decrypt/2passes through input that does not start withBR; a short plaintext starting withBRcrashes it.
Attachments stored in S3 through barrel_att_s3 are not encrypted, even when the database is (verified: the object in the bucket held the plaintext).