Barrel
FR GitHub

Code search for coding agents

Give a coding agent exact matches (an error code, a symbol, a pattern) and answers by meaning, from one database, as MCP tools.

Built from barrel_ngram for substring and regex, record mode for hybrid search, and barrel_server for the MCP tools ngram_search and search.

The problem

A coding agent needs exact matches as much as answers by meaning. Semantic search misses identifiers like `ERR_CONN_TIMEOUT`; grep misses intent; running both usually means two systems kept in sync with the repository.

How it fits together

  1. Coding agent (MCP client)

    • ngram_search exact substring or regex
    • search by meaning, keyword or both
  2. tools/call
  3. barrel_server

    • /mcp checks the token, runs the tool
    • Your indexer writes chunks over REST or Erlang
  4. reads and searches
  5. Database code

    • One document per chunk path, lang, symbol, text
    • Trigram index barrel_ngram, fed by the changes feed

Your indexer writes one document per chunk. The trigram index and the vector index both follow those documents, so the agent searches the same chunks either way and gets back ids it can open with doc_get.

Build it

Five steps. Step 1 needs an embedding provider for search by meaning; exact search works without one.

  1. Store one document per chunk

    Optional, needs setup

    A chunk is a function, a class or a span of lines. Give it a stable id from its path and lines, so re-indexing replaces it. Record mode embeds the text field for search by meaning.

    Erlang
    {ok, Db} = barrel:open(code, #{
        embedding => #{fields => [<<"text">>], mode => sync,
                       embedder => {local, #{}},
                       dimensions => 768,
                       metadata_fields => [<<"path">>,
                                           <<"lang">>]},
        vectordb => #{db_path => "data/code_vec"}}),
    {ok, _} = barrel:put_doc(Db, #{
        <<"id">> => <<"src/auth.py#40-88">>,
        <<"path">> => <<"src/auth.py">>,
        <<"lang">> => <<"python">>,
        <<"text">> => Body}).
  2. Find exact strings and patterns

    barrel_ngram keeps a trigram index over the fields you pick and confirms every candidate against the real text, so there are no false positives. Each hit carries its byte spans. This run used three runbook documents.

    Erlang
    ok = barrel_ngram:open(<<"rb_idx">>,
        #{db => <<"runbooks">>,
          fields => [<<"title">>, <<"body">>]}),
    barrel_ngram:search(<<"rb_idx">>, <<"ERR_CONN_TIMEOUT">>),
    barrel_ngram:regex(<<"rb_idx">>,
                       <<"pg-(primary|replica)-\\d+">>),
    barrel_ngram:regex(<<"rb_idx">>, <<"\\w+_timeout">>).
    From a recorded run
    {ok,[#{id => <<"rb:db">>,spans => [{11,16}]}]}
    {ok,[#{id => <<"rb:db">>,spans => [{35,13},{58,13}]}]}
    {ok,[#{id => <<"rb:api">>,spans => [{15,15},{35,12}]}]}
  3. Give the agent exact search over MCP

    The ngram_search tool opens a trigram index for the database the first time it is asked, catches it up, then searches. mode is literal (the default) or regex; limit defaults to 50.

    MCP request, POST /mcp
    {"jsonrpc":"2.0","id":11,"method":"tools/call",
     "params":{"name":"ngram_search",
      "arguments":{"db":"mcpdemo","query":"over MCP"}}}
    From a recorded run (wrapped)
    data: {"id":11,"jsonrpc":"2.0","result":{"content":[
      {"text":"{\"count\":1,\"hits\":[{\"id\":\"note-1\",
      \"spans\":[[7,8]]}]}","type":"text"}]}}
  4. Search by meaning in the same database

    Hybrid search fuses vector and keyword rankings and is the one an agent should reach for when it asks for intent rather than a string. It needs the embedder from step 1.

    {"jsonrpc":"2.0","id":12,"method":"tools/call",
     "params":{"name":"search",
      "arguments":{"db":"code","mode":"hybrid","k":10,
       "query":"retry a failed request with backoff"}}}
  5. Keep the index fresh

    The trigram index follows the database’s changes feed. refresh/1 is the point where it is fully caught up; the MCP tool calls it before each search. After an edit, the old string is gone.

    Erlang
    {ok, _} = barrel_docdb:put_doc(<<"runbooks">>,
        D#{<<"body">> => <<"Check upstream keepalive_timeout "
                            "in nginx.conf">>}),
    barrel_ngram:refresh(<<"rb_idx">>),
    barrel_ngram:search(<<"rb_idx">>, <<"connect_timeout">>).
    From a recorded run (watermark shortened)
    {ok,#{segments => 2,watermark => <<...>>}}
    {ok,[]}

What Barrel does

  • Keeps a trigram index in step with the documents through the changes feed
  • Answers substring, case-insensitive and regex queries with match spans
  • Embeds chunks and answers vector, BM25 and hybrid queries
  • Serves both as MCP tools, checked against the caller’s token

What your application does

  • Splits files into chunks, by function or class
  • Derives stable ids and re-indexes changed files
  • Deletes the chunks of removed or rewritten code
  • Lets the agent choose between exact and hybrid search

Limits

  • The trigram index is about 34 times the size of the text it covers.
  • Fields are fixed for the life of an index; changing them means delete_corpus and a rebuild. The MCP tool indexes every field of the database.
  • Spans are byte offsets into the indexed fields joined in sorted field order, not line numbers.
  • The barrel facade and BQL do not expose the trigram index; use barrel_ngram or the MCP tool.
  • An index written by barrel_ngram 0.10 or older needs on_legacy => reindex (the MCP tool sets it).

Go further

Up and down to moveEnter to open