At a shipping company, the same questions come up every day. How do I create a purchase order? When are liquidation reports due? What is the overtime policy? The answers are all written down somewhere, spread across manuals, memos and spreadsheets that nobody wants to dig through.
So I built Mnemosyne, a self-hosted Retrieval-Augmented Generation (RAG) server that answers questions using only the documents you give it. Then I connected it to SOFIA, our Viber bot, through the PHP (CodeIgniter 3) ERP we already run. Now employees can ask a question in the chat app they already have open.
This post covers:
- Why I built my own RAG server instead of using a hosted one
- How Mnemosyne is put together
- What I learned tuning retrieval on real company documents
- How a legacy PHP app talks to it and relays answers to Viber
- The gotchas that cost me the most time
Why build it instead of buying it
I had three requirements going in:
- It answers only from our documents. A wrong answer about payroll or safety procedures is worse than no answer at all. When nothing in the knowledge base matches, the bot should say so.
- It's self-hosted. Company manuals stay on our own infrastructure. The vector store and embeddings run locally, and only the generation step can go to a cloud model, if we allow it.
- It's an API first. The ERP, the Viber bot and a future web chat widget should all use the same backend. The admin UI is just one more client.
Hosted "chat with your docs" tools covered parts of this, but none let me plug the result into an existing CodeIgniter 3 ERP over a plain HTTP API with an API key. So I built my own.
Architecture
Mnemosyne runs as five containers under Docker Compose, plus an optional cloud API for generation. Everything goes through one Node.js server, so the Viber bot, the admin UI and anything I build later all use the same rules for auth, rate limits and guardrails.
![Mnemosyne architecture: Viber bot, admin UI and other clients call one Node.js RAG server, which uses ChromaDB, Redis, Ollama and optionally OpenRouter]

- ChromaDB stores chunks and their vectors, and runs the cosine-similarity search.
- Redis caches answers and runs background ingestion jobs through Bull.
- Ollama creates embeddings with
nomic-embed-text, and can also serve a local LLM. - OpenRouter is optional. It gives access to stronger cloud models when local ones aren't enough.
Getting it running takes three steps: copy .env.example to .env, set RAG_API_KEY and the admin credentials, then run docker compose up -d. The first boot pulls the embedding model.
From uploaded file to retrieved chunk
Every document goes through the same pipeline before it can answer anything.
- Parse. Uploads can be PDF, DOCX, XLSX, CSV, Markdown, TXT or images. Each format has its own extractor that turns it into plain text.
- OCR when needed. Scanned PDFs and photos of memos have little or no extractable text. When the text comes back shorter than
OCR_MIN_TEXT_LENGTH, Tesseract takes over. - Chunk. The text is split into chunks of
CHUNK_SIZEwithCHUNK_OVERLAP, so a sentence that sits on a boundary isn't lost. - Embed. Ollama runs
nomic-embed-textlocally and turns each chunk into a vector. - Store. The chunks, vectors and metadata (filename, tags) go into ChromaDB.
Ingestion runs as a Bull job on Redis. A 200-page manual doesn't block the API, and the admin UI polls the ingest status endpoint until the job finishes.
At query time, the question is embedded with the same model and ChromaDB runs a cosine-similarity search. Two settings do most of the work:
-
TOP_K: how many chunks go into the prompt. I settled on 5. -
MIN_RELEVANCE_SCORE: chunks below this score are dropped. Mine is 0.15. If nothing clears the bar, the server returns "I don't know" instead of letting the model improvise.
Queries can also be filtered by tags. A document tagged HR or Purchasing can be searched on its own, which matters later in the Viber flow.
Generation, guardrails and the boring-but-important stuff
Once the top chunks are picked, they go into a prompt that tells the model to answer only from the context it's given. The model itself can be swapped at runtime:
-
LLM_ENGINE=openroutersends generation to a cloud model through OpenRouter. -
LLM_ENGINE=localkeeps everything on the box with Ollama. - The admin UI can switch the active model live, with no redeploy. That made it easy to compare cost against answer quality on real questions.
Around that core, a few features turned a demo into something I could put in front of employees:
- Two auth methods. API keys (
X-API-Key) for server-to-server calls like the ERP, and session tokens (X-Session-Token) for the admin UI. - Guardrails. Input validation, prompt hardening against "ignore your instructions" tricks, output filtering, and checks for sensitive documents. Each one can be toggled in the UI.
- Caching. Identical questions are served from Redis for
CACHE_TTLseconds. In a company, "how do I file a leave?" gets asked a lot. - Rate limiting per endpoint, so one noisy client can't starve the others.
- Sync or async queries.
POST /api/querycan wait for the answer, or return ajobIdto poll at/api/query/status/:jobId. - Analytics and session history, so I can see which questions fail and fix the documents behind them.
What tuning on real documents taught me
The code was the easy part. Most of my time went into the knowledge base, because retrieval quality depends far more on how documents are written than on which model you use.
It took three tries:
- Restructured Markdown. I cleaned up the manuals with proper headings. Better, but long sections still produced chunks that mixed two unrelated procedures.
- Pre-chunked JSONL. I controlled chunk boundaries by hand. Precise, but painful to maintain whenever a policy changed.
- One Markdown file per section, under 500 words. This is the one that stuck. Each file covers one topic, so a chunk rarely spans two subjects, and editing a policy means editing one small file.
Takeaways if you're building your own:
- Match chunk size to how your documents are organized. I capped sections at 500 words, the same as my chunk size, so most sections land in a single chunk.
- Set a relevance floor and trust it. Without
MIN_RELEVANCE_SCORE, the model always gets something to work with, and it will confidently answer from the wrong manual. - Tag early. Tags cost almost nothing at upload time and let users narrow the search later.
- Read your failed questions. Session history showed me which questions came back empty, and almost every time the fix was in the documents, not the code.
Plugging it into Viber from a PHP ERP
Our ERP is a CodeIgniter 3 app, with a small Viber library for sending messages, buttons and contacts. Instead of standing up a separate bot service, I added one controller, Chatbot.php, that does three jobs:
- Receives Viber's webhook events.
- Checks the sender against our HR records.
- Forwards real questions to Mnemosyne and relays the answer back.
SOFIA (System Operational Flow and Information Assistant) greets new users with three buttons: company manuals, IT, and HR. Only the first one goes to the RAG server. The other two return canned contact info, because some questions need a person, not a model.
![SOFIA's welcome message in Viber, with buttons for company manuals, IT and HR]

SOFIA's welcome screen in Viber.
1. Configuration lives in the environment
The RAG settings are constants defined per environment, so the same controller runs against a local Mnemosyne in development and the production server in prod:
class Chatbot extends CI_Controller { private const MAX_HISTORY_PAIRS = 10; private $rag_base_url = RAG_SERVER_URL; // e.g. https://rag.internal/api private $rag_api_key = RAG_API_KEY; // same key as Mnemosyne's .env private $rag_timeout = RAG_TIMEOUT; private $rag_top_k = RAG_TOP_K; 2. Acknowledge fast, and ignore retries
Viber retries a webhook if it doesn't get a quick 200, so the same message can arrive twice. Each message carries a message_token. I drop a marker file per token and skip anything I've already seen. Old markers are cleaned up after 24 hours:
$message_token = $response['message_token'] ?? null; if ($message_token) { $cache_file = APPPATH . 'cache/viber_' . $message_token; if (file_exists($cache_file)) { echo json_encode(['status' => 0, 'message' => 'duplicate']); exit; } file_put_contents($cache_file, time()); $old = array_filter(glob(APPPATH . 'cache/viber_*') ?: [], fn($f) => time() - filemtime($f) > 86400); array_map('unlink', $old); } 3. Only employees get in
A Viber account is public, so anyone can message the bot. The first time someone writes, the controller looks up their Viber ID in a chatbot_subscriptions table. If they're not there, it asks for their employee ID and birthdate in one message, checks both against hr_employees (active employees only), and links the Viber ID to that employee:
$user = $this->_getAuthorizedUser($sender_id); if ($user === false) { $auth = $this->_authenticateUser($received_msg); // "EMP-000123 01/15/1990" if (is_array($auth)) { $this->db->insert('chatbot_subscriptions', [ 'viber_id' => $sender_id, 'viber_name' => $sender_name, 'employee_id_ref' => $auth['employee_id'], 'active' => 1, ]); // "Access granted, ..." } } The lookup result is cached in a file for 5 minutes, so the bot doesn't query the database on every message. Setting active = 0 revokes access within that window.
4. Route only real questions to the RAG server
Greetings and button presses are handled in PHP. Anything with at least two words, or a question mark, goes to Mnemosyne along with the user's selected tag and their recent conversation:
if (str_word_count($received_msg) >= 2 || strpos($received_msg, '?') !== false) { $selected_tag = $this->_getUserTag($sender_id); $history = $this->_getConversationHistory($sender_id); $reply = $this->_ragQuery($received_msg, $sender_id, $selected_tag, $history); $this->_saveConversationHistory($sender_id, $received_msg, $reply); } When a user taps "Ask about Company Manuals/Processes", the bot calls GET /documents/tags and turns each tag into a Viber button. Picking one stores it per user, and every search after that stays inside that category until they tap Clear Filter.
5. The actual RAG call
The call itself is plain cURL. The last 10 question-and-answer pairs go along as messages, so follow-up questions like "what about for crew members?" keep their context. The Viber user ID doubles as the sessionId:
$messages = array_reduce($history, function ($acc, $item) { $acc[] = ['role' => 'human', 'content' => $item['query']]; $acc[] = ['role' => 'ai', 'content' => $item['answer']]; return $acc; }, []); $options = ['topK' => $this->rag_top_k, 'sessionId' => $sender_id]; if ($messages) $options['messages'] = $messages; if ($tag) $options['tags'] = [$tag]; $payload = json_encode(['query' => $query, 'async' => false, 'options' => $options]); $ch = curl_init(rtrim($this->rag_base_url, '/') . '/query'); curl_setopt_array($ch, [ CURLOPT_POST => true, CURLOPT_POSTFIELDS => $payload, CURLOPT_RETURNTRANSFER => true, CURLOPT_TIMEOUT => $this->rag_timeout, CURLOPT_HTTPHEADER => [ 'Content-Type: application/json', 'X-API-Key: ' . $this->rag_api_key, 'X-Client: viber-chatbot', ], ]); $raw = curl_exec($ch); $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE); Each HTTP error gets its own friendly message. A 429 tells the user to wait a moment, and a 401 tells them to contact IT. That way a bad deploy never shows a raw stack trace in someone's chat. When more than one document contributed to an answer, up to three source filenames are appended so people know where to double-check.
6. Make the answer fit in a chat bubble
LLMs love Markdown, and Viber shows it as literal asterisks and pound signs. The last step strips it and trims the reply to a safe length:
private function _sanitizeForViber(string $text, int $limit = 1000): string { $text = preg_replace('/#{1,6}\s/', '', $text); $text = preg_replace('/\*\*(.*?)\*\*/', '$1', $text); $text = preg_replace('/\*(.*?)\*/', '$1', $text); $text = preg_replace('/`{1,3}[^`]*`{1,3}/', '', $text); $text = trim($text); return mb_strlen($text) <= $limit ? $text : mb_substr($text, 0, $limit - 1) . '…'; } A few admin-only endpoints round it out: webhook/set and webhook/unset register the bot with Viber, broadcast_message pushes an announcement to every subscriber, and check_rag_status checks the RAG server's health and confirms the API key is still accepted.
Gotchas that cost me the most time
- CodeIgniter routing and hooks. An ERP has years of routes and hooks built up, and some of them grabbed the webhook URL before my controller ever saw the request. Viber needs a clean JSON 200, so the webhook route has to skip login checks and anything that prints output.
- cURL timeouts against Viber's patience. A RAG answer can take several seconds, and Viber gets impatient with slow webhooks. Tune
RAG_TIMEOUTcarefully, and keep the duplicate-token check, because retries will happen. - Docker on WSL2. BuildKit layer corruption broke my admin UI image more than once in development. Clearing the build cache fixed it, but it took a while to figure out that the code wasn't the problem.
- Don't
docker compose downcasually. It can remove the volumes that hold ChromaDB and Redis data. Rebuild individual services instead, or you'll be re-ingesting your whole knowledge base. - File-based caches need a writable directory. The controller logs an error at startup if
application/cache/isn't writable. Without that check, dedupe, tags and history all fail silently. - Employee ID plus birthdate is a convenience, not strong auth. It's fine for manuals and processes. Before connecting anything sensitive, like payslips, I'd add a one-time code sent through a channel the employee already trusts.
Wrapping up
You don't have to rewrite a legacy system to give it an AI assistant. Mnemosyne does the heavy lifting behind a plain REST API, and the PHP side is one controller doing cURL calls and string cleanup. The same API key and POST /api/query work from a Next.js chat widget, a NestJS service or a shell script.
If you want to try it:
git clone https://github.com/jbcanillo/mnemosyne.git cd mnemosyne cp .env.example .env # set RAG_API_KEY, ADMIN_USERNAME, ADMIN_PASSWORD docker compose up -d # admin UI: http://localhost:3002 Upload a few documents, then hit the API with your key:
curl -X POST http://localhost:3001/api/query \ -H "Content-Type: application/json" \ -H "X-API-Key: $RAG_API_KEY" \ -d '{"query": "How do I create a purchase order?", "async": false}' The repo is MIT-licensed: github.com/jbcanillo/mnemosyne. Issues and PRs are welcome. If you've connected a RAG backend to an older stack, I'd like to hear how you handled it in the comments.