Internal Document Assistant
An AI assistant that answers from internal documents and shows its sources with each answer.
- Timeline
- About 5 months
- Team size
- 4 people
- Platforms
- Web · Server
- Deployment
- On-premises, internal network
Background
The client wanted to search internal policies and technical documents, but a security policy prohibited uploading documents to external LLM services. They needed a way to answer questions directly without documents ever leaving the company.
Scope
- Internal document indexing and ingestion
- Natural-language question answering
- Display of source documents behind each answer
- Reflecting document updates
- Query history management
Technical decisions
What we chose, and why it fitBuilt a language model that runs inside the internal network instead of an external LLM
Keeping documents from leaving the company was the core requirement. Placing the model on internal infrastructure let both queries and documents stay within company boundaries.
Used a RAG structure to present source documents alongside answers
If the model only generated an answer, there was no way to verify it was correct. We had it first retrieve relevant documents from a vector database and answer based on those, so users could check the source text alongside the answer.
Engineering challenges
Problem
Internal documents were revised continually, and if the index wasn't updated, the system risked answering based on regulations that had already been superseded.
Approach
We built a pipeline that re-indexed only the revised document in the vector database when a revision occurred, and recorded document versions so we could confirm whether the document behind an answer was the latest version.
Result
Even when a regulation changed, it was reflected in answers within the next indexing cycle, and we could trace which document version an answer was based on.
Problem
When a question was ambiguous or multiple relevant documents existed, the model sometimes mixed in content that wasn't actually supported by any source.
Approach
We cross-checked the retrieved documents against the answer content, and flagged the answer whenever it included content not present in the source documents.
Result
Users could now tell when an answer was insufficiently supported, reducing the risk of citing it without checking.
Handover and operations
We delivered a document registration and re-indexing method along with a model operations guide to internal admins. We handed over the source code and internal-network deployment configuration, and supported model tuning and document expansion under separate maintenance.