Skip to main content

Knowledge Base Architecture

OWL uses ChromaDB for RAG (Retrieval-Augmented Generation) capabilities.

Overview​

Document → Chunking → Embedding → ChromaDB → Search → Context

Components​

ChromaDB Store​

# knowledge/store.py
class KnowledgeStore:
    def __init__(self, path: Path):
        self.client = chromadb.PersistentClient(path=str(path))
        self.collection = self.client.get_or_create_collection(
            name="owl_knowledge",
            embedding_function=OllamaEmbedding()
        )

Ollama Embeddings​

class OllamaEmbedding(EmbeddingFunction):
    def __init__(self, model: str = "nomic-embed-text"):
        self.model = model
        self.host = get_config().llm.host

    def __call__(self, texts: List[str]) -> List[List[float]]:
        embeddings = []
        for text in texts:
            response = requests.post(
                f"{self.host}/api/embeddings",
                json={"model": self.model, "prompt": text}
            )
            embeddings.append(response.json()["embedding"])
        return embeddings

Document Processing​

Adding Documents​

def add_document(
    self,
    file_path: str,
    project_path: str = None
) -> dict:
    """Add document to knowledge base."""

    # Read file
    path = Path(file_path)
    content = path.read_text()

    # Chunk content
    chunks = self._chunk_text(content)

    # Remove old chunks for this file
    self._remove_document(file_path)

    # Add new chunks
    ids = [f"{path.name}_{i}" for i in range(len(chunks))]
    metadatas = [{
        "source": str(path),
        "filename": path.name,
        "project": project_path,
        "chunk_index": i
    } for i in range(len(chunks))]

    self.collection.add(
        ids=ids,
        documents=chunks,
        metadatas=metadatas
    )

    return {"filename": path.name, "chunks": len(chunks)}

Chunking Strategy​

Paragraph-based chunking that respects document structure:

def _chunk_text(self, text: str) -> List[str]:
    """Split text into chunks for embedding."""
    # Split by paragraphs first
    paragraphs = re.split(r'\n\s*\n', text)

    chunks = []
    current_chunk = []
    current_size = 0

    for para in paragraphs:
        para = para.strip()
        if not para:
            continue

        # Estimate tokens (rough: 1 token ≈ 4 chars)
        para_size = len(para) // 4

        if current_size + para_size > CHUNK_SIZE and current_chunk:
            # Save current chunk
            chunks.append("\n\n".join(current_chunk))
            current_chunk = []
            current_size = 0

        current_chunk.append(para)
        current_size += para_size

    # Don't forget last chunk
    if current_chunk:
        chunks.append("\n\n".join(current_chunk))

    return chunks
  • Chunk size: ~500 tokens
  • Splits on paragraph boundaries (blank lines)
  • Keeps related content together
def search(
    self,
    query: str,
    project_path: str = None,
    limit: int = 3
) -> List[dict]:
    """Search knowledge base semantically."""

    # Build filter
    where = None
    if project_path:
        where = {"project": project_path}

    # Query ChromaDB
    results = self.collection.query(
        query_texts=[query],
        n_results=limit,
        where=where
    )

    # Format results
    return [{
        "text": doc,
        "filename": meta["filename"],
        "relevance": 1 - dist,  # Convert distance to similarity
    } for doc, meta, dist in zip(
        results["documents"][0],
        results["metadatas"][0],
        results["distances"][0]
    )]

Context Integration​

# llm/context.py
class ContextBuilder:
    def with_knowledge_query(self, query: str):
        """Search and include relevant knowledge."""
        results = self.knowledge.search(
            query,
            project_path=self.project_path,
            limit=3
        )

        if results:
            self._knowledge_chunks = results
        return self

    def _build_knowledge_section(self) -> str:
        if not self._knowledge_chunks:
            return ""

        lines = ["## RELEVANT KNOWLEDGE"]
        for chunk in self._knowledge_chunks:
            lines.append(f"\n### From {chunk['filename']}")
            lines.append(chunk["text"])

        return "\n".join(lines)

Storage​

Directory Structure​

~/.owl/knowledge/
└── chroma/
    ├── chroma.sqlite3       # Metadata
    └── collections/         # Vector data

Persistence​

ChromaDB persists automatically:

  • Embeddings stored on disk
  • Survives daemon restart
  • Can be backed up by copying directory

Project Scoping​

Adding with Project​

# When in a project context
store.add_document(
    "docs/api.md",
    project_path="/home/user/my-project"
)

Searching with Project​

# Only searches within project
results = store.search(
    "authentication",
    project_path="/home/user/my-project"
)
# Search all knowledge (no project filter)
results = store.search("authentication")

Embedding Model​

Requirements​

# Pull the embedding model
ollama pull nomic-embed-text

Characteristics​

  • Model: nomic-embed-text
  • Dimensions: 768
  • Good semantic understanding
  • Runs locally via Ollama

Performance Considerations​

Chunking​

  • Chunk size: ~500 tokens
  • Overlap: 50 tokens
  • Balance: context vs granularity

Search​

  • Top K: 3 results default
  • No similarity threshold
  • Project filter applied first

Memory​

ChromaDB loads collection metadata into memory:

  • ~1KB per chunk overhead
  • Vectors loaded on demand
  • Efficient for thousands of chunks