How pixl-os works

What is pixl-os

A workspace-native knowledge OS. It ingests the documents, code, and decisions of a team, extracts an entity graph, and answers natural-language questions against that corpus with inline citations. Three truths: one sqlite file per workspace, one OpenRouter key per tenant, one HTTP API for every surface.

Architecture

A five-step pipeline from raw source to cited answer. Every stage is deterministic, swappable, and observable via /audit.

Chat & agent

The /ask page is an AI SDK v4 chat built on AI Elements. Pick a persona, pick a model, send a question — the agent loop streams tool calls, renders inline citations with hover-previewed source snippets, and logs every step to /audit.

Tool registry · 10 tools

retrieve_kb

defaultHybrid RAG over the workspace KB — BM25 + vector, reciprocal rank fusion.

search_graph

defaultWalk the entity graph from a seed entity. Returns neighbour entities + relation labels.

fetch_doc

defaultFetch the full markdown of a KB document by id (preferred) or fuzzy title match.

show_entity

defaultFull detail for one entity: type, description, relations both directions, top 5 source documents.

list_subjects

defaultList topic clusters (subjects) for the workspace, derived from entity co-occurrence.

search_symbols

defaultSearch AST code symbols (functions, classes, methods) by name or signature.

web_search

personaSearch the public web for recent or external information the workspace KB doesn't cover.

web_fetch

personaFetch a web page and return its main content (Trafilatura-cleaned, no nav/ads).

memory_read

personaRead a short conversation-scoped note previously persisted via memory_write.

memory_write

personaPersist a short note so later turns can recall user preferences or session facts.

The default tools are active when no persona is selected. The persona tools (web_*, memory_*) are opt-in per persona and not in the default allowlist.

Ingestion

Drop a file, paste a URL, or type prose. The dispatcher sniffs the source type and routes it to the right parser, then streams back every stage over SSE so you can see the pipeline move.

Source types

file · .md .txtChonkie semantic

Paragraph-aware semantic chunker for plain text and Markdown.

file · .pdfMistral OCR / pypdf

OCR fallback for scanned PDFs; direct parse when text layer is present.

file · .csv .jsonpandas + json parser

Structured tabular / object data pushed as typed chunks.

url · http(s)Trafilatura

HTTP GET + readability-cleaned main content extraction.

url · SPAPlaywright / Firecrawl

JS-heavy URLs routed through a headless browser fallback when Trafilatura returns empty.

code · repostree-sitter AST

Typed symbols per language (.py / .ts / .tsx / .js / .rs) — feeds search_symbols.

text · pastedplain-text passthrough

Direct paste box on /ingest — useful for meeting notes and snippets.

chatswhatsapp / slack / telegram / imessage / discord / csv

Multi-format parser wired through the MCP ingest_chat tool.

event: received
data: {"size":18432,"type":"file","name":"decisions.md"}

event: stage
data: {"stage":"parsing","progress":0.22,"note":"4,210 chars"}

event: stage
data: {"stage":"chunking","progress":0.25,"note":"Chonkie semantic · 4,210 chars"}

event: stage
data: {"stage":"embedding","progress":0.5,"note":"384-dim · batched"}

event: stage
data: {"stage":"extracting_entities","progress":0.75,"note":"LLM pass · 12 entities"}

event: stage
data: {"stage":"persisting","progress":0.9}

event: done
data: {"doc_id":"doc_5fc3","chunks":14,"entities":12,"cost_usd":0.0008}

Stages: received → fetching (URL only) → parsing → chunking → embedding → extracting_entities → persisting → done. Each stage is cost-tracked under ingest:<strategy>.

Knowledge graph

/graph is the workspace overview — stats, type tiles, top mentions, and a command-palette search across all entities. Open any entity at /entity/[id] for its 1-hop neighborhood, source documents, and typed relations. Subject clusters (entity co-occurrence groupings) live at /subject/[id].

Retrieval stack

Hybrid by construction: every query hits keyword + vector in parallel and fuses the results. Pure vector loses on unusual terminology; pure BM25 loses on paraphrases. Fusion covers both.

1. 1

   BM25 + vector in parallel

   sqlite FTS5 for keyword, sqlite-vec for cosine over 384-dim embeddings. Both run against the same chunks table in the same sqlite file.

2. 2

   Reciprocal rank fusion

   Results are fused via RRF (k=60) so top hits from either arm rise naturally. The /api/ask retrieval_plan records which arm produced each citation.

3. 3

   Process-local LRU cache

   Identical (workspace, question, top_k) tuples hit an in-memory LRU for instant repeat answers — useful for /audit playback and tests.

4. 4

   Optional cross-encoder rerank

   bge-reranker-v2-m3 can rerank the top-k before synthesis (kill-switch via env: pixl_os_rerank_off).

Models & providers

Every LLM call goes through one OpenRouter key — which fans out to five providers behind a single billing surface. The per-workspace whitelist decides which models the gateway actually accepts via the X-Model-Override header.

Providers on the feen whitelist

Whitelisted models

anthropic/claude-haiku-4.5

Default — cheap, fast.

anthropic/claude-opus-4-7

Deep reasoning fallback.

openai/gpt-4o-mini

Secondary general-purpose.

google/gemini-2.5-flash

Long-context backup.

deepseek/deepseek-v3

Open-weight cost anchor.

qwen/qwen-2.5-72b-instruct

Multilingual fallback.

Audit & cost

Every LLM call is logged to stage_cost_telemetry — no sampling, no estimation. /audit reads the table with filters (workspace · model · stage) and shows what was actually spent.

One audit row

Response bodies and prompts started being persisted in Sprint T5 — legacy rows show null for those fields in the drawer. Pagination is offset/limit, with totals computed over the filtered-but-unpaginated set.

Workspace config

One YAML file per tenant is the single source of truth — repos, connectors, LLM policy, budget, model whitelist. Secrets stay as ${VAR} placeholders and are scrubbed from every API read.

ref: feen
name: FeeN
data_dir: /tmp/pixl-kb-demo

llm_policy:
  tone: "Direct, no hedging, no emoji."
  monthly_budget_usd: null        # null = no cap
  model_whitelist:
    - anthropic/claude-haiku-4.5  # default
    - anthropic/claude-opus-4-7
    - openai/gpt-4o-mini
    - google/gemini-2.5-flash
    - deepseek/deepseek-v3
    - qwen/qwen-2.5-72b-instruct

repos:
  - slug: feen-api
    role: backend
    local_path: /Users/hamzamounir/code/nuva/feen-api

env:
  openrouter_api_key: ${OPENROUTER_API_KEY}

HTTP API

Everything the UI uses is available over HTTP — same shapes, same auth. Routes are auto-mounted in app.py; the canonical Swagger lives at /docs.

MCP server

A minimal stdio MCP server (plus two HTTP bridges) exposes five tools — pixl-os becomes a local tool that any MCP-speaking client can call.

Tools

ask

Heavy — runs the full classify + retrieve + rerank + synthesise pipeline against the KB.

ingest_chat

Ingest a chat export (whatsapp / slack / telegram / imessage / discord / csv) with PII redaction.

brief

Multi-repo brief for a Linear ticket — retrieval + analysis across every wired-up repo.

entities

HTTP bridge to /api/entities — graph nodes + edges + subjects.

subjects

HTTP bridge to /api/subjects — topic clusters over the workspace.

Claude Desktop / Cursor integration

{
  "mcpServers": {
    "pixl-os": {
      "command": "python",
      "args": ["-m", "pixl_os.mcp_server"],
      "env": { "OPENROUTER_API_KEY": "sk-or-..." }
    }
  }
}

Install: pip install pixl-os. The server advertises its tool list to the client on connect — no registration step.

FAQ

Questions that come up often enough to write down. Short answers, no hedging.