CLIRAG: CLI Retrieval-Augmented Generation

CLIRAG is a 100% offline, hardware-agnostic Edge AI document analysis engine that runs entirely on your local machine.
Developed by Shourya Varshney and the CLIRAG Core Team.

Key Highlight: Once installed, CLIRAG works globally from any directory on your PC. No need to cd into the project folder — just open any terminal and type clirag.

---

🌍 Project Vision

• Future of Work & Productivity: Private, air-gapped corporate document analysis without exposing sensitive data to cloud APIs.
• Sustainable AI: Energy-efficient Edge AI processing using local hardware (CPU/GPU/NPU) instead of power-hungry cloud GPUs.
• Intelligent Retrieval: Uses Qwen2.5-7B (3-bit Quantized) reasoning engine with adaptive graph/vector retrieval for cross-document understanding.

---

🏗️ Architecture

graph TD
    User([User CLI / Typer & Rich]) --> Router{Adaptive RAG Router}
    
    subgraph "Zero-LLM Asynchronous Ingestion"
        Ingest[Document Ingest] --> NLP[C++ NLP: GLiNER/spaCy]
        Ingest --> OCR[Tesseract OCR]
        NLP --> KG(Knowledge Graph)
    end
    
    subgraph "Disk-Bound Storage (SSD)"
        KG --> Kuzu[(KùzuDB: Graph Storage)]
        NLP --> VecMetadata[(DuckDB: Vector/Metadata)]
    end
    
    subgraph "Adaptive RAG Router"
        Router --> |Factual/Shallow| BM25[BM25 Lexical]
        Router --> |Conceptual| LightRAG[LightRAG Graph Traversal]
        Router --> |Deep/Needle| ColBERT[ColBERT Late Interaction]
    end
    
    subgraph "Reasoning Engine (RLM)"
        BM25 --> REPL[Python REPL Sandbox]
        LightRAG --> REPL
        ColBERT --> REPL
        REPL --> |Constrained Func Call| Tools[Tools: search_graph, search_text, read_chunk]
    end
    
    subgraph "Heterogeneous Inference Engine"
        REPL --> HardwareProbe{Hardware Probe}
        HardwareProbe --> |CPU Only| CPU[Generic CPU]
        HardwareProbe --> |AI Accelerator| NPU[NPU/IPU]
        HardwareProbe --> |Dedicated GPU| VRAM[CUDA FP16]
        CPU --> LlamaCPP[llama.cpp Bindings]
        NPU --> LlamaCPP
        VRAM --> LlamaCPP
    end
    
    LlamaCPP --> Output[TTFT Streaming < 1s]

Loading

---

💻 Minimum Specifications

Requirement	Minimum
RAM	8 GB (12–16 GB recommended)
Storage	SSD with ~5 GB free space
CPU	AVX2 Instruction Set support
OS	Windows / Linux / macOS
Python	3.8 or higher

---

🚀 Installation (One-Time Setup)

Step 1: Clone the Repository

git clone https://github.com/yourusername/clirag.git
cd clirag

Step 2: Install Everything (Single Command)

Run the automated installer. This will:

• Install all Python dependencies
• Download the Qwen2.5-7B model (~3.8 GB) and NLP models from Google Drive
• Register the clirag command globally on your PC

python install_clirag.py

⏳ Note: The model download is ~3.8 GB. Please wait for it to finish. You only need to download it once — it is stored permanently and works from any directory.

Step 3: Verify Installation

Open a new terminal window (from any folder) and run:

clirag --help

You should see the full list of commands. If you do, you're ready to go!

Alternative: Manual Model Setup

If the automated download fails, you can manually download models.zip from our Google Drive link and extract it into the CLIRAG/models/ folder. Then run:

pip install -e .
clirag bootstrap --force

---

🕹️ Usage Guide — All Commands

After installation, clirag is available globally. Open any terminal on your PC and use these commands:

📄 clirag parse — One-Shot Parse & Ask (Recommended)

Ingest a file and ask a question about it in a single command. Best for quick, one-off analysis.

clirag parse "C:/Users/you/Documents/report.pdf" "Who is the Managing Director?"

• Provide the full absolute path to the PDF or TXT file.
• The engine will automatically ingest the file (if not already ingested) and then answer your query.

---

📥 clirag ingest — Ingest a Document

Add a document to the database for future querying. Use this when you want to index a file without asking a question immediately.

clirag ingest "D:/reports/annual_report_2024.pdf"
clirag ingest "C:/docs/notes.txt"

• Provide the full absolute path to the file.
• Supported formats: PDF, TXT, Markdown.
• Files are deduplicated by SHA-256 hash — re-ingesting the same file is instant.

---

❓ clirag ask — Query the Database

Ask a question about previously ingested documents.

clirag ask "What is polymorphism?"
clirag ask "What is the revenue?" --doc "annual_report.pdf"

• Use --doc to filter your query to a specific file.
• If the Edge Engine is not running, it will auto-start (first query takes ~15–30s to load models).

---

🖥️ clirag shell — Interactive Mode

Enter an interactive session where the model stays loaded in RAM. Best for asking multiple questions rapidly.

clirag shell

• The first load takes ~15–30 seconds.
• Subsequent queries are near-instant (< 2s).
• Type exit or quit to leave.

---

⚡ clirag serve — Start the Edge Engine

Keep the model permanently in RAM as a background server for ultra-fast responses.

clirag serve

• Runs on http://127.0.0.1:8000 by default.
• While this is running, clirag ask commands in other terminals will respond in < 2 seconds.

---

📋 clirag list — List Ingested Documents

Show all documents currently in the database.

clirag list

---

🗑️ clirag delete — Remove a Document

Remove a document and all its associated data from the database.

clirag delete "annual_report.pdf"

---

⚙️ clirag settings — View System Info

Display hardware detection results, model paths, and storage stats.

clirag settings

---

🔧 clirag bootstrap — Verify Model Integrity

Re-verify that all model weights are present and correct.

clirag bootstrap --force

---

🛠️ Performance Metrics

Metric	Value
CLI Startup	< 0.9s (lazy imports)
Query Response	< 2.0s (with Edge Engine)
Deduplication	100% hash-based (SHA-256)
Context Window	Auto-scaled to your RAM (1024–4096 tokens)

---

🔮 Future Enhancements

• Gemma 4 Multimodal Integration: Once stable quantized versions are released, CLIRAG will support native image extraction and reasoning from PDFs (charts, diagrams, tables as images).
• CLIP/SigLIP Image Embeddings: Retrieve source images directly from your CLI responses.

---

Developed with ❤️ by Shourya Varshney (CLIRAG Core Team).