🔍 RepeatRadar

PhD Manuscript Repetition Detector & Fixer

Stop repeating yourself — let AI find it for you

---

🎯 The Problem

When writing a PhD thesis over months or years, the same ideas inevitably appear in multiple chapters, sometimes word-for-word, sometimes paraphrased. RepeatRadar finds these repetitions semantically (not just exact matches), scores them by severity, and lets you fix them with one click using Gemini AI.

---

✨ Features

• 📄 Upload PDF or DOCX — supports any academic document format
• 🧠 Semantic detection — finds repetitions even when phrased differently
• 🎯 Smart scoring — 🔴 High / 🟡 Medium / 🟢 Low based on similarity, length, and signpost keywords
• 🛠 Three fix strategies — rewrite, shorten to recall sentence, or replace with cross-reference
• ✅ Accept / Reject / Edit — you stay in control, AI just suggests
• 📥 Export fixes — download a .txt changelog to apply to your Word/LaTeX document
• ⚡ Caching — embeddings saved to disk, analysis is instant on repeat runs
• 💰 100% free — uses Gemini free tier, no paid API needed

---

🏗 Architecture

PDF/DOCX Upload
    ↓
LangChain RecursiveCharacterTextSplitter → tagged chunks
    ↓
Gemini gemini-embedding-001 → FAISS FlatIP index (cached)
    ↓
FAISS search (top-K) → cosine similarity pairs
    ↓
Filter: skip preamble / references / math / adjacent chunks
    ↓
Score: similarity + length + signpost keywords → 🔴/🟡/🟢
    ↓
Streamlit UI — side-by-side pair view
    ↓
Fix on demand → Gemini Flash rewrite → Accept / Reject

---

🖥 Screenshots

Upload	Analyse	Review & Fix
Drag & drop PDF/DOCX	Real-time embedding progress bar	Side-by-side pair comparison with fix panel

---

🚀 Quick Start

Prerequisites

• Python 3.11+
• Free Gemini API key → Get one here

1. Clone the repository

git clone https://github.com/bazzal99/RepeatRadar.git
cd RepeatRadar

2. Set up environment

cp .env.example .env
# Open .env and add your Gemini API key

3. Install dependencies

pip install -r requirements.txt
pip install streamlit==1.40.0 requests==2.32.3

4. Run the backend

cd backend
uvicorn app.main:app --reload --port 8080

5. Run the frontend (new terminal)

cd frontend
streamlit run app.py

6. Open the app

http://localhost:8501

---

🐳 Docker (Optional)

cp .env.example .env
# Add your API key to .env

cd infra
docker compose up -d

• Frontend: http://localhost:8501
• API docs: http://localhost:8080/docs

---

⚙️ Configuration

Edit RepeatRadar/.env:

# Embedding key (gemini-embedding-001)
GOOGLE_API_KEY=your_key_here

# Optional: separate key for fix agent to avoid quota conflicts
GOOGLE_API_KEY_GENERATION=your_second_key_here

# Detection sensitivity (0.92 = recommended, lower = more pairs found)
SIMILARITY_THRESHOLD=0.92

# Chunk size for splitting (characters)
CHUNK_SIZE=400
CHUNK_OVERLAP=80

Tip: Use two separate Gemini API keys — one for embeddings and one for the fix agent. This prevents quota conflicts since embedding and generation share the same free tier limits.

---

📁 Project Structure

RepeatRadar/
├── backend/
│   ├── app/
│   │   ├── api/routes/
│   │   │   ├── documents.py      # Upload & chunk endpoints
│   │   │   └── analysis.py       # Embed, analyse, fix, accept endpoints
│   │   ├── core/
│   │   │   └── config.py         # Settings from .env
│   │   ├── services/
│   │   │   ├── ingestor.py       # PDF/DOCX → LangChain chunks
│   │   │   ├── embedder.py       # Gemini embeddings + FAISS cache
│   │   │   ├── detector.py       # FAISS search + pair filtering
│   │   │   ├── scorer.py         # 🔴/🟡/🟢 scoring logic
│   │   │   └── fixer.py          # Gemini Flash rewriting agent
│   │   └── main.py
│   └── Dockerfile
├── frontend/
│   ├── app.py                    # Full Streamlit UI
│   └── Dockerfile
├── infra/
│   └── docker-compose.yml
├── .env.example
├── requirements.txt
├── LICENSE
└── README.md

---

🔬 How Scoring Works

Each flagged pair is scored on three signals:

Signal	Criteria	Points
Similarity	≥ 0.95 (near-identical)	+2
	≥ 0.90 (very similar)	+1
	≥ 0.92 threshold	0
Length	Both chunks ≥ 200 chars	+2
	One chunk ≥ 200 chars	+1
Signpost	"as introduced in", "recall that", etc.	−2

Final score → severity:

• 🔴 High (4-5 pts) — fix strongly recommended
• 🟡 Medium (2-3 pts) — review recommended
• 🟢 Low (0-1 pts) — likely intentional, shown collapsed

---

🛠 Fix Strategies

Strategy	What it does	Best for
Rewrite	Different phrasing, same meaning	Same-chapter repetition
Shorten	Compress to one recall sentence	Intro repeated in Conclusion
Reference	Replace with pointer to original	When repetition adds no value

---

🧪 Tech Stack

Layer	Technology
LLM + Embeddings	Google Gemini (free tier)
Vector Index	FAISS FlatIP
Chunking	LangChain RecursiveCharacterTextSplitter
API	FastAPI + Uvicorn
Frontend	Streamlit
PDF parsing	pypdf
DOCX parsing	python-docx
Containers	Docker Compose

---

👤 Author

Mohammad Bazzal — ML Engineer | PhD Candidate in Telecommunications
IMT Atlantique, Lab-STICC, Brest, France

---

📄 License

This project is licensed under the MIT License — see the LICENSE file for details.

---

Built to solve a real PhD problem — writing the same thing twice without realizing it.