βββββββ ββββββ βββββββ ββββββββββββ βββ βββββββββββββββ βββ ββββββ βββββββ βββ
ββββββββββββββββββββββββββββββββββββ βββ βββββββββββββββ ββββββββββββββββββββββ
ββββββββββββββββββββββββ βββ ββββββββ βββ ββββββ ββ ββββββββββββββββββββββ
βββββββ ββββββββββββββββ βββ ββββββββ βββ ββββββββββββββββββββββββββββββββ
βββ βββ ββββββ βββ βββ βββ βββ βββ ββββββββββββββββ ββββββ ββββββ
βββ βββ ββββββ βββ βββ βββ βββ βββ βββ ββββββββ βββ ββββββ ββββββ
$ initializing parth_tiwari.profile ...
[β] identity β AI Systems Engineer
[β] location β Bengaluru, India
[β] status β open to the right problem
[β] philosophy β evidence before claims
[β] vibe-coding β NOT DETECTED
[β] evaluation β ACTIVE
[β] evidence systems β 9 mapped in EVIDENCEBOUND
[β] current build β SecondSelf
[β] work node β Stick and Dot (AI/ML Intern)
[READY] parth_tiwari.profile loaded successfully.Most profiles show you the wins. Here's what actually happened.
Building a fraud engine. Backtesting revealed this:
train ROC-AUC β 0.895 β model looked great
production ROC β 0.60 β system was lying to itself the whole time
cause: temporal features bled future signal into past training windows
fix: leakage validation, point-in-time enforcement, rebuilt from scratch
result: precision stayed useful under a real alert budget
Shipped a Text-to-SQL agent. Hallucination detector reported 100% hallucination:
hallucination_rate β 100% β every query hallucinating?
actual rate β 0% β the metric was wrong, not the system
cause: schema_tables_used returned ["schema_dict", "tables"] β dict keys, not table names
fix: one-line patch
lesson: I found this because I wrote a hallucination detector in the first place
Deployed to Render. LLM mixed up two different databases:
question β "what is the total revenue?" (ecommerce schema)
sql β SELECT SUM(amount) FROM fines (library schema β wrong database entirely)
cause: both schemas lived in the same Chroma collection, embeddings leaked cross-schema
fix: prompt isolation + schema-scoped retrieval + re-evaluated full 82-query benchmark
The pattern: I find these things because I build evaluation harnesses before I trust results.
- "it works on my machine" β ship it
+ measure β break it intentionally β fix it β measure again β then ship itmodel_id : parth-tiwari-v2
type : early-career AI systems engineer
architecture : first-principles β build β evaluate β break β fix β deploy
training_data : production constraints, real failure modes, measurable outcomes
benchmarks:
text_to_sql_execution_success : 95.7% # 82-query ecommerce benchmark
cross_schema_generalization : 100% # zero-shot on unseen library schema
syntactic_hallucination_rate : 0.0% # schema-grounded generation
fraud_precision_in_budget : 92.06% # 0.5% daily alert constraint
fraud_p95_latency : ~386ms # API scoring path
medrag_answered_faithfulness : ~0.99 # cited medical retrieval answers
medrag_refusal_accuracy : 100% # insufficient evidence => refusal
vivid_beta_users : 10+ # creative AI work under Stick and Dot
serving:
portfolio : EVIDENCEBOUND β 9 evidence systems, same-world overlays
deployment : Docker Β· Render Β· Streamlit Β· HuggingFace Β· Vercel
current_focus : SecondSelf Β· evidence-bound career/application OS
known_limitations : early-career Β· still learning Β· high ownership Β· ships with boundariesFeatured below: 3 public systems. Full map: EVIDENCEBOUND β 9 nodes across personal projects, work evidence, current builds, and tooling.
β‘ Β QUERYPILOT Β Β·Β Self-Correcting Text-to-SQL Agent
Natural Language
β
βΌ
Schema-Aware RAG βββΊ SQL Generator
β
Static Validator
β
βββββββββββββββββΌββββββββββββββββ
Regex Repair LLM Fix Executor
βββββββββββββββββ΄ββββββββββββββββ
Self-Correction Loop
(max 3 attempts)
| Metric | Result | Context |
|---|---|---|
| First-attempt success | 90.0% |
No correction, cold generation |
| After self-correction | 95.7% |
3-stage loop on 82-query benchmark |
| Hallucination rate | 0.0% |
Zero invented tables or columns |
| Cross-schema generalization | 100% |
Library schema, zero domain tuning |
| Cold-start reduction | ~400ms |
Per-schema agent caching |
Python LangGraph FastAPI ChromaDB PostgreSQL Docker GitHub Actions
π‘ Β UPI FRAUD ENGINE Β Β·Β Real-Time Fraud Decision System
HARD CONSTRAINTS (non-negotiable):
βββ score transaction at T using only pre-T features (no future leakage)
βββ β€ 0.5% daily alert budget (precision is everything)
βββ simulate delayed fraud labels (real-world label lag)
transactions β point-in-time features β leakage tests β alert-budget model
train/serve drift surfaced β rebuilt β re-tested under real decision constraints
| Metric | Result | Context |
|---|---|---|
| Precision in alert budget | 92.06% |
Only flags what matters |
| P95 latency | ~386ms |
API scoring path |
| Leakage tests | 55+ |
Temporal integrity checks |
| Backtest mode | day-by-day |
Production-like replay |
Python XGBoost FastAPI DuckDB Great Expectations Docker
𧬠 EVIDENCE-BOUND DRUG RAG  · Medical Knowledge Retrieval
HARD CONSTRAINT: medical domain β hallucination is patient harm
βββ every claim needs source evidence
βββ insufficient evidence must trigger refusal, not a guess
βββ faithfulness is measured, not assumed
FDA + NICE PDFs β semantic chunks β retrieval β citation β refusal policy
| Metric | Result | Context |
|---|---|---|
| Answered faithfulness | ~0.99 |
Claims grounded in source |
| Refusal accuracy | 100% |
Unsupported requests refused |
| Eval cost | $0.168 |
Cost-aware evaluation |
| Boundary | non-diagnostic |
Not medical advice |
Python FastAPI ChromaDB SentenceTransformers LangChain RAGAS Streamlit
step 1 β define what "working" means before writing a single line
step 2 β build the evaluation harness
step 3 β write the system
step 4 β break it intentionally (adversarial inputs, edge cases, drift simulation)
step 5 β fix what breaks
step 6 β measure again
step 7 β deploy with monitoring hooks
step 8 β repeat when production proves you wrong
This is how suspicious metrics become trustworthy. This is how a metric bug gets caught before it becomes a product lie. This is how a smaller system with gates beats a bigger prompt with vibes.
| Signal | Current State |
|---|---|
| Evidence systems | 9 mapped in EVIDENCEBOUND |
| Featured public systems | QueryPilot Β· UPI Fraud Engine Β· MedRAG |
| Main stack | Python Β· FastAPI Β· RAG Β· XGBoost Β· Vue Β· Three.js |
| Current build | SecondSelf - evidence-bound career OS |
$ ./parth --shutdown
[saving state] β 9 evidence systems mapped
[saving state] β 3 featured systems public
[saving state] β all evaluation harnesses active
[saving state] β open to the right problem
[goodbye] see you on the other side of the next PR.