Skip to content

Add PITH Inter-Agent Payload Compressor skill#1303

Open
VjAlbert wants to merge 7 commits into
anthropics:mainfrom
VjAlbert:add-pith-skill
Open

Add PITH Inter-Agent Payload Compressor skill#1303
VjAlbert wants to merge 7 commits into
anthropics:mainfrom
VjAlbert:add-pith-skill

Conversation

@VjAlbert

Copy link
Copy Markdown

Overview

This PR introduces PITH (and reorganizes video-analyzer into the skills directory structure), two complementary skills for the Anthropics skills repository.

PITH — Inter-Agent Payload Compressor

What it does:
PITH compresses the payloads exchanged between agents in a multi-agent pipeline, filling a gap that existing compression tools leave untouched. While Caveman compresses user-facing output and LLMLingua compresses user input, PITH targets the intermediate results—tool outputs, reasoning traces, context handoffs—that accumulate token waste across agent chains.

Key Features:

  • 🎯 30–60% typical token savings on verbose agent output without semantic loss
  • 🧮 Mathematically principled: Uses Zipf word-density scoring (rare words = high information) combined with Benford's Law validation
  • Zero dependencies: Pure Python stdlib. No API calls, no latency
  • 🛡️ Structural integrity gate: Detects over-compression and auto-relaxes the ratio to prevent artificial-sounding output
  • 🔒 Preserves critical content: Code blocks, JSON, URLs, file paths, and numbers are quarantined and never touched

Technical Foundation:

  • Zipf's Law: Words ≥7 characters are systematically rarer and carry more information. PITH ranks sentences by the density of such words
  • Benford's Law: Natural text has Benford-compliant sentence-length distributions (MAD < 6%). Over-compressed or AI-generated text deviates (MAD > 7%). PITH uses this as a quality validator—if compression crosses the threshold, it relaxes and retries

Compression Modes:

Mode Ratio Use Case
Conservative 0.8 Sensitive reasoning traces
Default 0.6 Most tool results
Aggressive 0.4 Search results, bulk data
Maximum 0.3 Context window critical

Usage Examples:

# Pipe in verbose text
echo "<agent output>" | python3 scripts/compress.py

# Explicit ratio and JSON output
python3 scripts/compress.py --payload "<text>" --ratio 0.6 --json

# Python integration
from compress import compress
compressed, meta = compress(raw_output, target_ratio=0.6)

VjAlbert and others added 7 commits June 8, 2026 23:52
Contribute → Open pull request
- SKILL.md, LICENSE.txt, README.md -> skills/video-analyzer/
- process_video.py, detect_characters.py, build_json_report.py -> skills/video-analyzer/scripts/
PITH compresses verbose agent-to-agent handoff payloads using Zipf
word-density scoring with Benford's Law as a structural integrity
validator. Fills the gap between Caveman (output compression) and
LLMLingua (input compression): the Agent→Agent communication layer.

- Zipf proxy: word length ≥7 chars = rare vocabulary = keep
- Benford gate: MAD validation prevents over-compression
- Preserves code blocks, JSON, URLs, file paths untouched
- 30-60% token savings on typical verbose agent outputs
- Zero external dependencies, works offline

Closes: add PITH skill
Topics: skillsmp skill-md-skillsmp agent-skills claude-skills
- SKILL.md: expanded origin + how-it-works sections, JSON output docs
- README.md: full mathematical foundations, installation section, 5 benchmarks
- compress.py: annotated version with inline section headers
- tests/evals.json: 8 test cases (was 4) — added json_preservation, aggressive_compression, url_preservation, benford_validation
Aligns with anthropics/skills repo structure where all skills
live under skills/<name>/. Added LICENSE.txt (MIT) matching the
upstream skill format (license field in SKILL.md frontmatter).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant