A Python-based CLI tool designed to scan directories for hardcoded secrets. It detects 7 types of hardcoded secrets (AWS Access Key IDs, AWS Secret Keys, Generic API Keys, Private Key Blocks, Database Connection Strings, Slack Tokens, GitHub Tokens) and reduces false positives using context-aware confidence scoring based on file path, surrounding keywords, and Shannon entropy.
- 7 Secret Type Detectors: Accurately matches common formats for AWS, API keys, Private Keys, Databases, Slack, and GitHub tokens.
- 3-Factor Confidence Scoring: Filters out noisy placeholder/dummy credentials by analyzing path contexts, surrounding placeholder keywords, and string entropy.
- Flexible Output: Supports JSON output (default) or readable Markdown summaries, with optional exit codes for CI integrations.
- Benchmark-Proven Noise Reduction: Tested against massive real-world repositories to dramatically lower false-positive rates.
scanner/patterns.py: Regular expressions and definitions for the 7 secret types.scanner/entropy.py: Shannon entropy calculation logic to distinguish random secrets from low-entropy placeholders.scanner/context.py: Lists of suspicious paths and keywords, and binary file extension exclusions.scanner/core.py: Main scanning and confidence-scoring loop.cli.py: Command-line interface entry point.benchmark.py: Benchmarking script to test raw matches vs. filtered matches across repos.tests/: Comprehensive unit test suite.
git clone https://github.com/ManassYPanicker/Secret-scanner.git
cd secret-scanner
pip install -r requirements.txtTo scan a directory for secrets, run the cli.py script and pass the path to the directory:
python cli.py <path-to-directory>--format {json,markdown}: Choose the output format (default is JSON). Markdown mode prints a clean table and list of reviewable findings.--fail-on-high: Instructs the CLI to exit with a non-zero status code (exit code 1) if any "High" confidence secrets are found. This is highly recommended when running inside CI/CD pipelines (e.g. Jenkins, GitLab CI).
Scan and print a markdown summary, failing the CI build if high-confidence secrets are discovered:
python cli.py ./my_project --format markdown --fail-on-highTo test the scanner's context-aware false-positive reduction, we benchmarked it against three large, real-world Python repositories (boto3, aws-cli, and stripe-python) known for containing substantial test fixtures and sample documentation. We measured "Raw Matches" (findings caught by naive regex patterns) against "Reviewable" matches (High+Medium confidence findings) and "Suppressed" matches (Low confidence findings that the system successfully filtered out as dummy or placeholder credentials).
| Repo | Raw Matches | Reviewable (High+Medium) | Suppressed (Low) | Noise Reduced |
|---|---|---|---|---|
| boto3 | 3 | 1 | 2 | 66.7% |
| aws-cli | 306 | 126 | 180 | 58.8% |
| stripe-python | 0 | 0 | 0 | N/A |
| TOTAL | 309 | 127 | 182 | 58.9% |
The improvement in our scanner occurred in two stages. Initially, refining the base regex patterns and excluding common binary/encoded files (like .crt or .pem) reduced the total raw matches from a staggering 2,533 down to 309, successfully eliminating huge swaths of false positives caused by things like Git commit hashes and SSL certificates. On top of that cleaned baseline, our context-aware scoring layer—which analyzes file paths, surrounding keywords, and Shannon entropy—further suppressed 58.9% of the remaining matches as low-confidence noise. This leaves only 127 findings across three massive repositories that would actually require a human's attention.
It is important to note that this benchmark was conducted against three repositories of a similar type (Python SDKs/CLIs with heavy documentation), so results may vary on codebases with different naming conventions. For example, a repository that stores mock data in a folder named mock_data/ instead of tests/ would require expanding our SUSPICIOUS_PATH_SEGMENTS list to achieve the same noise reduction. Additionally, this tool currently supports 7 specific secret types and is not intended to be a wholesale replacement for comprehensive enterprise tools like Gitleaks or TruffleHog. Instead, it serves to demonstrate a highly effective, working approach to context-aware false-positive reduction.