|
1 | | -# Locust Compare |
| 1 | +# Python Tools |
2 | 2 |
|
3 | | -Compare performance results between two Locust runs and show changes relative to a base run. Works with both Locust CSV `report.csv` outputs and the per-feature HTML reports generated by the Locust web UI. |
| 3 | +A collection of small, independent Python utilities. Each tool is self-contained with its own dependencies and can be installed/run independently. |
4 | 4 |
|
5 | | -## Features |
| 5 | +## Tools |
6 | 6 |
|
7 | | -- Compare any two runs (base vs. current). |
8 | | -- Parses CSV `report.csv` for aggregated and per-endpoint metrics. |
9 | | -- Parses per-feature `.html` pages and compares the latest history sample. |
10 | | -- Outputs human-readable tables, markdown with emoji indicators, or machine-friendly JSON. |
11 | | - |
12 | | -## Requirements |
13 | | - |
14 | | -- Python 3.8+ (no third-party dependencies). |
| 7 | +| Tool | Description | |
| 8 | +|------|-------------| |
| 9 | +| [locust-compare](tools/locust-compare/) | Compare performance metrics between two Locust runs | |
15 | 10 |
|
16 | 11 | ## Installation |
17 | 12 |
|
18 | | -### With uvx (recommended) |
19 | | - |
20 | | -Run directly from GitHub without cloning: |
21 | | - |
22 | | -```bash |
23 | | -uvx --from git+https://github.com/dev-ankit/locust-compare.git locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 |
24 | | -``` |
25 | | - |
26 | | -Or from a local directory: |
27 | | - |
28 | | -```bash |
29 | | -uvx --from . locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 |
30 | | -``` |
31 | | - |
32 | | -Or from a cloned repository: |
33 | | - |
34 | | -```bash |
35 | | -git clone https://github.com/dev-ankit/locust-compare.git |
36 | | -cd locust-compare |
37 | | -uvx --from . locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 |
38 | | -``` |
39 | | - |
40 | | -Once published to PyPI, you can run without any prefix: |
41 | | - |
42 | | -```bash |
43 | | -uvx locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 |
44 | | -``` |
45 | | - |
46 | | -### With pip |
47 | | - |
48 | | -```bash |
49 | | -pip install . |
50 | | -locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 |
51 | | -``` |
52 | | - |
53 | | -### Direct execution |
54 | | - |
55 | | -```bash |
56 | | -python3 compare_runs.py test_runs/HTML-Report-292 test_runs/HTML-Report-294 |
57 | | -``` |
58 | | - |
59 | | -## Quick Start |
60 | | - |
61 | | -- Compare two run directories (each containing a `report.csv` and HTML files): |
62 | | - |
63 | | -```bash |
64 | | -locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 |
65 | | -``` |
66 | | - |
67 | | -- Compare two specific CSV files: |
| 13 | +Each tool can be installed independently using `uvx` directly from GitHub: |
68 | 14 |
|
69 | 15 | ```bash |
70 | | -locust-compare test_runs/HTML-Report-292/report.csv test_runs/HTML-Report-294/report.csv |
| 16 | +uvx --from 'git+https://github.com/dev-ankit/python-tools.git#subdirectory=tools/<tool-name>' <tool-name> [args] |
71 | 17 | ``` |
72 | 18 |
|
73 | | -- JSON output for scripting: |
| 19 | +For example: |
74 | 20 |
|
75 | 21 | ```bash |
76 | | -locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 -o json |
77 | | -``` |
78 | | - |
79 | | -- Markdown output with emoji indicators (✅ better, ❌ worse, ➖ same): |
80 | | - |
81 | | -``` |
82 | | -python3 compare_runs.py test_runs/HTML-Report-292 test_runs/HTML-Report-294 -o markdown |
| 22 | +uvx --from 'git+https://github.com/dev-ankit/python-tools.git#subdirectory=tools/locust-compare' locust-compare <base_dir> <current_dir> |
83 | 23 | ``` |
84 | 24 |
|
85 | | -- Colorize text output (green=better, red=worse): |
| 25 | +Or clone and run locally: |
86 | 26 |
|
87 | 27 | ```bash |
88 | | -locust-compare test_runs/HTML-Report-292 test_runs/HTML-Report-294 --color |
| 28 | +git clone https://github.com/dev-ankit/python-tools.git |
| 29 | +cd python-tools/tools/<tool-name> |
| 30 | +uvx --from . <tool-name> [args] |
89 | 31 | ``` |
90 | 32 |
|
91 | | -Exit code is `0` on success and `1` on error. |
92 | | - |
93 | | -## What It Compares |
94 | | - |
95 | | -From CSV `report.csv` (Aggregated and each request row): |
96 | | -- Requests/s, Request Count, Failure Count |
97 | | -- Average, Median, Min, Max Response Time |
98 | | -- Percentiles: 50%, 66%, 75%, 80%, 90%, 95%, 98%, 99%, 99.9%, 99.99%, 100% (if present) |
99 | | - |
100 | | -From HTML feature pages (last entry in `window.templateArgs.history`): |
101 | | -- Requests/s (`current_rps`) |
102 | | -- Average Response Time (`total_avg_response_time`) |
103 | | -- 50% (`response_time_percentile_0.5`) |
104 | | -- 95% (`response_time_percentile_0.95`) |
| 33 | +## Repository Structure |
105 | 34 |
|
106 | | -If a metric is not available for an item, it is shown as `-`. |
107 | | - |
108 | | -## Example Output (truncated) |
109 | | - |
110 | | -<img width="598" height="255" alt="image" src="https://github.com/user-attachments/assets/f5394045-6d1e-498e-aa3f-624928ec70a7" /> |
111 | | - |
112 | | - |
113 | | -## Markdown Output Example |
114 | | - |
115 | | -The `-o markdown` flag produces markdown tables with emoji indicators for verdicts: |
116 | | - |
117 | | -```markdown |
118 | | -## Aggregated |
119 | | - |
120 | | -| Metric | Base | Current | Diff | % Change | Verdict | |
121 | | -| --- | --- | --- | --- | --- | --- | |
122 | | -| Requests/s | 286.200 | 300 | +13.800 | +4.8% | ✅ | |
123 | | -| Request Count | 1500 | 1800 | +300 | +20.0% | ✅ | |
124 | | -| Failure Count | 7 | 4 | -3 | -42.9% | ✅ | |
125 | | -| Average Response Time | 85.200 | 78.500 | -6.700 | -7.9% | ✅ | |
126 | | -| 95% | 150 | 140 | -10 | -6.7% | ✅ | |
127 | 35 | ``` |
128 | | - |
129 | | -Verdict emojis: |
130 | | -- ✅ Better performance |
131 | | -- ❌ Worse performance |
132 | | -- ➖ No change |
133 | | - |
134 | | - |
135 | | -## JSON Schema |
136 | | - |
137 | | -The `-o json` output is a single JSON object containing keys for each compared item. |
138 | | - |
139 | | -- CSV items use their request name; the aggregated row is keyed as `Aggregated`. |
140 | | -- HTML feature pages are keyed as `HTML:<feature_file_stem>`. |
141 | | - |
142 | | -Each item maps metric names to an object with: |
143 | | - |
144 | | -``` |
145 | | -{ |
146 | | - "base": number | null, |
147 | | - "current": number | null, |
148 | | - "diff": number | null, |
149 | | - "pct_change": number | null |
150 | | -} |
| 36 | +python-tools/ |
| 37 | +├── README.md # This file |
| 38 | +├── LICENSE # MIT License (shared) |
| 39 | +├── .github/ |
| 40 | +│ └── workflows/ # CI/CD pipelines |
| 41 | +└── tools/ |
| 42 | + └── locust-compare/ # Locust performance comparison tool |
| 43 | + ├── compare_runs.py |
| 44 | + ├── pyproject.toml |
| 45 | + ├── README.md |
| 46 | + └── tests/ |
151 | 47 | ``` |
152 | 48 |
|
153 | | -Example (truncated): |
| 49 | +## Adding a New Tool |
154 | 50 |
|
155 | | -``` |
156 | | -{ |
157 | | - "Aggregated": { |
158 | | - "Requests/s": {"base": 268.623, "current": 196.786, "diff": -71.836, "pct_change": -26.72}, |
159 | | - "Average Response Time": {"base": 71.801, "current": 98.069, ...} |
160 | | - }, |
161 | | - "HTML:conferences_widget_all_lists": { |
162 | | - "Requests/s": {"base": 271.5, "current": 189.8, ...}, |
163 | | - "95%": {"base": 160, "current": 190, ...} |
164 | | - } |
165 | | -} |
166 | | -``` |
167 | | - |
168 | | -## Notes & Limitations |
169 | | - |
170 | | -- For HTML pages, only the last sample in `window.templateArgs.history` is compared. This typically represents the end-state of the run. If you prefer a different aggregation (mean/max), open an issue or adjust the code where noted. |
171 | | -- Request Count and Failure Count are not available from HTML pages and are displayed as `-`. |
172 | | -- If the base value is `0` or missing, percent change is shown as `-`. |
173 | | -- The tool skips non-feature HTML pages such as `htmlpublisher-wrapper.html`. |
174 | | -- The tool prints a Verdict column. By default, it evaluates improvements as: |
175 | | - - Higher is better: `Requests/s`, `Request Count`. |
176 | | - - Lower is better: all response-time metrics and percentiles, `Failure Count`, `Failures/s`. |
177 | | - - Neutral (no verdict): other metrics (e.g., `Average Content Size`). |
178 | | - |
179 | | -## Repository Layout |
180 | | - |
181 | | -``` |
182 | | -. |
183 | | -├── compare_runs.py # CLI tool |
184 | | -└── test_runs/ # Sample Locust outputs for trying the tool |
185 | | - ├── HTML-Report-292/ |
186 | | - └── HTML-Report-294/ |
187 | | -``` |
| 51 | +1. Create a new directory under `tools/`: `tools/your-tool-name/` |
| 52 | +2. Add your tool's source files and a `pyproject.toml` |
| 53 | +3. Add a `README.md` with usage instructions |
| 54 | +4. Update the CI workflow if needed |
| 55 | +5. Add an entry to the Tools table in this README |
188 | 56 |
|
189 | | -## Contributing |
| 57 | +## License |
190 | 58 |
|
191 | | -Small and simple by design. If you need additional metrics, output formats, or aggregation modes, feel free to extend `compare_runs.py` or open a PR. |
| 59 | +MIT License - see [LICENSE](LICENSE) for details. |
0 commit comments