A high-performance, lightweight background proxy rotator and forwarder built in Go with zero external dependencies. The tool runs as a local proxy gateway (127.0.0.1:8080), automatically scrapes or loads custom proxy lists, continuously benchmarks them for speed/downtime in the background, and forwards client connections via the optimal upstream proxy.
To keep the tool fast, modular, and thread-safe, the architecture is split into a Data Forwarding Path (handling client requests) and a Maintenance Loop (running concurrent benchmark pings) running in parallel.
flowchart LR
%% Active Data Path (Top Track)
Client([Client: Browser/Curl]) -->|1. Requests| Server[Local Proxy Server: 8080]
Server -->|2. Tunnel Auth & Connect| Upstream[Upstream Proxy: HTTP/SOCKS5]
Upstream -->|3. Route Data| Website([Target Website])
%% Background Management (Bottom Track)
Pool[(Proxy Pool)] -.->|Provides Best Nodes| Server
Checker[Background Checker] -->|Populate & Prune| Pool
Checker -->|Verify TLS & Latency| Upstream
Scraper[Scraper: File/URL] -->|Load Raw Proxies| Checker
Standard proxy rotators use simple round-robin selection (e.g., Request 1 gets Proxy A, Request 2 gets Proxy B). While this rotates IPs rapidly, it breaks websites in browsers because:
- A webpage loading its assets (images, CSS, JS) from 10 different IP addresses simultaneously triggers anti-scraping and session-hijack protections.
- Persistent connections (HTTP Keep-Alives) are broken, forcing a new TCP/TLS handshake for every single file.
Our Solution: The server cleans the target hostname (e.g., stripping api.github.com:443 down to github.com) and hashes it to pick a stable index from the top-performing proxies. All requests to github.com will use the same proxy, while requests to google.com will route through another. This preserves cookies, session states, and connection speeds.
Creating a new network transport for every HTTP request is highly expensive. We implement a thread-safe client cache (transports map) in the server:
- An
http.Clientandhttp.Transportare instantiated once per upstream proxy and cached. - Go's underlying transport keeps idle TCP connections to the upstream proxy alive (
MaxIdleConns = 100,MaxIdleConnsPerHost = 10). - Subsequent requests to the same proxy skip the TCP 3-way handshake and TLS negotiation entirely, cutting latency by up to 70%.
Proxies are ranked dynamically in the pool based on their performance using the following score formula:
- Lowest score is best. If a proxy has a low latency but starts failing, its score increases rapidly, pushing it down the list.
- Transparent Failover: If a proxy fails to connect during a client request, our server intercepts the error, penalizes the proxy in the pool, chooses the next best proxy matching the hostname hash, and retries the request. The client application sees zero connection timeouts or connection reset errors.
Our custom proxy server implements connection-level handshakes directly over TCP raw sockets:
- HTTP Proxy Basic Auth: Base64-encodes your credentials and injects the
Proxy-Authorization: Basic <credentials>header during theCONNECThandshake. - SOCKS5 Authentication (RFC 1929): During the SOCKS5 handshake, our dialer negotiates the authentication method (
0x02). It writes a subnegotiation packet containing the username and password length, followed by their raw bytes. It reads the response, checks the status code (0x00for success), and then proceeds to dial the destination.
The parser automatically detects, extracts credentials, and standardizes schemes from multiple formats:
| Format | Example | Details |
|---|---|---|
| Standard IP/Port | 192.168.1.1:8080 |
Defaults to HTTP protocol |
| Protocol Specified | socks5://1.2.3.4:1080 |
Forces SOCKS5 proxy protocol |
| Inline Auth (URL) | http://user:pass@1.2.3.4:80 |
HTTP proxy with basic auth |
| Double Colon Auth | 1.2.3.4:80:user:pass |
Highly common format for paid lists |
| SOCKS5 with Auth | socks5://user:pass@1.2.3.4:1080 |
SOCKS5 proxy with RFC 1929 auth |
Configure the daemon using standard CLI flags:
./proxy_switcher -h| Flag | Default | Description |
|---|---|---|
-addr |
127.0.0.1:8080 |
Local address for the rotating server to listen on |
-file |
none | Path to a local text file containing proxies (one per line) |
-url |
none | Custom API URL to pull proxies from |
-timeout |
5s |
Maximum time allowed to establish a connection with a proxy |
-max-latency |
2s |
Maximum latency allowed for a proxy to stay in the pool |
-check-interval |
1m |
Frequency of background active pool benchmark tests |
-scrape-interval |
10m |
Frequency of scraping new proxy entries |
-test-url |
https://clients3.google.com/generate_204 |
HTTPS URL used for latency checks and TLS validation |
-max-failures |
3 |
Consecutive check failures before a proxy is permanently evicted |
Build a single standalone optimized executable:
go build -o proxy_switcher .Tip
For web browsers, we recommend a strict latency limit (-max-latency 1.5s) to keep page loads fast.
- Using default public lists (Scraping mode):
./proxy_switcher -addr 127.0.0.1:8080 -max-latency 2s
- Using your private paid proxies file:
./proxy_switcher -addr 127.0.0.1:8080 -file my_proxies.txt -max-latency 1.5s
- Using your proxy provider API endpoint:
./proxy_switcher -addr 127.0.0.1:8080 -url "https://api.paid-provider.com/get?auth=xyz"
Use curl to send a request through your local rotator:
curl -x http://127.0.0.1:8080 https://httpbin.org/ipSend it a few times. You will notice that:
- The returned IP address switches/rotates.
- The requests are fast (since high-latency and hijacking proxies are discarded).
Verify proxy scoring, sorting, and eviction logic:
go test -v ./...