Skip to content

requests-whaor [ri-kwests hawr]

Requests + Docker + HAproxy + Tor

Requests **W**ith **H**igh **A**vailability **O**nion **R**outer. For the filthiest web scrapers that have no time for rate-limits.

Black GitHub

Overview

requests-whaor proxies GET requests through a local Docker network of TOR circuits. It takes care of starting and stopping a pool of TOR proxies behind an HAproxy load balancer, which acts as a round robin reverse proxy network. This will give each request a new IP address. If you start having issues with the initial pool of IPs, requests-whaor can gather a new pool of IP addresses by restarting all TOR containers.

Install with pip

pip install requests-whaor

Install with Poetry

poetry add requests-whaor

>> Quickstart / Docs <<

Projects to highlight.

Useful Docker commands.

If things get out of hand you may need these commands for debugging or killing containers.

docker ps -q --filter ancestor=osminogin/tor-simple | xargs -L 1 docker logs --follow

docker ps -q --filter ancestor=osminogin/haproxy | xargs -L 1 docker logs --follow

docker stop $(docker ps -q --filter ancestor=osminogin/tor-simple)

docker stop $(docker ps -q --filter ancestor=haproxy)

docker network rm $(docker network ls -q -f name=whaornet)

TODO

  • Testing.
  • More request methods if requested.
  • Options for using different Tor containers.
  • Options for different load balancer containers.

Contact Information

Telegram = Twitter = Tradingview = Discord = @dgnsrekt

Email = dgnsrekt@pm.me