Requests got popular for a simple reason. You could read a snippet of it once and basically know how to use it, which back in 2011 was not something anyone said about urllib2.
Fifteen-odd years later it's still the first thing most folks pip install, and for a lot of work it holds up fine. Where it starts creaking is volume. And the creaking gets louder slowly enough that nobody notices until a job that used to wrap up before lunch is still chugging along at 4 p.m.
One Request at a Time
The core issue is that Requests blocks. Call get() and that thread is parked until bytes come back from the server, full stop.
Say a typical response takes 400 ms. A plain loop then caps out around 9,000 calls an hour, and the box it runs on mostly twiddles its thumbs (CPU graphs for these jobs look almost flat).
Throwing a ThreadPoolExecutor at it is what nearly everyone tries next. It helps some, but the default pool only holds 10 connections per host, so 32 threads aimed at a single site will fill the logs with "Connection pool is full, discarding connection" pretty quickly.
Folks who've been through that usually go hunting for a python requests alternative around here. HTTPX and aiohttp get picked the most, both sitting on top of asyncio, so one process can hold a few thousand connections open and just react to whichever answer lands first.
Stuck on HTTP/1.1
Requests has never supported HTTP/2, and there's no sign it'll start. That's a little odd, given HTTP/2 became a standard back in 2015 and the big CDNs switched it on by default ages ago.
What you lose is multiplexing. A client making dozens of small calls to one API has to either queue them on one connection or open several, and each new TLS connection burns at least one extra round trip before any real data moves.
HTTPX has it behind a flag, http2=True (you need the httpx[http2] extra installed). Niquests, a Requests fork with almost the same API, goes a step further and does HTTP/3 too, which the IETF wrote up as RFC 9114 in June 2022.
Swapping import requests for import niquests as requests is often most of the migration. Not always, but often enough to try first.
The Missing Timeout
Out of the box, Requests will wait forever. If a server takes the connection and then goes silent, so does the worker, and it stays that way until somebody restarts it.
The standard fix is a homegrown wrapper that passes timeout=(3.05, 27) on every call, usually with a urllib3 Retry bolted onto the session. Works fine, right up until a new module calls requests.get() directly because whoever wrote it never heard about the wrapper.
HTTPX defaults to 5 seconds. aiohttp is way more relaxed with a 5-minute total, but even that eventually throws something you can catch and log.
Five seconds can be too tight for slow report endpoints, so plenty of people bump it per client. The point is the default fails loudly, which beats a worker quietly sitting there for three days.
Proxy Pools
Anyone running scrapers or uptime checks at scale is probably routing traffic through proxies, both to stay under rate limits and to keep thousands of requests from tracing back to one IP. HTTPX and aiohttp take a proxy per client, so twenty clients on twenty IPs inside one script is no big deal.
With Requests you'd need a Session per proxy plus threads to drive them all. Doable, but when one of them hangs, figuring out which one is a chore, especially with a stack dump full of near-identical threads.
The async version is tidier. An asyncio.Semaphore keeps things to roughly 50 in-flight requests, and a client whose proxy starts getting 429s can sleep for 30 seconds without stalling anybody else.
So Is Requests Done?
Nope. Little scripts, CLI tools, anything making a few hundred calls a day: it's still the nicest thing to read, and nothing above really bites at that size.
Services that live and breathe HTTP are another matter. Porting one small, boring service to HTTPX or Niquests first is a cheap way to find out if the switch is worth it, before anybody commits to touching the big stuff.
