asyncio.gather(*(call(x) for x in items)) starts every call at once: with a thousand prompts, that's a thousand simultaneous requests to an LLM API, and an immediate wall of 429s. Cap it with asyncio.Semaphore(n): each task waits for a slot before it starts. Give each call a timeout with asyncio.timeout(), and pass return_exceptions=True so one failure doesn't cancel the rest. The order matters: the timeout must go inside the semaphore. Put it outside, and the clock starts while the task is still queueing, so tasks time out without ever running; in my test, 8 of 10 did. Here's the correct version in a dozen lines, tested on Python 3.11 and later.
The function
```python title="bounded.py"
import asyncio
async def map_limit(items, limit: int, worker, timeout: float | None = None):
"""Run worker(item) for every item, at most limit at once, each with its own timeout.
Returns results in input order; failures come back as exceptions instead of cancelling the rest."""
Discussion
Get the discussion rolling
A single comment can start something great.