Capacity and Retry-After
No per-key rate limit. When we are full you get 429 with Retry-After, free.
There is no per-key rate limit and no concurrency cap. Requests from all keys are served first in, first out, as capacity allows.
When we are full
If we cannot answer within the time limit, you get 429 over_capacity straight away. It has a Retry-After header and the same number in retry_after_s. Nothing is charged.
HTTP/1.1 429 Too Many Requests
Retry-After: 12{"success":false,"error":"over_capacity","errorStatus":429,"message":"no capacity to answer within the time limit; retry after retry_after_s seconds","credits_charged":0,"retry_after_s":12}Wait that many seconds and retry. The value is our estimate of when capacity frees up. During a restart, or when too many requests are already held open, it is 30 seconds.
Timeouts
Set your client timeout above 180 seconds
We hold a request open for up to 180 seconds. Past that you get 504 timeout, free. A client that gives up sooner sees its own timeout instead of our answer.
| client | setting |
|---|---|
| curl | --max-time 200 |
Python requests | timeout=200 |
Node.js fetch | signal: AbortSignal.timeout(200_000) |
Go net/http | http.Client{Timeout: 200 * time.Second} |
Cold starts
An endpoint with no capacity running must start some first. The first request after a quiet period can take over a minute.
Related limits
- Free-answer limit: 100 free answers per 10 minutes per key.
- Keyword search can answer
503 search_unavailablewithRetry-After: 60.