Rate Limiting

How It Works#

The limit is applied per client, identified by the full source IP address of the connection, on IPv4 and IPv6 alike. A client is therefore an address, not a host: a host that sends from several addresses gets a budget for each of them, and only the global limit holds them together. This isolates the offender: an address that sends too many requests exhausts only its own budget, so no other address gets throttled on its account. That holds within one subnet as well: every address in it counts as a client of its own, also on IPv6, where many hosts usually share one /64 network. On a large machine, a single budget covers much of what the database can handle, so a runaway client can still slow the others down. Forwarded headers such as X-Forwarded-For are deliberately not trusted, because EventSourcingDB is meant to run in a protected network with controlled clients. So clients that reach EventSourcingDB through the same reverse proxy, API gateway, or NAT count as one client and share its budget; raise the limit as described below if they need more.

Requests are weighted, because writing is markedly more expensive than reading: a write counts as several requests, while a read counts as one. A request that no route takes, such as one for a path that does not exist or with a method that its route does not take, counts as a read. So does a request whose path is not in its clean form, such as one with // or /./, which EventSourcingDB answers with a redirect to the clean form, so a client that follows the redirect pays for both requests. The limit you configure is therefore best understood as a budget of read-equivalent requests per second.

When a client exceeds its limit, the database responds with 429 Too Many Requests and a Retry-After header that tells the client how long to wait. On normal responses, the database also sends the RateLimit and RateLimit-Policy headers, so that a cooperative client can slow down before it is throttled instead of running into errors.

The remaining quota in these headers is expressed in request units, not in plain request counts – a write consumes several units, a read one. So a remaining value of, say, 4 means a read would still pass while a write would not. The Retry-After header remains the authoritative answer to when a rejected request may be retried.

Rate limiting is always active and not tied to your license; it is a safeguard, not a premium feature.

Limits for All Clients#

Besides the limit per client, two further limits apply, which you cannot configure: the global limit, which always applies, since it guards the server as a whole rather than against a single client, and a limit of open streams for each client, which goes together with the limit per client:

• A global limit of 10,000 read-equivalent requests per second for every CPU the database runs on, counted in the same way as the limit per client, which all clients share. It keeps many clients that each stay below their own limit from overwhelming the server together. The global limit exists twice, at the same size: once for the requests that carry a valid API token, and once for all other requests, those without a token or with one that is not valid, on the routes that need no token as well. A flood of requests without a valid token, from however many addresses, can therefore use up only its own global limit, and never holds back the clients with a valid token, with one exception: the files of the management UI, which a browser requests without the token, so during such a flood the UI does not load, also where the browser holds its files, since it asks for each of them again whenever it loads the UI. Only ping and health are exempt from both, in the v2 API as in the v1 API, so that a flood of requests cannot starve the probes of a healthy server; they still count against the limit per client. Every other route counts, the OpenAPI definitions, the API catalog, the files of the management UI, and the profiles of development mode included, and so does every request that no route takes.
• At most 256 open streams per client on the routes of both APIs that stream their results, and so keep a connection open for as long as they run: reading events, listing event types and subjects, observing events, running EventQL queries, creating a backup, exporting events, verifying the store, and auditing its history. Writing events, registering an event schema, restoring a backup, and importing events do not count, although the v2 API answers them as a stream as well: they answer for a single command rather than streaming results, and only one restore or import runs at a time anyway. Importing events counts as a read, as restoring a backup does. Opening such a stream counts as a request, and a client that already holds 256 of them is refused with 429 Too Many Requests and a Retry-After of one second, until one of them ends. This guards against connections that leak, such as a client that reconnects in a loop without closing its previous streams. It is part of the limit per client, so 0 switches it off together with that limit, see Configuring the Limit.

Configuring the Limit#

By default, each client may send 1,000 read-equivalent requests per second for every CPU the database runs on, so 8,000 with 8 CPUs. In a container with a CPU limit, that limit counts rather than the CPUs of the host, rounded up to whole CPUs and at least 2. If the environment variable GOMAXPROCS is set, its value counts instead. This is high enough not to interfere with ordinary workloads and low enough to contain accidents. For most deployments, you should not need to change anything.

This is an advanced setting. Only change it if you understand what you are doing and what the consequences are – setting it too high defeats the protection, while setting it too low can throttle legitimate clients.

If a legitimate workload runs into the limit, raise it above the default with the --rate-limit-requests-per-second flag, for example to twice the default on a machine with 8 CPUs:

./eventsourcingdb run [...] --rate-limit-requests-per-second 16000

The value has to be 0 or more; with a negative value, the server refuses to start, with the error you must not specify a negative rate limit. To disable the per-client limit entirely, set the value to 0:

./eventsourcingdb run [...] --rate-limit-requests-per-second 0

This switches off the limit per client as a whole, the limit of 256 open streams per client included, for example for clients that reach EventSourcingDB through the same reverse proxy, API gateway, or NAT and together need more streams. The global limit remains, from which only ping and health stay exempt, and opening a stream still counts against it.

The recommended approach is to start with the default and raise the limit only if you actually hit it, rather than guessing a higher value up front.

For More Information#

• To restrict who may access the database at all, see Security and Access Control.
• To temporarily disable writes, see Read-Only Mode.