Token buckets, sliding windows and secure key generation
A public API without rate limiting is an open invitation for abuse. Token bucket and sliding window algorithms cap the request load per client, API keys enable differentiated limits, and abuse prevention patterns detect and block malicious behavior before it hits the infrastructure.
Table of Contents
- 1. Why rate limiting is mandatory for every production API
- 2. Token bucket vs. sliding window vs. fixed window
- 3. Symfony RateLimiter: configuration and integration
- 4. API key generation, storage and validation
- 5. Abuse prevention: detecting and blocking patterns
- 6. Rate limit headers: RFC 6585 and the RateLimit header fields
- 7. Differentiated limiting strategies by endpoint and tier
- 8. Comparing rate limiting strategies
- 9. Summary
- 10. FAQ
1. Why rate limiting is mandatory for every production API
Rate limiting protects API infrastructure against three classes of problems: DoS attacks (deliberate), broken clients (retry loops without backoff), and viral growth (suddenly high load from a new consumer). Without rate limiting, a single broken client can render the entire infrastructure unusable for every other client. This is not a theoretical risk: in practice, tight-loop bugs in clients (retrying immediately after an error instead of backing off) are a common cause of API outages.
Rate limiting alone is not sufficient. It must be combined with sensible limits, transparent response headers and a clear response to exceeding those limits (429 Too Many Requests). Clients need to know how many requests they have left, when the limit resets and how to adjust their request rate. API keys enable differentiated limits: a free tier user gets 100 requests per hour, a premium user gets 10,000. The Symfony RateLimiter component provides all the necessary building blocks for production-grade rate limiting.
2. Token bucket vs. sliding window vs. fixed window
The token bucket algorithm is the most flexible rate limiting approach. Each client has a "bucket" with a maximum capacity (e.g. 100 tokens). Tokens are refilled per time unit (e.g. 10 per second). Every request consumes tokens. If the bucket is empty, the request is rejected. The advantage: short bursts are allowed as long as the bucket is full. A client can therefore send 100 requests immediately if it has waited long enough. This is fair to clients that periodically generate a lot of load.
The sliding window counts requests within a moving time window. A request always counts against the last hour, no matter exactly when it arrived. This prevents the "burst at boundary" effect of the fixed window, where a client can generate double the load shortly before and shortly after the reset point. The fixed window (e.g. 1000 requests per hour, reset every full hour) is simple to implement and communicate, but has the boundary effect. For most APIs, the sliding window is the best balance of fairness and implementation complexity.
# config/packages/rate_limiter.yaml: Symfony RateLimiter configuration
framework:
rate_limiter:
# Token bucket: burst-friendly, refills over time
api_anonymous:
policy: token_bucket
limit: 60 # max tokens in bucket
rate:
interval: '1 minute'
amount: 10 # tokens added per interval
# Sliding window: fair, no boundary burst
api_authenticated:
policy: sliding_window
limit: 1000
interval: '1 hour'
# Fixed window per API key tier
api_premium:
policy: fixed_window
limit: 10000
interval: '1 hour'
# Strict limit for write endpoints
api_write:
policy: sliding_window
limit: 100
interval: '1 minute'
# Login endpoint: very strict
login_attempts:
policy: fixed_window
limit: 5
interval: '15 minutes'
3. Symfony RateLimiter: configuration and integration
The symfony/rate-limiter component integrates cleanly via an event listener or directly in controller methods. The recommended approach for APIs: a RateLimitSubscriber as a KernelEvents::REQUEST listener that checks before the controller runs. The limiter key is built from the client identity, an IP address for anonymous clients, an API key hash for authenticated clients. When the limit is exceeded, a 429 response is returned immediately, before the controller is reached.
For the storage backend, Redis is recommended for production environments, because Redis provides the atomic increment operations required for correct sliding window implementations. Symfony configures the storage via the cache.pool mechanism. A common mistake: using the in-memory store for rate limiting in multi-instance deployments, since each app instance then counts separately, which makes rate limiting ineffective. Redis as shared storage is mandatory.
['onRequest', 10]];
}
public function onRequest(RequestEvent $event): void
{
$request = $event->getRequest();
// Only limit API routes
if (!str_starts_with($request->getPathInfo(), '/api/')) {
return;
}
$apiKey = $request->headers->get('X-API-Key');
if ($apiKey !== null) {
$limiter = $this->apiAuthenticatedLimiter->create(hash('sha256', $apiKey));
} else {
$limiter = $this->apiAnonymousLimiter->create($request->getClientIp());
}
$limit = $limiter->consume(1);
if (!$limit->isAccepted()) {
$retryAfter = $limit->getRetryAfter()->getTimestamp() - time();
$event->setResponse(new JsonResponse([
'type' => 'https://mironsoft.de/errors/rate-limit-exceeded',
'title' => 'Too Many Requests',
'status' => 429,
'detail' => sprintf(
'Rate limit exceeded. Retry after %d seconds.',
max(1, $retryAfter)
),
], 429, [
'Content-Type' => 'application/problem+json',
'Retry-After' => (string) max(1, $retryAfter),
'X-RateLimit-Limit' => (string) $limit->getLimit(),
'X-RateLimit-Remaining' => '0',
'X-RateLimit-Reset' => (string) $limit->getRetryAfter()->getTimestamp(),
]));
} else {
// Attach rate limit headers to the response via request attribute
$request->attributes->set('_rate_limit', $limit);
}
}
}
4. API key generation, storage and validation
API keys must be generated cryptographically secure. In PHP that means random_bytes(32) for 256 bits of entropy, base64url-encoded for URL-safe representation. An API key is never stored in plaintext in the database. The hash (SHA-256 or bcrypt) is stored, and the key is shown to the user only once. The same principle as with passwords. For identifying the request, a prefix is used (e.g. msi_live_ for production keys) that is stored in plaintext in the database and enables a fast lookup without storing the full key.
Validation follows a two-step process: first the prefix is extracted from the key and the matching database entry is loaded, then the hash of the full key is checked against the stored hash. Redis caching of the validated key (TTL: 5 minutes) reduces database access to a minimum. Revocation is effective immediately, because cached keys must be revalidated after the TTL. Important: never use a string comparison vulnerable to timing attacks, always use hash_equals() for hash comparisons.
5. Abuse prevention: detecting and blocking patterns
Rate limiting alone is not enough for abuse prevention. An attacker can rotate through many IP addresses and elegantly bypass the per-IP rate limit. Abuse prevention requires additional signals: fingerprinting via user agent, accept header combination and TLS fingerprint; behavioral analysis (always the same endpoints in the same order, no browsing behavior); geographic anomalies (suddenly a thousand requests from the same data center). These signals are combined and an anomaly score is calculated.
Simple abuse prevention measures that show effect immediately without complex ML models: honeypot endpoints that no legitimate traffic should hit; burst detection that blocks clients sending many requests in a very short time window; account sharing detection via parallel session indicators. For implementation in Symfony, a dedicated AbuseDetectionService is suitable, accumulating metrics in Redis and evaluating thresholds. Blocked clients are kept in a Redis set with a TTL, and a 403 response with a clear explanation is returned.
repository->save($entity);
return ['prefix' => $prefix, 'key' => $fullKey];
}
/**
* Validates an API key against the stored hash.
* Uses Redis cache to reduce database load.
* Never uses == for hash comparison: uses hash_equals().
*/
public function validate(string $rawKey): ?ApiKey
{
$cacheKey = 'apikey_' . hash('sha256', $rawKey);
$cached = $this->cache->getItem($cacheKey);
if ($cached->isHit()) {
return $cached->get(); // null means invalid but cached
}
$prefix = substr($rawKey, 0, self::PREFIX_LENGTH);
$entity = $this->repository->findByPrefix($prefix);
$isValid = $entity !== null
&& hash_equals($entity->getKeyHash(), hash('sha256', $rawKey))
&& !$entity->isRevoked();
$result = $isValid ? $entity : null;
$cached->set($result)->expiresAfter(self::CACHE_TTL);
$this->cache->save($cached);
return $result;
}
public function revoke(string $keyId): void
{
$entity = $this->repository->find($keyId);
if ($entity !== null) {
$entity->revoke();
$this->repository->save($entity);
// Cache will expire naturally: revocation is eventually consistent
}
}
}
6. Rate limit headers: RFC 6585 and the RateLimit header fields
Transparent rate limit headers are essential for API consumers: they enable adaptive clients that adjust their own request rate before a 429 arrives. The IETF draft "RateLimit Header Fields for HTTP" defines three headers: RateLimit-Limit (maximum requests in the window), RateLimit-Remaining (remaining requests) and RateLimit-Reset (Unix timestamp of the next reset). In addition, Retry-After is mandatory on 429 responses, either as a number of seconds or as an HTTP date.
The Symfony RateLimiter component provides all the necessary information via the RateLimit object: getLimit(), getRemainingTokens() and getRetryAfter(). A response listener (or middleware) adds this information as headers to every API response, not just on 429, but on every response. This lets clients throttle proactively before they hit the limit. Important: with Redis clusters the values can be slightly inaccurate due to replication delays. That is acceptable and should be mentioned in the API documentation.
7. Differentiated limiting strategies by endpoint and tier
Not all endpoints carry the same load or the same abuse potential. GET endpoints for catalog data can have more generous limits than POST endpoints that create orders. Search requests with complex queries put more load on the database server than simple ID lookups. A differentiated strategy defines limit profiles per endpoint group and API tier. In Symfony this can be solved elegantly via an attribute on the controller: #[RateLimit(policy: 'api_write')] overrides the global limit for individual endpoints.
Tier-based rate limiting requires that the API key is linked to tier information. A Security Voter or a dedicated TierResolver determines the tier of the authenticated client and selects the appropriate RateLimiterFactory. Free tier: 100 req/h, Starter: 1,000 req/h, Pro: 10,000 req/h, Enterprise: custom. The limits must be clearly communicated in the API documentation, ideally per endpoint in the OpenAPI specification as an extension (x-rate-limit).
8. Comparing rate limiting strategies
The choice of rate limiting algorithm has a direct impact on fairness, implementation complexity and client experience. The following table compares the three most common algorithms across the most relevant criteria.
| Criterion | Fixed Window | Sliding Window | Token Bucket |
|---|---|---|---|
| Boundary burst | Possible (up to 2x limit briefly) | Not possible | Controlled (bucket capacity) |
| Implementation | Simple | Medium | Medium |
| Burst-friendly | No | No | Yes (up to bucket capacity) |
| Redis overhead | Low (1 key) | Medium (sorted set) | Medium (2 keys) |
| Recommendation | Login endpoints | Main API limits | Burst-tolerant endpoints |
9. Summary
Rate limiting, API key management and abuse prevention are not an afterthought feature but an integral part of a production API. The Symfony RateLimiter component with a Redis backend provides all the building blocks for token bucket, sliding window and fixed window. API keys are generated with random_bytes(32), stored as a SHA-256 hash and cached via Redis. Abuse prevention complements rate limiting with behavioral analysis and anomaly detection. Transparent rate limit headers enable adaptive clients. Differentiated limits by endpoint type and API tier make the system fair and scalable.
The most common mistake in practice: rate limiting is kept in memory per app instance instead of a shared Redis. Under horizontal scaling, the effective limit is then multiplied by the number of instances compared to what was configured. Redis as central storage is the only correct solution for multi-instance deployments.
Rate Limiting and API Keys in Symfony: the essentials at a glance
Algorithm choice
Sliding window for main API limits (no boundary burst). Token bucket for burst-tolerant endpoints. Fixed window for login endpoints.
API key security
random_bytes(32), store only the hash, hash_equals() for comparison. Redis cache with a 5-minute TTL for validation. Prefix for fast lookup.
Response headers
X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset on every response. Retry-After on 429. Enables adaptive clients.
Storage
Redis as central storage for rate limit counters. Use the in-memory store only for single-instance development environments.