Rate Limiting in Java
Here’s the translation of the Rate Limiting example from Go to Java:
Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Java supports rate limiting with threads, executors, and scheduled tasks.
import java.time.Instant;
import java.util.concurrent.*;
public class RateLimiting {
public static void main(String[] args) throws InterruptedException {
// First we'll look at basic rate limiting. Suppose
// we want to limit our handling of incoming requests.
// We'll serve these requests off a BlockingQueue of the same name.
BlockingQueue<Integer> requests = new ArrayBlockingQueue<>(5);
for (int i = 1; i <= 5; i++) {
requests.offer(i);
}
// This ScheduledExecutorService will send a value
// every 200 milliseconds. This is the regulator in
// our rate limiting scheme.
ScheduledExecutorService limiter = Executors.newSingleThreadScheduledExecutor();
// By waiting for the limiter before serving each request,
// we limit ourselves to 1 request every 200 milliseconds.
limiter.scheduleAtFixedRate(() -> {
try {
Integer req = requests.take();
System.out.println("request " + req + " " + Instant.now());
} catch (InterruptedException e) {
e.printStackTrace();
}
}, 0, 200, TimeUnit.MILLISECONDS);
// We may want to allow short bursts of requests in
// our rate limiting scheme while preserving the
// overall rate limit. We can accomplish this by
// using a semaphore. This burstyLimiter
// will allow bursts of up to 3 events.
Semaphore burstyLimiter = new Semaphore(3);
// Every 200 milliseconds we'll try to add a new
// permit to burstyLimiter, up to its limit of 3.
ScheduledExecutorService burstyLimiterRefresher = Executors.newSingleThreadScheduledExecutor();
burstyLimiterRefresher.scheduleAtFixedRate(() -> {
burstyLimiter.release();
}, 200, 200, TimeUnit.MILLISECONDS);
// Now simulate 5 more incoming requests. The first
// 3 of these will benefit from the burst capability
// of burstyLimiter.
BlockingQueue<Integer> burstyRequests = new ArrayBlockingQueue<>(5);
for (int i = 1; i <= 5; i++) {
burstyRequests.offer(i);
}
ExecutorService executor = Executors.newSingleThreadExecutor();
for (int i = 0; i < 5; i++) {
executor.submit(() -> {
try {
burstyLimiter.acquire();
Integer req = burstyRequests.take();
System.out.println("request " + req + " " + Instant.now());
} catch (InterruptedException e) {
e.printStackTrace();
}
});
}
// Wait for all tasks to complete
executor.shutdown();
executor.awaitTermination(10, TimeUnit.SECONDS);
limiter.shutdown();
burstyLimiterRefresher.shutdown();
}
}
Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.
request 1 2023-06-01T10:15:00.123Z
request 2 2023-06-01T10:15:00.323Z
request 3 2023-06-01T10:15:00.523Z
request 4 2023-06-01T10:15:00.723Z
request 5 2023-06-01T10:15:00.923Z
For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.
request 1 2023-06-01T10:15:01.123Z
request 2 2023-06-01T10:15:01.123Z
request 3 2023-06-01T10:15:01.123Z
request 4 2023-06-01T10:15:01.323Z
request 5 2023-06-01T10:15:01.523Z
In this Java version, we use BlockingQueue
instead of channels, ScheduledExecutorService
for timed events, and a Semaphore
for the bursty limiter. The overall logic and rate limiting concept remain the same, but the implementation is adapted to Java’s concurrency utilities.