Rate Limiting in Java

Here’s the translation of the Rate Limiting example from Go to Java:

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Java supports rate limiting with threads, executors, and scheduled tasks.

import java.time.Instant;
import java.util.concurrent.*;

public class RateLimiting {
    public static void main(String[] args) throws InterruptedException {
        // First we'll look at basic rate limiting. Suppose
        // we want to limit our handling of incoming requests.
        // We'll serve these requests off a BlockingQueue of the same name.
        BlockingQueue<Integer> requests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            requests.offer(i);
        }

        // This ScheduledExecutorService will send a value
        // every 200 milliseconds. This is the regulator in
        // our rate limiting scheme.
        ScheduledExecutorService limiter = Executors.newSingleThreadScheduledExecutor();

        // By waiting for the limiter before serving each request,
        // we limit ourselves to 1 request every 200 milliseconds.
        limiter.scheduleAtFixedRate(() -> {
            try {
                Integer req = requests.take();
                System.out.println("request " + req + " " + Instant.now());
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }, 0, 200, TimeUnit.MILLISECONDS);

        // We may want to allow short bursts of requests in
        // our rate limiting scheme while preserving the
        // overall rate limit. We can accomplish this by
        // using a semaphore. This burstyLimiter
        // will allow bursts of up to 3 events.
        Semaphore burstyLimiter = new Semaphore(3);

        // Every 200 milliseconds we'll try to add a new
        // permit to burstyLimiter, up to its limit of 3.
        ScheduledExecutorService burstyLimiterRefresher = Executors.newSingleThreadScheduledExecutor();
        burstyLimiterRefresher.scheduleAtFixedRate(() -> {
            burstyLimiter.release();
        }, 200, 200, TimeUnit.MILLISECONDS);

        // Now simulate 5 more incoming requests. The first
        // 3 of these will benefit from the burst capability
        // of burstyLimiter.
        BlockingQueue<Integer> burstyRequests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            burstyRequests.offer(i);
        }

        ExecutorService executor = Executors.newSingleThreadExecutor();
        for (int i = 0; i < 5; i++) {
            executor.submit(() -> {
                try {
                    burstyLimiter.acquire();
                    Integer req = burstyRequests.take();
                    System.out.println("request " + req + " " + Instant.now());
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            });
        }

        // Wait for all tasks to complete
        executor.shutdown();
        executor.awaitTermination(10, TimeUnit.SECONDS);
        limiter.shutdown();
        burstyLimiterRefresher.shutdown();
    }
}

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 2023-06-01T10:15:00.123Z
request 2 2023-06-01T10:15:00.323Z
request 3 2023-06-01T10:15:00.523Z
request 4 2023-06-01T10:15:00.723Z
request 5 2023-06-01T10:15:00.923Z

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-06-01T10:15:01.123Z
request 2 2023-06-01T10:15:01.123Z
request 3 2023-06-01T10:15:01.123Z
request 4 2023-06-01T10:15:01.323Z
request 5 2023-06-01T10:15:01.523Z

In this Java version, we use BlockingQueue instead of channels, ScheduledExecutorService for timed events, and a Semaphore for the bursty limiter. The overall logic and rate limiting concept remain the same, but the implementation is adapted to Java’s concurrency utilities.