Rate Limiting in Java

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Java supports rate limiting with threads, executors, and scheduled tasks.

import java.time.Instant;
import java.util.concurrent.*;

public class RateLimiting {
    public static void main(String[] args) throws InterruptedException {
        // First we'll look at basic rate limiting. Suppose
        // we want to limit our handling of incoming requests.
        // We'll serve these requests off a BlockingQueue of the same name.
        BlockingQueue<Integer> requests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            requests.offer(i);
        }

        // This ScheduledExecutorService will send a value
        // every 200 milliseconds. This is the regulator in
        // our rate limiting scheme.
        ScheduledExecutorService limiter = Executors.newSingleThreadScheduledExecutor();

        // By waiting for the limiter before serving each request,
        // we limit ourselves to 1 request every 200 milliseconds.
        limiter.scheduleAtFixedRate(() -> {
            try {
                Integer req = requests.take();
                System.out.println("request " + req + " " + Instant.now());
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }, 0, 200, TimeUnit.MILLISECONDS);

        // We may want to allow short bursts of requests in
        // our rate limiting scheme while preserving the
        // overall rate limit. We can accomplish this by
        // using a semaphore. This burstyLimiter
        // will allow bursts of up to 3 events.
        Semaphore burstyLimiter = new Semaphore(3);

        // Every 200 milliseconds we'll try to add a new
        // permit to burstyLimiter, up to its limit of 3.
        ScheduledExecutorService burstyLimiterRefresher = Executors.newSingleThreadScheduledExecutor();
        burstyLimiterRefresher.scheduleAtFixedRate(() -> {
            burstyLimiter.release();
        }, 200, 200, TimeUnit.MILLISECONDS);

        // Now simulate 5 more incoming requests. The first
        // 3 of these will benefit from the burst capability
        // of burstyLimiter.
        BlockingQueue<Integer> burstyRequests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            burstyRequests.offer(i);
        }

        ExecutorService executor = Executors.newSingleThreadExecutor();
        for (int i = 0; i < 5; i++) {
            executor.submit(() -> {
                try {
                    burstyLimiter.acquire();
                    Integer req = burstyRequests.take();
                    System.out.println("request " + req + " " + Instant.now());
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            });
        }

        // Wait for all tasks to complete
        executor.shutdown();
        executor.awaitTermination(10, TimeUnit.SECONDS);
        limiter.shutdown();
        burstyLimiterRefresher.shutdown();
    }
}

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 2023-06-01T10:15:00.123Z
request 2 2023-06-01T10:15:00.323Z
request 3 2023-06-01T10:15:00.523Z
request 4 2023-06-01T10:15:00.723Z
request 5 2023-06-01T10:15:00.923Z

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-06-01T10:15:01.123Z
request 2 2023-06-01T10:15:01.123Z
request 3 2023-06-01T10:15:01.123Z
request 4 2023-06-01T10:15:01.323Z
request 5 2023-06-01T10:15:01.523Z

In this Java version, we use BlockingQueue instead of channels, ScheduledExecutorService for timed events, and a Semaphore for the bursty limiter. The overall logic and rate limiting concept remain the same, but the implementation is adapted to Java’s concurrency utilities.

查看推荐产品

Comments powered by Disqus