Rate Limiting in Logo

Our rate limiting example demonstrates how to control resource utilization and maintain quality of service. Java provides various mechanisms for implementing rate limiting, including threads and timers.

import java.time.Instant;
import java.util.concurrent.*;

public class RateLimiting {
    public static void main(String[] args) throws InterruptedException {
        // First we'll look at basic rate limiting. Suppose
        // we want to limit our handling of incoming requests.
        // We'll serve these requests off a BlockingQueue.
        BlockingQueue<Integer> requests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            requests.offer(i);
        }

        // This ScheduledExecutorService will send a value
        // every 200 milliseconds. This is the regulator in
        // our rate limiting scheme.
        ScheduledExecutorService limiter = Executors.newScheduledThreadPool(1);
        limiter.scheduleAtFixedRate(() -> {
            try {
                int req = requests.take();
                System.out.println("request " + req + " " + Instant.now());
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }, 0, 200, TimeUnit.MILLISECONDS);

        // We may want to allow short bursts of requests in
        // our rate limiting scheme while preserving the
        // overall rate limit. We can accomplish this by
        // using a semaphore. This semaphore will allow
        // bursts of up to 3 events.
        Semaphore burstyLimiter = new Semaphore(3);

        // Every 200 milliseconds we'll try to add a new
        // permit to burstyLimiter, up to its limit of 3.
        ScheduledExecutorService burstyLimiterRefill = Executors.newScheduledThreadPool(1);
        burstyLimiterRefill.scheduleAtFixedRate(() -> {
            burstyLimiter.release();
        }, 200, 200, TimeUnit.MILLISECONDS);

        // Now simulate 5 more incoming requests. The first
        // 3 of these will benefit from the burst capability
        // of burstyLimiter.
        BlockingQueue<Integer> burstyRequests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            burstyRequests.offer(i);
        }

        for (int i = 0; i < 5; i++) {
            burstyLimiter.acquire();
            int req = burstyRequests.take();
            System.out.println("request " + req + " " + Instant.now());
        }

        // Shutdown the executors
        limiter.shutdown();
        burstyLimiterRefill.shutdown();
    }
}

Running our program, we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 2023-06-01T10:15:00.123Z
request 2 2023-06-01T10:15:00.323Z
request 3 2023-06-01T10:15:00.523Z
request 4 2023-06-01T10:15:00.723Z
request 5 2023-06-01T10:15:00.923Z

For the second batch of requests, we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-06-01T10:15:01.123Z
request 2 2023-06-01T10:15:01.123Z
request 3 2023-06-01T10:15:01.123Z
request 4 2023-06-01T10:15:01.323Z
request 5 2023-06-01T10:15:01.523Z

This example demonstrates how to implement basic and bursty rate limiting in Java. We use BlockingQueue to simulate incoming requests, ScheduledExecutorService for timing, and Semaphore for bursty rate limiting. The overall structure and logic remain similar to the original example, adapted to Java’s concurrency utilities.