Rate Limiting in Karel

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Java supports rate limiting through various mechanisms, including threads, concurrent data structures, and schedulers.

import java.time.Instant;
import java.util.concurrent.*;

public class RateLimiting {
    public static void main(String[] args) throws InterruptedException {
        // First we'll look at basic rate limiting. Suppose
        // we want to limit our handling of incoming requests.
        // We'll serve these requests off a BlockingQueue.
        BlockingQueue<Integer> requests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            requests.offer(i);
        }

        // This ScheduledExecutorService will execute a task
        // every 200 milliseconds. This is the regulator in
        // our rate limiting scheme.
        ScheduledExecutorService limiter = Executors.newScheduledThreadPool(1);
        limiter.scheduleAtFixedRate(() -> {
            try {
                int req = requests.take();
                System.out.println("request " + req + " " + Instant.now());
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }, 0, 200, TimeUnit.MILLISECONDS);

        // We may want to allow short bursts of requests in
        // our rate limiting scheme while preserving the
        // overall rate limit. We can accomplish this by
        // using a semaphore. This burstyLimiter
        // will allow bursts of up to 3 events.
        Semaphore burstyLimiter = new Semaphore(3);

        // Every 200 milliseconds we'll try to add a new
        // permit to burstyLimiter, up to its limit of 3.
        ScheduledExecutorService burstyRefresher = Executors.newScheduledThreadPool(1);
        burstyRefresher.scheduleAtFixedRate(() -> {
            burstyLimiter.release();
        }, 200, 200, TimeUnit.MILLISECONDS);

        // Now simulate 5 more incoming requests. The first
        // 3 of these will benefit from the burst capability
        // of burstyLimiter.
        BlockingQueue<Integer> burstyRequests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            burstyRequests.offer(i);
        }

        ExecutorService burstyWorker = Executors.newSingleThreadExecutor();
        for (int i = 0; i < 5; i++) {
            final int req = burstyRequests.take();
            burstyWorker.submit(() -> {
                try {
                    burstyLimiter.acquire();
                    System.out.println("request " + req + " " + Instant.now());
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            });
        }

        // Allow time for all requests to be processed
        Thread.sleep(2000);

        // Shutdown the executors
        limiter.shutdown();
        burstyRefresher.shutdown();
        burstyWorker.shutdown();
    }
}

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 2023-06-01T10:15:00.123Z
request 2 2023-06-01T10:15:00.323Z
request 3 2023-06-01T10:15:00.523Z
request 4 2023-06-01T10:15:00.723Z
request 5 2023-06-01T10:15:00.923Z

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-06-01T10:15:01.123Z
request 2 2023-06-01T10:15:01.123Z
request 3 2023-06-01T10:15:01.123Z
request 4 2023-06-01T10:15:01.323Z
request 5 2023-06-01T10:15:01.523Z

This Java implementation uses ScheduledExecutorService for timing, BlockingQueue for managing requests, and Semaphore for implementing the bursty limiter. The overall structure and behavior are similar to the original example, adapted to Java’s concurrency utilities.