Rate Limiting in CLIPS

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Java supports rate limiting with threads, executors, and timers.

import java.time.Instant;
import java.util.concurrent.*;

public class RateLimiting {
    public static void main(String[] args) throws InterruptedException {
        // First we'll look at basic rate limiting. Suppose
        // we want to limit our handling of incoming requests.
        // We'll serve these requests off a BlockingQueue of the
        // same name.
        BlockingQueue<Integer> requests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            requests.offer(i);
        }

        // This ScheduledExecutorService will execute a task
        // every 200 milliseconds. This is the regulator in
        // our rate limiting scheme.
        ScheduledExecutorService limiter = Executors.newScheduledThreadPool(1);

        // By waiting for the next execution before serving each request,
        // we limit ourselves to 1 request every 200 milliseconds.
        limiter.scheduleAtFixedRate(() -> {
            try {
                Integer req = requests.take();
                System.out.println("request " + req + " " + Instant.now());
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }, 0, 200, TimeUnit.MILLISECONDS);

        // We may want to allow short bursts of requests in
        // our rate limiting scheme while preserving the
        // overall rate limit. We can accomplish this by
        // using a semaphore. This burstyLimiter
        // will allow bursts of up to 3 events.
        Semaphore burstyLimiter = new Semaphore(3);

        // Every 200 milliseconds we'll try to add a new
        // permit to burstyLimiter, up to its limit of 3.
        ScheduledExecutorService burstyLimiterRefill = Executors.newScheduledThreadPool(1);
        burstyLimiterRefill.scheduleAtFixedRate(burstyLimiter::release, 200, 200, TimeUnit.MILLISECONDS);

        // Now simulate 5 more incoming requests. The first
        // 3 of these will benefit from the burst capability
        // of burstyLimiter.
        BlockingQueue<Integer> burstyRequests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            burstyRequests.offer(i);
        }

        ExecutorService burstyWorker = Executors.newSingleThreadExecutor();
        burstyWorker.execute(() -> {
            while (true) {
                try {
                    burstyLimiter.acquire();
                    Integer req = burstyRequests.take();
                    System.out.println("request " + req + " " + Instant.now());
                } catch (InterruptedException e) {
                    break;
                }
            }
        });

        // Let the program run for a while
        Thread.sleep(2000);

        // Shutdown executors
        limiter.shutdown();
        burstyLimiterRefill.shutdown();
        burstyWorker.shutdown();
    }
}

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 2023-06-08T10:15:00.123Z
request 2 2023-06-08T10:15:00.323Z
request 3 2023-06-08T10:15:00.523Z
request 4 2023-06-08T10:15:00.723Z
request 5 2023-06-08T10:15:00.923Z

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-06-08T10:15:01.123Z
request 2 2023-06-08T10:15:01.123Z
request 3 2023-06-08T10:15:01.123Z
request 4 2023-06-08T10:15:01.323Z
request 5 2023-06-08T10:15:01.523Z

This Java implementation uses ScheduledExecutorService for timed tasks, BlockingQueue for managing requests, and Semaphore for implementing the bursty limiter. The overall structure and behavior mimic the original example, demonstrating both steady rate limiting and bursty rate limiting in Java.