Rate Limiting in Squirrel

Here’s the translation of the rate limiting example from Go to Java:

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Java supports rate limiting using threads and timers.

First, let’s look at basic rate limiting. Suppose we want to limit our handling of incoming requests. We’ll serve these requests from a queue.

import java.util.concurrent.*;
import java.time.Instant;

public class RateLimiting {
    public static void main(String[] args) throws InterruptedException {
        BlockingQueue<Integer> requests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            requests.offer(i);
        }

        // This ScheduledExecutorService will execute a task every 200 milliseconds.
        // This is the regulator in our rate limiting scheme.
        ScheduledExecutorService limiter = Executors.newScheduledThreadPool(1);
        limiter.scheduleAtFixedRate(() -> {
            try {
                Integer req = requests.take();
                System.out.println("request " + req + " " + Instant.now());
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }, 0, 200, TimeUnit.MILLISECONDS);

        // Allow the program to run for a while
        Thread.sleep(2000);
        limiter.shutdown();

        // Now let's implement a bursty rate limiter
        BlockingQueue<Integer> burstyRequests = new ArrayBlockingQueue<>(5);
        for (int i = 1; i <= 5; i++) {
            burstyRequests.offer(i);
        }

        ScheduledExecutorService burstyLimiter = Executors.newScheduledThreadPool(1);
        Semaphore semaphore = new Semaphore(3); // Allow bursts of up to 3 events

        // Every 200 milliseconds, add a permit to the semaphore, up to a maximum of 3
        burstyLimiter.scheduleAtFixedRate(() -> {
            semaphore.release(1);
        }, 0, 200, TimeUnit.MILLISECONDS);

        // Process the bursty requests
        ExecutorService executor = Executors.newSingleThreadExecutor();
        executor.submit(() -> {
            while (!burstyRequests.isEmpty()) {
                try {
                    semaphore.acquire();
                    Integer req = burstyRequests.take();
                    System.out.println("bursty request " + req + " " + Instant.now());
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });

        // Allow the bursty limiter to run for a while
        Thread.sleep(2000);
        burstyLimiter.shutdown();
        executor.shutdown();
    }
}

Running our program, we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 2023-06-11T12:34:56.789Z
request 2 2023-06-11T12:34:56.989Z
request 3 2023-06-11T12:34:57.189Z
request 4 2023-06-11T12:34:57.389Z
request 5 2023-06-11T12:34:57.589Z

For the second batch of requests, we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

bursty request 1 2023-06-11T12:34:58.789Z
bursty request 2 2023-06-11T12:34:58.789Z
bursty request 3 2023-06-11T12:34:58.789Z
bursty request 4 2023-06-11T12:34:58.989Z
bursty request 5 2023-06-11T12:34:59.189Z

In this Java implementation, we use ScheduledExecutorService to simulate the ticking behavior, BlockingQueue to manage requests, and Semaphore to implement the bursty rate limiting. The overall structure and logic remain similar to the original example, adapted to Java’s concurrency utilities.

Waitgroups in Squirrel

Atomic Counters in Squirrel

Learn X By Example

Title here

Rate Limiting in Squirrel