Rate Limiting in Ruby

Here’s the translation of the Go rate limiting example to Ruby:

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Ruby supports rate limiting through threads and queues.

require 'thread'

def main
  # First we'll look at basic rate limiting. Suppose
  # we want to limit our handling of incoming requests.
  # We'll serve these requests off a queue of the same name.
  requests = Queue.new
  5.times { |i| requests.push(i + 1) }

  # This limiter will receive a value every 200 milliseconds.
  # This is the regulator in our rate limiting scheme.
  limiter = Queue.new
  Thread.new do
    loop do
      sleep(0.2)
      limiter.push(true)
    end
  end

  # By blocking on a receive from the limiter queue
  # before serving each request, we limit ourselves to
  # 1 request every 200 milliseconds.
  until requests.empty?
    limiter.pop
    req = requests.pop
    puts "request #{req} #{Time.now}"
  end

  # We may want to allow short bursts of requests in
  # our rate limiting scheme while preserving the
  # overall rate limit. We can accomplish this by
  # using a SizedQueue. This bursty_limiter
  # queue will allow bursts of up to 3 events.
  bursty_limiter = SizedQueue.new(3)

  # Fill up the queue to represent allowed bursting.
  3.times { bursty_limiter.push(Time.now) }

  # Every 200 milliseconds we'll try to add a new
  # value to bursty_limiter, up to its limit of 3.
  Thread.new do
    loop do
      sleep(0.2)
      bursty_limiter.push(Time.now) rescue nil
    end
  end

  # Now simulate 5 more incoming requests. The first
  # 3 of these will benefit from the burst capability
  # of bursty_limiter.
  bursty_requests = Queue.new
  5.times { |i| bursty_requests.push(i + 1) }

  until bursty_requests.empty?
    bursty_limiter.pop
    req = bursty_requests.pop
    puts "request #{req} #{Time.now}"
  end
end

main

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

$ ruby rate_limiting.rb
request 1 2023-05-26 12:34:56 +0000
request 2 2023-05-26 12:34:56 +0000
request 3 2023-05-26 12:34:56 +0000
request 4 2023-05-26 12:34:57 +0000
request 5 2023-05-26 12:34:57 +0000

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-05-26 12:34:57 +0000
request 2 2023-05-26 12:34:57 +0000
request 3 2023-05-26 12:34:57 +0000
request 4 2023-05-26 12:34:57 +0000
request 5 2023-05-26 12:34:58 +0000

In this Ruby version, we use Queue and SizedQueue to implement the rate limiting behavior. Threads are used to simulate the ticking behavior and to run the main logic concurrently. The sleep method is used to introduce delays, and Time.now is used to get the current time for logging purposes.