Rate Limiting in Crystal

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Crystal elegantly supports rate limiting with fibers, channels, and timers.

require "time"

# First we'll look at basic rate limiting. Suppose
# we want to limit our handling of incoming requests.
# We'll serve these requests off a channel of the
# same name.
requests = Channel(Int32).new(5)
5.times do |i|
  requests.send(i + 1)
end
requests.close

# This `limiter` channel will receive a value
# every 200 milliseconds. This is the regulator in
# our rate limiting scheme.
limiter = Channel(Time).new
spawn do
  loop do
    sleep 200.milliseconds
    limiter.send(Time.utc)
  end
end

# By blocking on a receive from the `limiter` channel
# before serving each request, we limit ourselves to
# 1 request every 200 milliseconds.
requests.each do |req|
  limiter.receive
  puts "request #{req} #{Time.utc}"
end

# We may want to allow short bursts of requests in
# our rate limiting scheme while preserving the
# overall rate limit. We can accomplish this by
# buffering our limiter channel. This `bursty_limiter`
# channel will allow bursts of up to 3 events.
bursty_limiter = Channel(Time).new(3)

# Fill up the channel to represent allowed bursting.
3.times { bursty_limiter.send(Time.utc) }

# Every 200 milliseconds we'll try to add a new
# value to `bursty_limiter`, up to its limit of 3.
spawn do
  loop do
    sleep 200.milliseconds
    bursty_limiter.send(Time.utc)
  end
end

# Now simulate 5 more incoming requests. The first
# 3 of these will benefit from the burst capability
# of `bursty_limiter`.
bursty_requests = Channel(Int32).new(5)
5.times do |i|
  bursty_requests.send(i + 1)
end
bursty_requests.close

bursty_requests.each do |req|
  bursty_limiter.receive
  puts "request #{req} #{Time.utc}"
end

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

$ crystal run rate_limiting.cr
request 1 2023-05-25 12:34:56.789012000
request 2 2023-05-25 12:34:56.989012000
request 3 2023-05-25 12:34:57.189012000
request 4 2023-05-25 12:34:57.389012000
request 5 2023-05-25 12:34:57.589012000

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-05-25 12:34:57.789012000
request 2 2023-05-25 12:34:57.789012000
request 3 2023-05-25 12:34:57.789012000
request 4 2023-05-25 12:34:57.989012000
request 5 2023-05-25 12:34:58.189012000

In this Crystal version, we use Channels to implement the rate limiting mechanism. The spawn keyword is used to create new fibers (lightweight threads) for handling the timing operations. The Time.utc method is used to get the current time, which is equivalent to time.Now() in Go.

The overall structure and logic of the program remain the same as the original Go version, demonstrating how similar concepts can be expressed in Crystal’s syntax.