Rate Limiting in Julia

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Julia elegantly supports rate limiting with tasks, channels, and timers.

using Dates

function main()
    # First we'll look at basic rate limiting. Suppose
    # we want to limit our handling of incoming requests.
    # We'll serve these requests off a channel of the
    # same name.
    requests = Channel{Int}(5)
    for i in 1:5
        put!(requests, i)
    end
    close(requests)

    # This limiter channel will receive a value
    # every 200 milliseconds. This is the regulator in
    # our rate limiting scheme.
    limiter = Channel{DateTime}(Inf)
    @async while true
        put!(limiter, now())
        sleep(0.2)
    end

    # By blocking on a receive from the limiter channel
    # before serving each request, we limit ourselves to
    # 1 request every 200 milliseconds.
    for req in requests
        take!(limiter)
        println("request ", req, " ", now())
    end

    # We may want to allow short bursts of requests in
    # our rate limiting scheme while preserving the
    # overall rate limit. We can accomplish this by
    # buffering our limiter channel. This burstyLimiter
    # channel will allow bursts of up to 3 events.
    burstyLimiter = Channel{DateTime}(3)

    # Fill up the channel to represent allowed bursting.
    for i in 1:3
        put!(burstyLimiter, now())
    end

    # Every 200 milliseconds we'll try to add a new
    # value to burstyLimiter, up to its limit of 3.
    @async while true
        put!(burstyLimiter, now())
        sleep(0.2)
    end

    # Now simulate 5 more incoming requests. The first
    # 3 of these will benefit from the burst capability
    # of burstyLimiter.
    burstyRequests = Channel{Int}(5)
    for i in 1:5
        put!(burstyRequests, i)
    end
    close(burstyRequests)
    for req in burstyRequests
        take!(burstyLimiter)
        println("request ", req, " ", now())
    end
end

main()

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

$ julia rate-limiting.jl
request 1 2023-06-01T12:00:00.000
request 2 2023-06-01T12:00:00.200
request 3 2023-06-01T12:00:00.400
request 4 2023-06-01T12:00:00.600
request 5 2023-06-01T12:00:00.800

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-06-01T12:00:01.000
request 2 2023-06-01T12:00:01.000
request 3 2023-06-01T12:00:01.000
request 4 2023-06-01T12:00:01.200
request 5 2023-06-01T12:00:01.400

In this Julia version:

  1. We use Channel instead of Go’s channels.
  2. We use @async to create asynchronous tasks, similar to Go’s goroutines.
  3. We use put! and take! for channel operations instead of <-.
  4. We use sleep instead of time.Tick for creating regular intervals.
  5. We use now() from the Dates module to get the current time.

The overall structure and logic of the rate limiting example remains the same, showcasing how Julia can handle similar concurrency patterns to Go.