Rate Limiting in F#

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. F# supports rate limiting with asynchronous workflows and MailboxProcessor.

open System
open System.Threading

let main() =
    // First we'll look at basic rate limiting. Suppose
    // we want to limit our handling of incoming requests.
    // We'll serve these requests off a MailboxProcessor.
    let requests = MailboxProcessor.Start(fun inbox -> 
        async {
            for i in 1..5 do
                do! inbox.PostAndAsyncReply(fun ch -> i, ch)
        })

    // This limiter will receive a value every 200 milliseconds.
    // This is the regulator in our rate limiting scheme.
    let limiter = new Timer((fun _ -> ()), null, TimeSpan.Zero, TimeSpan.FromMilliseconds(200.))

    // By waiting for the limiter before serving each request,
    // we limit ourselves to 1 request every 200 milliseconds.
    for i in 1..5 do
        limiter.WaitHandle.WaitOne() |> ignore
        let req = requests.PostAndReply(id)
        printfn "request %d %O" req DateTime.Now

    // We may want to allow short bursts of requests in
    // our rate limiting scheme while preserving the
    // overall rate limit. We can accomplish this by
    // using a semaphore. This burstyLimiter will allow
    // bursts of up to 3 events.
    let burstyLimiter = new SemaphoreSlim(3, 3)

    // Every 200 milliseconds we'll try to add a new
    // permit to burstyLimiter, up to its limit of 3.
    let burstyTimer = new Timer((fun _ -> burstyLimiter.Release(1) |> ignore), 
                                null, TimeSpan.Zero, TimeSpan.FromMilliseconds(200.))

    // Now simulate 5 more incoming requests. The first
    // 3 of these will benefit from the burst capability
    // of burstyLimiter.
    let burstyRequests = MailboxProcessor.Start(fun inbox -> 
        async {
            for i in 1..5 do
                do! inbox.PostAndAsyncReply(fun ch -> i, ch)
        })

    for i in 1..5 do
        burstyLimiter.Wait()
        let req = burstyRequests.PostAndReply(id)
        printfn "request %d %O" req DateTime.Now

main()

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 7/1/2023 12:00:00 AM
request 2 7/1/2023 12:00:00 AM
request 3 7/1/2023 12:00:00 AM
request 4 7/1/2023 12:00:00 AM
request 5 7/1/2023 12:00:01 AM

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 7/1/2023 12:00:01 AM
request 2 7/1/2023 12:00:01 AM
request 3 7/1/2023 12:00:01 AM
request 4 7/1/2023 12:00:01 AM
request 5 7/1/2023 12:00:01 AM

In this F# version, we use MailboxProcessor as a channel-like construct to handle requests. The Timer class is used to implement the rate limiting, and SemaphoreSlim is used for the bursty limiter. The overall structure and logic of the program remain similar to the original, demonstrating basic and bursty rate limiting in F#.