Rate Limiting in Dart

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. Dart supports rate limiting through async programming, Streams, and Timers.

import 'dart:async';

void main() async {
  // First we'll look at basic rate limiting. Suppose
  // we want to limit our handling of incoming requests.
  // We'll serve these requests off a Stream.
  final requests = Stream.fromIterable(List.generate(5, (i) => i + 1));

  // This Timer will fire every 200 milliseconds. This is the regulator in
  // our rate limiting scheme.
  final limiter = Timer.periodic(Duration(milliseconds: 200), (_) {});

  // By awaiting the limiter before processing each request, we limit ourselves to
  // 1 request every 200 milliseconds.
  await for (var req in requests) {
    await limiter.tick;
    print('request $req ${DateTime.now()}');
  }

  // Cancel the limiter timer
  limiter.cancel();

  // We may want to allow short bursts of requests in
  // our rate limiting scheme while preserving the
  // overall rate limit. We can accomplish this by
  // using a bursty limiter.
  final burstyLimiter = StreamController<DateTime>();

  // Fill up the controller to represent allowed bursting.
  for (var i = 0; i < 3; i++) {
    burstyLimiter.add(DateTime.now());
  }

  // Every 200 milliseconds we'll try to add a new
  // value to burstyLimiter, up to its limit of 3.
  Timer.periodic(Duration(milliseconds: 200), (timer) {
    if (burstyLimiter.hasListener && !burstyLimiter.isClosed) {
      burstyLimiter.add(DateTime.now());
    } else {
      timer.cancel();
    }
  });

  // Now simulate 5 more incoming requests. The first
  // 3 of these will benefit from the burst capability
  // of burstyLimiter.
  final burstyRequests = Stream.fromIterable(List.generate(5, (i) => i + 1));

  await for (var req in burstyRequests) {
    await burstyLimiter.stream.first;
    print('request $req ${DateTime.now()}');
  }

  // Close the bursty limiter
  await burstyLimiter.close();
}

Running our program we see the first batch of requests handled once every ~200 milliseconds as desired.

request 1 2023-06-13 10:15:00.000
request 2 2023-06-13 10:15:00.200
request 3 2023-06-13 10:15:00.400
request 4 2023-06-13 10:15:00.600
request 5 2023-06-13 10:15:00.800

For the second batch of requests we serve the first 3 immediately because of the burstable rate limiting, then serve the remaining 2 with ~200ms delays each.

request 1 2023-06-13 10:15:01.000
request 2 2023-06-13 10:15:01.000
request 3 2023-06-13 10:15:01.000
request 4 2023-06-13 10:15:01.200
request 5 2023-06-13 10:15:01.400

This example demonstrates how to implement basic and bursty rate limiting in Dart using asynchronous programming concepts. The Timer class is used to create a periodic timer that regulates the rate, while Stream and StreamController are used to manage the flow of requests and implement the bursty behavior.