Rate Limiting in C++

Rate limiting is an important mechanism for controlling resource utilization and maintaining quality of service. C++ can support rate limiting using threads, mutexes, and condition variables.

#include <iostream>
#include <chrono>
#include <thread>
#include <queue>
#include <mutex>
#include <condition_variable>

class RateLimiter {
private:
    std::queue<int> requests;
    std::mutex mtx;
    std::condition_variable cv;
    std::chrono::milliseconds interval;

public:
    RateLimiter(std::chrono::milliseconds interval) : interval(interval) {}

    void addRequest(int req) {
        std::unique_lock<std::mutex> lock(mtx);
        requests.push(req);
        cv.notify_one();
    }

    void processRequests() {
        while (true) {
            std::unique_lock<std::mutex> lock(mtx);
            cv.wait(lock, [this] { return !requests.empty(); });

            int req = requests.front();
            requests.pop();

            lock.unlock();

            std::cout << "Processing request " << req << " at " 
                      << std::chrono::system_clock::now().time_since_epoch().count() << std::endl;

            std::this_thread::sleep_for(interval);
        }
    }
};

int main() {
    // Basic rate limiting
    RateLimiter limiter(std::chrono::milliseconds(200));

    // Start the processor thread
    std::thread processor(&RateLimiter::processRequests, &limiter);

    // Add requests
    for (int i = 1; i <= 5; ++i) {
        limiter.addRequest(i);
    }

    // Wait for a while to let the basic rate limiting complete
    std::this_thread::sleep_for(std::chrono::seconds(2));

    std::cout << "\nStarting bursty requests\n" << std::endl;

    // Bursty rate limiting
    RateLimiter burstyLimiter(std::chrono::milliseconds(200));

    // Start the bursty processor thread
    std::thread burstyProcessor(&RateLimiter::processRequests, &burstyLimiter);

    // Add bursty requests
    for (int i = 1; i <= 5; ++i) {
        burstyLimiter.addRequest(i);
    }

    // Wait for the program to complete
    processor.join();
    burstyProcessor.join();

    return 0;
}

This C++ implementation uses a RateLimiter class to manage rate limiting. Here’s how it works:

We create a RateLimiter with a specified interval (200 milliseconds in this case).
The addRequest method adds requests to a queue and notifies the processor.
The processRequests method runs in a separate thread, processing requests at the specified interval.
For basic rate limiting, we add 5 requests and process them one every 200 milliseconds.
For bursty rate limiting, we use the same mechanism but the nature of thread scheduling might allow some requests to be processed in bursts.

To run the program, compile it with a C++11 (or later) compatible compiler and execute the resulting binary.

Running our program, we’ll see the first batch of requests handled once every ~200 milliseconds as desired. The second batch of requests (bursty requests) might show some clustering of request processing times due to the nature of thread scheduling, but will still maintain an average rate of one request per 200 milliseconds.

This implementation doesn’t provide an exact equivalent to Go’s channel-based rate limiting, but it achieves a similar effect using C++ threading primitives. The bursty behavior in this C++ version is less controlled than in the Go version, as it depends more on the operating system’s thread scheduling.

Waitgroups in C++

Atomic Counters in C++

Learn X By Example

Title here

Rate Limiting in C++