Worker Pools in Rust

In this example we’ll look at how to implement a worker pool using threads and channels.

use std::thread;
use std::sync::mpsc;
use std::time::Duration;

// Here's the worker, of which we'll run several
// concurrent instances. These workers will receive
// work on the `jobs` channel and send the corresponding
// results on `results`. We'll sleep a second per job to
// simulate an expensive task.
fn worker(id: i32, jobs: mpsc::Receiver<i32>, results: mpsc::Sender<i32>) {
    for job in jobs {
        println!("worker {} started  job {}", id, job);
        thread::sleep(Duration::from_secs(1));
        println!("worker {} finished job {}", id, job);
        results.send(job * 2).unwrap();
    }
}

fn main() {
    // In order to use our pool of workers we need to send
    // them work and collect their results. We make 2
    // channels for this.
    const NUM_JOBS: i32 = 5;
    let (jobs_sender, jobs_receiver) = mpsc::channel();
    let (results_sender, results_receiver) = mpsc::channel();

    // This starts up 3 workers, initially blocked
    // because there are no jobs yet.
    for w in 1..=3 {
        let jobs = jobs_receiver.clone();
        let results = results_sender.clone();
        thread::spawn(move || {
            worker(w, jobs, results);
        });
    }

    // Here we send 5 `jobs` and then `close` that
    // channel to indicate that's all the work we have.
    for j in 1..=NUM_JOBS {
        jobs_sender.send(j).unwrap();
    }
    drop(jobs_sender);

    // Finally we collect all the results of the work.
    // This also ensures that the worker threads have
    // finished.
    for _ in 1..=NUM_JOBS {
        results_receiver.recv().unwrap();
    }
}

Our running program shows the 5 jobs being executed by various workers. The program only takes about 2 seconds despite doing about 5 seconds of total work because there are 3 workers operating concurrently.

$ cargo run
   Compiling worker-pools v0.1.0 (/path/to/worker-pools)
    Finished dev [unoptimized + debuginfo] target(s) in 1.21s
     Running `target/debug/worker-pools`
worker 1 started  job 1
worker 3 started  job 3
worker 2 started  job 2
worker 1 finished job 1
worker 1 started  job 4
worker 3 finished job 3
worker 2 finished job 2
worker 2 started  job 5
worker 1 finished job 4
worker 2 finished job 5

real    0m2.007s
user    0m0.004s
sys     0m0.000s

In this Rust version, we use std::thread for concurrency and std::sync::mpsc for channel-based communication between threads. The overall structure and functionality remain similar to the original example, with workers processing jobs concurrently and sending results back to the main thread.

查看推荐产品

Comments powered by Disqus