Worker Pools in Miranda

Our example demonstrates how to implement a worker pool using threads and queues in Java.

import java.util.concurrent.*;

public class WorkerPools {
    // Here's the worker, of which we'll run several concurrent instances.
    // These workers will receive work on the `jobs` queue and send the
    // corresponding results on `results`. We'll sleep a second per job to
    // simulate an expensive task.
    static class Worker implements Runnable {
        private final int id;
        private final BlockingQueue<Integer> jobs;
        private final BlockingQueue<Integer> results;

        Worker(int id, BlockingQueue<Integer> jobs, BlockingQueue<Integer> results) {
            this.id = id;
            this.jobs = jobs;
            this.results = results;
        }

        public void run() {
            try {
                while (true) {
                    Integer j = jobs.take();
                    System.out.println("worker " + id + " started  job " + j);
                    Thread.sleep(1000);
                    System.out.println("worker " + id + " finished job " + j);
                    results.put(j * 2);
                }
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
    }

    public static void main(String[] args) throws InterruptedException {
        // In order to use our pool of workers we need to send them work
        // and collect their results. We create 2 queues for this.
        final int numJobs = 5;
        BlockingQueue<Integer> jobs = new LinkedBlockingQueue<>(numJobs);
        BlockingQueue<Integer> results = new LinkedBlockingQueue<>(numJobs);

        // This starts up 3 workers, initially blocked because there are no jobs yet.
        for (int w = 1; w <= 3; w++) {
            Thread worker = new Thread(new Worker(w, jobs, results));
            worker.start();
        }

        // Here we send 5 `jobs` and then `close` that queue to indicate
        // that's all the work we have.
        for (int j = 1; j <= numJobs; j++) {
            jobs.put(j);
        }

        // Finally we collect all the results of the work.
        // This also ensures that the worker threads have finished.
        for (int a = 1; a <= numJobs; a++) {
            results.take();
        }
    }
}

Our running program shows the 5 jobs being executed by various workers. The program only takes about 2 seconds despite doing about 5 seconds of total work because there are 3 workers operating concurrently.

$ javac WorkerPools.java
$ java WorkerPools
worker 1 started  job 1
worker 2 started  job 2
worker 3 started  job 3
worker 1 finished job 1
worker 1 started  job 4
worker 2 finished job 2
worker 2 started  job 5
worker 3 finished job 3
worker 1 finished job 4
worker 2 finished job 5

This example demonstrates the use of Java’s concurrency utilities to create a worker pool. We use BlockingQueue for thread-safe communication between the main thread and worker threads. The Worker class implements Runnable, allowing it to be executed in separate threads. This approach achieves similar functionality to the original example, showcasing concurrent execution of tasks in a pool of workers.