Title here
Summary here
In this example, we’ll look at how to implement a worker pool using threads and channels.
import System.Concurrency
import Control.Monad.Managed
import Data.IORef
import System.Clock
-- Here's the worker, of which we'll run several
-- concurrent instances. These workers will receive
-- work on the `jobs` channel and send the corresponding
-- results on `results`. We'll sleep a second per job to
-- simulate an expensive task.
worker : Int -> Channel Int -> Channel Int -> IO ()
worker id jobs results = do
loop
where
loop : IO ()
loop = do
job <- readChannel jobs
case job of
Nothing -> pure ()
Just j -> do
putStrLn $ "worker " ++ show id ++ " started job " ++ show j
sleep 1
putStrLn $ "worker " ++ show id ++ " finished job " ++ show j
writeChannel results (j * 2)
loop
main : IO ()
main = do
-- In order to use our pool of workers we need to send
-- them work and collect their results. We make 2
-- channels for this.
let numJobs = 5
jobs <- makeChannel
results <- makeChannel
-- This starts up 3 workers, initially blocked
-- because there are no jobs yet.
for_ [1..3] $ \w ->
fork $ worker w jobs results
-- Here we send 5 `jobs` and then `close` that
-- channel to indicate that's all the work we have.
for_ [1..numJobs] $ \j ->
writeChannel jobs j
closeChannel jobs
-- Finally we collect all the results of the work.
-- This also ensures that the worker threads have
-- finished. An alternative way to wait for multiple
-- threads is to use a semaphore.
for_ [1..numJobs] $ \_ ->
ignore $ readChannel results
pure ()
-- To run the program:
-- :exec main
Our running program shows the 5 jobs being executed by various workers. The program only takes about 2 seconds despite doing about 5 seconds of total work because there are 3 workers operating concurrently.
$ idris -o worker-pools worker-pools.idr
$ ./worker-pools
worker 1 started job 1
worker 2 started job 2
worker 3 started job 3
worker 1 finished job 1
worker 1 started job 4
worker 2 finished job 2
worker 2 started job 5
worker 3 finished job 3
worker 1 finished job 4
worker 2 finished job 5
Note that Idris doesn’t have built-in support for concurrency like some other languages. This example uses the System.Concurrency
module which provides basic concurrency primitives. The implementation might not be as efficient as in languages with native support for concurrency, but it demonstrates the concept of worker pools.