Waitgroups in Elixir

To wait for multiple processes to finish in Elixir, we can use a concept similar to wait groups called Task.async_stream/3 combined with Enum.to_list/1.

defmodule WaitGroups do
  def worker(id) do
    IO.puts("Worker #{id} starting")
    # Sleep to simulate an expensive task
    Process.sleep(1000)
    IO.puts("Worker #{id} done")
  end

  def main do
    1..5
    |> Task.async_stream(fn id ->
      worker(id)
    end)
    |> Enum.to_list()

    # Note that this approach automatically waits for all tasks to complete
  end
end

WaitGroups.main()

This is the function we’ll run in every process:

def worker(id) do
  IO.puts("Worker #{id} starting")
  # Sleep to simulate an expensive task
  Process.sleep(1000)
  IO.puts("Worker #{id} done")
end

In the main function:

We use Task.async_stream/3 to launch several processes. This function automatically manages the concurrency and waiting for all processes to finish.

1..5
|> Task.async_stream(fn id ->
  worker(id)
end)
|> Enum.to_list()

The Enum.to_list/1 call at the end ensures that we wait for all tasks to complete before moving on.

To run the program:

$ elixir waitgroups.exs
Worker 1 starting
Worker 2 starting
Worker 3 starting
Worker 4 starting
Worker 5 starting
Worker 1 done
Worker 2 done
Worker 3 done
Worker 4 done
Worker 5 done

The order of workers starting up and finishing is likely to be different for each invocation.

Note that this approach in Elixir automatically handles errors and terminates all tasks if any of them fail. For more advanced use cases, you might want to look into the Task module and OTP behaviors like GenServer.