Here's a small change that will allow idle MPI workers to yield their core. I've sent this to Andreas in case he wants to use it. With this change I can use one worker per hyperthread without slowing the end of phase 2.
Code:
--- lib/mpi.c.orig 2022-05-11 10:13:12.940302581 -0700
+++ lib/mpi.c 2022-05-11 11:26:12.823859553 -0700
@@ -555,6 +555,12 @@
finish = false;
while (!finish) {
/* Wait for a message from the server. */
+ int message_ready = 0;
+ MPI_Iprobe(0, MPI_ANY_TAG, MPI_COMM_WORLD, &message_ready, &status);
+ while (!message_ready) {
+ usleep(100);
+ MPI_Iprobe(0, MPI_ANY_TAG, MPI_COMM_WORLD, &message_ready, &status);
+ }
MPI_Recv (&job, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, &status);
cm_stat_init (stat);
switch (status.MPI_TAG) {