/*MPI_Allreduce is equivalent of doing MPI_Reduce followed by an MPI_Bcast. 
The MPI_Reduce does two things: 
1. Send all local values from non-zero PEs to the PE 0; 
2. The PE 0 receive all values from non-zero PEs, then calculate the sum. 
The MPI_Bcast also does two things: 
1. Send the sum from PE 0 to all non-zero PEs; 
2. All non-zero PEs received the sum from PE 0. Finally all the PEs have the same value of sum. 
 
If we write down what the exercise code really does using the MPI_Send and MPI_Recv functions, it is as following.
*/ 

if(rank % 2 == 0){  // even processes

    // Following is a code block equivalent to MPI_Reduce
     if(rank==0){  // This block will only be executed by PE 0
       MPI_Recv( local_values from non-zero PEs to PE 0);  // Step 2: recv_10, Here the local_values are the ranks of non-zero PEs
       evensum+=local_values;   // Step 3 : evensum=0+1+2+3=6 on PE 0
     }
     else{   // This block will only be executed by PEs 2
        MPI_Send( local_values from non-zero PEs to PE 0);  // Step 1: send_10
     }

    // Following is a code block equivalent to MPI_Bcast
    if(rank==0){ // This block will only be executed by PE 0
        MPI_Send( evensum from PE 0 to non-zero PEs);    // Step 4: send_20
    }
    else{   // This block will only be executed by PEs 2
        MPI_Recv( evensum from PE 0 to non-zero PEs);  // Step 5: recv_20, evensum=6 on even PEs
    }

}else{  // odd processes

   // Following is a code block equivalent to MPI_Reduce
     if(rank==0){    // This block will NEVER be executed
       MPI_Recv( local_values from non-zero PEs to PE 0);  
       oddsum+=local_values; 
     }
     else{   // This block will only be executed by PEs 1 and 3
        MPI_Send( local_values from non-zero PEs to PE 0);  // Step 1: send_10
     }

    // Following is a code block equivalent to MPI_Bcast
    if(rank==0){ // This block will NEVER be executed
        MPI_Send( oddnsum from PE 0 to non-zero PEs);  
    }
    else{   // This block will only be executed by PEs 1 and 3
        MPI_Recv( evensum from PE0 and assign it to oddsum on non-zero PEs); // Step 5: recv_20, oddsum=6 on odd PEs
    }

}
 
/*
After analysis of the above code, the following steps are executed in sequence:
 
1. All non-zero PEs send the local values, which are the ranks in this case.  
2. PE 0 receives all of the local values from non-zeron PEs. 
3. PE 0 calculates the sum of all local values (which is 6 in the case) and assign it to the variable named evensum. 
4. PE 0 sends the value of evensum (which is 6).
5. The PEs 2 receives the value of evensum from PE0 and assign it to their local variable evensum. At the same time, PEs 1 and 3 receive the value of evensum from PE0 and assign it to their local variable oddsum. 
 
So finally the variable evensum on all even PEs has the value 6 and the variable oddsum on all odd PEs has the value 6 too. Now you can see why answer C is correct. An addition conclusion is that the variable evensum on all odd PEs has the value 0 and the variable oddsum on all even PEs has the value 0 too.

rank  evensum  oddsum
0     6        0
1     0        6
2     6        0
3     0        6

*/