Computer Science 335
Parallel Processing and High Performance Computing

Fall 2021, Siena College

Home     Syllabus     Schedule     Examples

In this lab, we'll look at MPI's non-blocking point-to-point communication.

You may work alone or with a partner on this lab.

Learning goals:

1. To learn how to do error checking on MPI calls.
2. To learn about non-blocking point-to-point communication in MPI.

Getting Set Up

You can find the link to follow to set up your GitHub repository nb-lab-yourgitname for this Lab in Canvas. One member of the group should follow the link to set up the repository on GitHub, then that person should email the instructor with the other group members' GitHub usernames so they can be granted access. This will allow all members of the group to clone the repository and commit and push changes to the origin on GitHub. At least one group member should make a clone of the repository to begin work.

Please answer the lab questions in the README.md file in the top-level directory of your repository.

Error Checking with MPI

The mpimsg directory of your repository contains a program similar to the mpiexchange program you wrote recently. This one demonstrates a few things beyond the earlier examples.

• All MPI calls return a status value, and it's a good idea to check it as is done in this example for the MPI_Init call.
  • Most class examples will not be thorough in this to keep things looking simpler.
  • For our purposes, any MPI error will cause the program to terminate with an error message, so it usually is not that important to us.
  • When developing large-scale software, we often wish to return error codes rather than crash the whole program, so error checking becomes more important there.
  • The error checking includes a messy little chunk of code to print out appropriate messages, so it's probably worth putting this into your own error reporting function if you want to use it.
• MPI_Status status - structure which contains additional info following a receive. We often ignore it (we passed in MPI_STATUS_IGNORE in earlier examples), but we will see some instances where it comes in handy.

Modify the program so the MPI_Recv uses a different "from" parameter.

Now, let's experiment with using MPI_ANY_SOURCE on our MPI_Recv call. That parameter allows the receive call to match an MPI_Send from any rank process, as long as the other parameters still match. We can also do the same for the message tag by specifying MPI_ANY_TAG. Replace the "from" and tag parameters in the MPI_Recv call with MPI_ANY_SOURCE and MPI_ANY_TAG, respectively, and try the program.

In general, it's good for efficiency purposes to avoid forcing processes to do their work in any particular order except for when there is a good reason for doing so. We will see many of those reasons soon. Let's look back at the mpi_trap2 program from the text's examples, a copy of which is in the mpi_trap2 directory of your repository. When the rank 0 process receives the messages with the partial answers from all of the other processes, the order in which they're received and processed doesn't really matter. As written, it must receive the messages in order by sender rank. However, suppose a that a process with a low rank, for whatever reason, takes longer than the others to finish its work. Messages that are already sent and which could be processed wait until all messages from lower-rank senders have been received. The messages and the work done to process the results received are very small here, so this doesn't matter much, but in some cases, it could.

Timings with MPI

In an earlier lab, you added timers to mpi_trap2 using the C library's gettimeofday function and the diffgettime function from our timer.c. When using MPI, there's an easier way to do this: the MPI_Wtime function. The function returns the number of seconds elapsed since some time in the past (much like Java's System.getCurrentTimeMillis, just in seconds instead of milliseconds).

Non-Blocking Point-to-point Communication

Consider again the mpiexchange.c program you wrote for the previous lab. Suppose we wanted that program to be generalized to work for more processes. That is, each process picks and number to send to the process with the next highest rank (except that with the highest rank, which sends it to 0). Then each process will receive that value, modify it in some way, then send it back. Those messages then need to be received and printed.

The directory mpiring contains the programs we'll use in this section.

The program mpiring_danger.c does just this. We can compile and run it on any number of processes and it will likely work.

However, it is not guaranteed to work, since an MPI_Send call will not necessarily return until there is a corresponding MPI_Recv call to receive the message. If each process performs the first MPI_Send and those calls cannot complete until the MPI_Recv, we can enter a deadlock condition, where each process is waiting for something to happen in some other process before it can continue execution.

The chances of this problem increase with larger messages.

The program mpiring_danger_large.c sends arrays in the same pattern as the previous program sent single int values. The #define at the top of the program determines how large these messages are.

These kinds of communication patterns are very common in parallel programming, so we need a way to deal with them. MPI provides another variation on point-to-point communication that is non-blocking. These are also known as immediate mode sends and receives. These functions are named MPI_Isend and MPI_Irecv, and will always return immediately.

The program mpiring_safe_large.c uses non-blocking sends to remove the danger of deadlock in this program. Please see the comments there about the need to wait for the message to be delivered (or at least to have it in progress) before the send buffer can be modified. This example uses MPI_Wait to accomplish that.

Submission

Commit and push!

Grading

This assignment will be graded out of 35 points.