Multi-Process Message Alert System

Multi-Process Message Alert System

Overview

The purpose of this project is to explore the use of multiple processing communicating using the ZMQ (Zero Message Queuing) library and to write code in both Python and C / C++. We will be using the publish / subscribe mode of ZMQ to have Python clients subscribe to the messages distributed by the C / C++ server. Most of the hard work (networking, threading) is already done by ZMQ leaving you to write the relatively simple Python client and the slightly more intricate server. Your task will be to write robust code that catches errors, has a fail-safe to ensure authorization for messaging, and exits gracefully.

You might have seen a recent case where such a system did not have an appropriate authorization.

Goals

You should gain experience in the following:

Properly setting up / using the team dropbox / compiling with the right environment for the class
Use a Makefile to properly specify your code for compilation
Reading on-line documentation for various packages (ZeroMQ) and parsing how to use them
Adding in appropriate error catching for user input errors
Adding in authorization steps to prevent inadvertent usage
Graceful exiting for the server code
Logging in to the same machine via multiple ssh sessions

Approach: Pre-Project and Project

For this project, you will be able to work as a pair. You can also complete the lab as an individual. Whichever option you choose, be sure to let us know by following the instructions in this post.

Clone the project1 repository here: https://gitlab.com/nd-cse-30341-sp18/project1.git.

In the pre-project, you will write some basic Python and C / C++ code to accomplish the following three tasks:

Catch Control-C in C / C++ code for a graceful exit
Write C / C++ code to publish to a ZMQ socket
Write Python code to subscribe to a ZMQ socket

For your pre-project, you will not need to submit a Makefile, only your example code. Your code does not have to be perfectly operational, just a good attempt at getting things to work correctly. We do recommend though testing it out though to avoid any issues with the actual project submission.

For the project, you will write C / C++ and Python code to accomplish the following tasks:

Server [C/C++] - Be able to send the contents of a reasonably sized file of ASCII text via the publish mechanism of ZMQ
Server [C/C++] - Make sure to prompt / confirm before sending the contents of a file
Server [C/C++] - Provide error checking to handle key cases (non-ASCII, file is too long, file does not exist, cannot read file, etc.)
Server [C/C++] - Gracefully exit either by the user choosing to exit or via Control-C
Client [Python] - Continuously loop and display messages on the screen

Clients (Python)

Our clients will be Python-based. The clients will be the subscribers in the publish / subscribe model. We will cover this model in more detail in class but the idea is as follows:

Each client will create a ZMQ subscriber socket to listen to the C / C++ server publisher
The hostname for the publisher will always be localhost (aka the machine that you are currently logged into)
The port number will be one selected from the following table
Each client should display the message on the screen as it is received along with a timestamp of when it was received
Each client should have one argument as an input (the port number)
The code should loop forever until exited (via Control-C)

For each Python client that we will want to start up, you will need to log in separately via a separate SSH session. Make sure that you are logging in to the same CSE student machine as the firewall rules will prevent you from making network connections amongst the different machines.

We will not be focusing on making the client-side robust, only the C / C++ side of the house.

We recommend using the python interpreter provided at /afs/nd.edu/user14/csesoft/2017-fall/anaconda3/bin/python3.6. As a hint, you should start your python script with these two lines:
#!/afs/nd.edu/user14/csesoft/2017-fall/anaconda3/bin/python3.6
# -*- coding: UTF-8 -*-

Server (C/C++)

Our server will be C or C++ based. You may choose to use whatever language you are most comfortable with for this particular task. The key is that you will need to properly write your Makefile to ensure that the code will compile correctly for the TAs. You are more than welcome to ask the TAs to confirm that your code indeed compiles in the expected environment.

Our goal with the server is to create a loop that continually waits for instructions by the user. The server will support three commands that can be entered by the user:

help - Show an informative list of the available commands
send XXXX - Send the file XXX provided that it satisfies the appropriate safeguards and that sending is confirmed by the user via a YES response
quit - Gracefully exit the code and cleanup all sockets / information as appropriate

Your code should essentially sit in a giant loop waiting for a new command from the user. It should be helpful and informative to the user as described below.

% ./project1 1500
Welcome to the Multi-Process Message Broadcast System
  Now publishing on port 1500.  

> help
The commands for MP-MBS are as follows:
   help          This command
   send XXX      Send the file XXX out to all subscribed clients
   quit          Exit this code
   
> send testText.txt
The following message will be sent (18 characters):
CSE30341 is great!
Type YES to confirm: YES
Message sent!

> send testText.tx
Error: There was no file named testText.tx.

> quit
Thanks for using MP-MBS! Exiting....

% ./project1
Welcome to the Multi-Process Message Broadcast System
  Error: Unable to launch - port not specified!
  Proper Usage:  ./project1   XXXX   where XXXX is the port number

Your code should take in a single argument which is the port number to use for the ZMQ publish port.

Publishing Messages

Your server will act as a ZMQ publisher on a specific port. All of the Python clients will then subscribe to that particular port. Notably, ZMQ will take care of all of the various operations allowing you to start / stop the server and clients as you see fit. The library itself is pretty nifty in that it brings multithreading and robustness to messaging saving you the time and trouble developing such a library.

As a reference, you will want to look at the code for the ZMQ Weather Publisher in C. We will talk about this in class as well as share a few examples via the class mailing list.

Linking to the ZMQ Library: ZMQ is not part of the C++ standard library. You will need to set up your build process appropriately to use this 3rd party library. In the Project 1 Repository, we have given you a Makefile with the basic compiler and loader flags to pass to appropriately include the ZMQ library headers link to the ZMQ library binary on the student machines. However, you will still need to make sure the path to the ZMQ library binary is in your loader’s path for dynamic libraries for your executable to run by completing the instructions below.

Run the command echo $LD_LIBRARY_PATH. It should give you a list of directories separated by the colon : character. You must have both /afs/nd.edu/user14/csesoft/new/lib64 (to use the most up-to-date version of the C++ standard library) and /afs/nd.edu/user14/csesoft/2018-spring/lib (to use the ZMQ library).

Use the following commands to add the paths: setenv LD_LIBRARY_PATH /afs/nd.edu/user14/csesoft/new/lib64:$LD_LIBRARY_PATH and setenv LD_LIBRARY_PATH /afs/nd.edu/user14/csesoft/2018-spring/lib:$LD_LIBRARY_PATH

Add both of these commands to the end of your .cshrc file in your home directory to automatically add the paths each time you log in.

Error Protection

As you can see from the above example and the prior description, you will need to add in a bit of error-checking. Consider the following errors or cases to handle:

What if the file does not exist
What if the file exists but you cannot open it (wrong permission)
What if the file is too long (> 120 characters)
What if the file contains non-ASCII readable text (>= 32 (space) and <= 126 ~)

Using Safeguards

As noted in the above example, you will need to have the user confirm that they indeed want to send the message by typing YES as in input.

Graceful Exits

You will also need to gracefully catch a user pressing Control-C (SIGINT) such that you can properly close down the ZMQ socket before exiting. To catch Control-C, you will need to use the signal function to register a handler (a function that you write) that is invoked whenever Control-C (SIGINT) is pressed.

#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

void mySignalHandler (int sig)
{
   /* Do something */
}

int main ()
{
  signal (SIGINT, mySignalHandler);

  while(1)
  {
     pause();

  }
}

In the above-listed code, the signal call inside of main registers a handler (callback) mySignalHandler. If the noted signal (SIGINT) occurs, that handler in the code is invoked. Normally, there is already a handler in place courtesy of the standard libraries. With signal, you can tell the OS that you would like to have a very specific function called for that type of signal. There are of course numerous types of OS signals that you can choose to camp on / react to with specific behavior as you see fit.

Catching Signals: Best Practices

When the Operating System sends a signal to your process, the signal handler registered for that signal is executed in an asynchronous manner. The ramifications of this can have unwanted side-effects to your program if special consideration is not made. To be explicitly safe, your signal handler (and any functions it calls) needs to be reentrant. Notably, most implementations of C stdlib’s printf and C++ stdlib’s iostream are not, though C stdlib’s puts function usually is. See this note about best practices for signal handling from GNU, the makers of GCC.

Putting It All Together

Your task then is to write a C or C++ server that meets all of the above listed specifications (set up / publish via ZMQ, wait / react to user input, protect against errors, gracefully exit).

For all practical purposes, you should have a loop something like this:

/* Parse the input arguments and check for a port */

/* Catch Control-C via the signal handler */

/* Set up the ZMQ Socket as a Publisher on the right port */ 

while ( keepGoing )
{
     /* Wait for user input */
     /* Parse what the user typed */
     /* If it is a send command, try to read the file - make sure it succeeded and the content is good */
     /* If the user confirms with a YES, blast it out via the ZMQ sending mechanism */
     /* Repeat */ 
}

Submission

See the submission document at https://nd-cse-30341-sp18.gitlab.io/submissions/

It is extremely important that you properly follow the directions for setting up your environmental variables to ensure that your code will correctly run / compile for the TAs.

Submission: Pre-Project (5%)

Due: Thursday, January 25th, 23:59:59 EST

Write your pre-lab code and put it into the project1 sub-folder into a sub-folder called preproject (aka project1/preproject).

You should have two submissions in this folder:

Client.py : Your first attempt at a subscriber client for Python
Server.c or Server.cc : Your code that catches Control-C as an example for graceful exiting

Note that you do not need to have working code (though it should be reasonably close). We will not check your code for compilation. It might be a good idea as well to try to successfully compile / include the ZMQ libraries as well in your code as well as a test case.

We will run through discussion on writing the client code in class on Tuesday and answer questions on the baseline server code on Thursday.

Each file is worth 2.5 points.

Submission: Project (95%)

Due: Thursday, February 1st, 23:59:59 EST

Create / edit your code in the AFS dropbox directory for your team. Remember to frequently use git to maintain past versions just in case a more recent version has problems.

You should be submitting the following files:

Makefile - Use the provided Makefile as an example starting point
Appropriate source files - Include the appropriate source files for your submission
README : A brief file of your name and NetID for all team members

Make sure to submit your well-commented code that compiles to an executable named project1. While the TAs will make some effort to be helpful, there will significant point subtracted for submissions that require assistance to be made testable.

Project 1 Rubric