Goals of the project:

Implement a simple-*ish* MapReduce with a thread pool using MPI processes.
- The goal of the project was to produce a high-throughput packet analysis system, so it makes sense to allow it to be distributed!
- As it stands, I'm going to try and implement both a pthread only version as well as one that is implemented using MPI. This shouldn't be too painful!
- UPDATE this is infact not possible directly and is probably more costly in communication costs than a simpler threading model. If I were to use MPI, I'd have to write alot more boilerplate and the program would be incredibly hard to write without using C++'s more implicit memory-management style. I am therefore choosing not to implement this with MPI. Sorry for making you excited!
Make it as robust as possible
- (Try to) follow RAII!
- Having said that, this is a linux-only build

Coursework Markers here!

HELLO COURSEWORK MARKERS after you have inevitably unzipped and gotten my repo set up on your ~/cs241 directory, make sure you are on the VM with this mounted. Then, for your convenience, I provide an easy way to compile:

cd cs241

source ./configure

This script will automatically build everything required and run tests.

You may also notice this is slightly overkill, but the justification for this goes as far as wanting to make, firstly, as robust a program as possible, and secondly something as generic as possible in order to allow for strong scalability.

The command wget --no-hosts www.google.co.uk works incorrectly on my laptop. I have to force it to use IPv4 resolution with:

wget --no-hsts --inet4-only -O /dev/null www.google.co.uk

Build

To build the project, the only dependency you will need installed is libpcap. If you don't already have this, run:

sudo apt update

sudo apt install libpcap-dev

Then, to clone (it's private):

git clone https://github.com/a1exxd0/cs241

Documentation

Also HELLO MARKERS please check my docs! Relies on google-chrome existing, as well as doxygen (don't run on the VM).

cd cs241-cw
doxygen Doxyfile
google-chrome html/index.html

Notes on difficulties building

Unfortunately due to this being ran in a 1-core VM for the sake of the coursework unexposed to the internet, the build is quite a long process. Please be patient!

Notes on testing

Testing this with Valgrind is incredibly slow - I've done debug runs with address sanitization as well as thread sanitization, but if you do choose to run the test suite through Valgrind, be prepared for a wait.

Also, the $PROJECT_ROOT/configure file currently specifies build type as with no tests. If you wish to enable them, you may find you struggle a little - unfortunately I have spent far too long on this coursework to force it to work on the DCS VM and so will not be able to help.

Of course, the tests run perfectly for me on my laptop: feel free to check the $PROJECT_ROOT/test directory and subdirectory .../utils/ for more on the testing files.