Computer Science 432/563
Operating Systems

Spring 2016, The College of Saint Rose

Lab 7: Security
Due: 4:00 PM, Monday, May 2, 2016

In this last lab, you will spend some time looking at a common security problem based on the problem of the buffer overflow.

You may work alone or with a partner on this lab.

Readings

Start by reading these pages, which are quite long and detailed, but also interesting and informative. You will find there is some overlap in content.

Trying Things Out

We will use ascg.strose.edu to try this out. In the shared area, you will find a directory called bufferoverflow containing several files, including some from the documents you just read, compiled for use on this FreeBSD system.

The only one we will try is in bufferoverflow.c, which has been compiled up to an executable called bufferoverflow.

Here, we read characters into a buffer that's local to main then call a function that copies it into a smaller buffer. If the input we type is longer than 80 characters, it doesn't fit.

The script trysizes is provided to allow you to run this program with various interesting numbers of spaces on the input. We can also watch what's happening here by compiling with -g (which the executables in the shared area have been) and running in gdb. Note: you might need to modify trysizes to skip some cases if they do not run to completion.

Question 1: Explain what is happening here. You will see that some inputs lead to a correct execution, while others cause a data corruption but successful execution to completion, and others cause the program crash.

Look also at vulnerable.c, which contains a similar error and exploit.c which attempts to exploit this vulnerability. In versions of FreeBSD prior to 7.0 (which I no longer have available to me, unfortunately), this could be used to demonstrate an exploited buffer overflow, resulting in a regular user gaining root access if the vulnerable executable was installed as setuid root.

Submission and Evaluation

This lab will be graded out of 10 points (based on your answer to the one question above).

By 4:00 PM, Monday, May 2, 2016, submit your answer to the lab questions by email to terescoj AT strose.edu.