Lecture 19 - RAID, Hierarchical Storage, Protection and Security


Elan Sleazebaggano: You wanna buy some death sticks?
Obi-Wan: You don't want to sell me death sticks.
Elan Sleazebaggano: I don't want to sell you death sticks.
Obi-Wan: You want to go home and rethink your life.
Elan Sleazebaggano: I want to go home and rethink my life.
-- Attack of the Clones

Agenda

Announcements

  • Today's office hours will end a bit early.
  • Final Words on File Systems

    Things to consider when working with file systems:

  • how to partition the disk
  • how to organize blocks within a partition to form a file system
  • structure of directories
  • allocation of space for files
  • free space management
  • We went over a lot of the file system things pretty quickly. I got the sense that many people had seen a lot of this before. So we move on.

    RAID

    To this point, we have talked about "partitions" as subdivisions of an individual disk. It is also possible to have a logical "partition" span multiple disks.

    RAID - Redundant Array of Independent/Inexpensive Disks

  • multiple disks to provide reliability through redundancy
  • efficiency - work can be spread across a number of disks or even disk controllers
  • convenience of one large partition instead of many small ones
  • The bullpen cluster includes a RAID:

    12 disks, 18 GB each. Connected to one Wide-SCSI controller. The system sees it as one big partition:

    > df -k /export/raid
    Filesystem            kbytes    used   avail capacity  Mounted on
    /dev/dsk/c1t5d0s6    191175687 25709437 163554494    14%    /export/raid
    

    There are many ways to organize a RAID (Tanenbaum, Figure 5-19):


  • RAID Level 0
  • basically just paste together a bunch of disks to see them as one big partition
  • striping
  • not really a RAID, as it is not redundant
  • reliability: one disk failure results in potential loss of entire partition! Actually lower the MTBF (mean time between failures)
  • little or no overhead on writes
  • 100% of disk space is usable for storage
  • RAID Level 1
  • mirroring
  • reliability: any failed disk can be reconstructed from its mirror with a simple copy
  • all writes must be written to two disks
  • reads can come from either of two disks - spread out the load
  • 50% of disk space is usable for storage
  • RAID Level 2
  • use 7 synchronized disks to store 4 disks worth of data
  • for each 4 bits, compute a 7-bit Hamming-coded word
  • Hamming codes are self-correcting for one error, can detect two errors
  • 57.1% of disk space is usable for storage
  • RAID Level 3
  • like RAID-2, but use a single parity bit
  • still can recover a lost disk using the parity bit
  • two lost disks means entire partition is lost
  • can work with any number of disks
  • can include multiple parity disks
  • space overhead of the parity disk(s)
  • RAID Level 4
  • use stripes for parity unit, allowing disks to work independently
  • still can recover a lost disk
  • parity disk can be a bottleneck as all writes require a write to the parity disk
  • space overhead of the parity disk(s)
  • RAID Level 5
  • like RAID-4, but parity bit is distributed across all disks
  • Where does this happen?

  • RAID controller - work is done by hardware, OS sees a single drive
  • kernel/device driver - work is done by the OS in software - OS uses disks independently, but presents them to "users" as a single unit
  • Hierarchical Storage

    Recall our memory hierarchy:

    Just as virtual memory uses disks to simulate a larger main memory, tapes and other removeable media can be used to simulate a larger disk.

  • extend the filesystem
  • small and frequently-used files remain on disk
  • large, old, rarely-used files are archived on tapes
  • when one of the old files is requested, the file is brought back onto the disk from the appropriate tape
  • usually implemented as a jukebox of tapes or removeable disks
  • tape latency is typically 1000 times that of a disk
  • add in a tape robot that has to go fetch a tape and it is even worse
  • or worse yet, a human who has to be notified, go to the "tape room", find the tape, bring it to the drive, load it
  • These systems are found at large supercomputing centers.

  • HPSS - High Performance Storage System
  • Unitree
  • Other issues:

  • how to decide when to archive to tape
  • retrieval from archive may be fully automated or users may need to explicitly request files from tapes
  • duplicate tapes? - tapes can be unreliable
  • when is this worthwhile? Can't we just archive information manually?
  • Protection and Security

    The operating system is (in part) responsible for protection - to ensure that each object (hardware or software) is accessed correctly and only by those processes that are allowed to access it.

    We have seen protection in a few contexts:

  • memory protection
  • file protection
  • CPU protection
  • We want to protect against both malicious and unintentional problems.

    Protection Domains

    There are a number of ways that an OS might organize the access rights for its protection system. A domain is a set of access rights and the objects (devices, files, etc.) they refer to.

    An abstract example from Tanenbaum:

    This just means that processes executing in, for example, domain 1, can read file1 and can read and write file2.

    The standard Unix protection system can be viewed this way. A process' domain is defined by its user id (UID) and group id (GID).

    Files (and devices, as devices in Unix are typically accessed through special files) have permissions for: user, group, and other. Each can have read, write, and/or execute permission.

    A process begins with the UID and GID of the process that created it, but can set its UID and GID during execution. Whether a process is allowed a certain type of access to a certain file is determined by the UID and GID of the process, and the file ownership and permissions.

    The superuser (UID=0) has the ability to set processes to any UID/GID. Regular users usually rely on setuid and setgid programs in the filesystem to change UID/GID.

    An executable file with the setuid (setgid) bit set will execute with the effective UID (GID) of the file's ownership rather than the process that runs it.

    Example: see /usr/X11R6/bin/xterm on your favorite Unix system. Yes, each time you start an xterm, you're the superuser (!), at least for a few moments.

    Other possible protection mechanisms:

  • protection matrix
  • access control lists
  • capabilities
  • See Tanenbaum for details on these.

    In any case, the OS must enforce the protection mechanism as appropriate.