Security in F/OSS

The following is a paper that I wrote for an Advanced Linux Server Administration class, this doesn't have much to do with what we learned about in the class, but it was a required assignment anyway.  I have taken some things out and added a few things, but overall it's the same.  Edit: The citations were a little screwy, so I've taken out most inline citations for the time being (aka probably forever).

Abstract

Security in software is a big deal.  People live their lives on their computers quite literally.  A typical user will store personal pictures, movies and documents on their computer, not to mention visiting their banking, investing and other financial websites using this very same computer.  Users do not understand their computers very well; they only use them as a means to an end, much like cars.  Since end users do not completely understand computer security, this leaves a lot of the work of securing a system up to power users, administrators and the programmers of the software applications.  One way that attackers are invading end users is by executing buffer overflows on their computers.

Buffer overflows are an extremely common and popular way for attackers out in the wild to compromise a system by simply telling their target to do exactly what they want them to.  Since computers are very literal, they’ll execute the code exactly as you told them to, sometimes handing over root/administrator access without any fuss.  Buffer overflows are very scary and should be an item to watch out for no matter who you are.  Fortunately they are mostly detectable, preventable and fixable with a little knowhow and elbow grease.

Free and Open Source Software (F/OSS) is no exception to these rules and must be programmed with the highest security standards in mind.  One professed advantage of F/OSS is that other users besides the original programmers can review the code and find potential security problems and fix it themselves and/or report the problem to the original manufacturer(s).

Buffer Overflows

Buffer overflows are a unique beast because it gives the attackers exactly what they need in order to own the system: execution of code.  In Figure 1 there is a simple example of two variables stored in memory, before and after a buffer was overrun.  The word OVERFLOW fell (or was pushed) into part of variable 2’s address space and corrupted the data residing there.

Figure 1, Overflow example

There are several ways that code can be injected into a system buffer so that it will be executed, but the most common is by injecting the code there manually and having the program step into it naturally, meaning the program will execute it.  This can be done by force-modifying what is called an activation record.  An activation record does many things, but the part buffer overflows are concerned with is what happens when a function is called; the function lays down on the top of the stack a memory address that the code needs to return to once the function has finished processing.  If an attacker can corrupt the activation record, they can cause the program to jump to their attack code.  This type of attack is called a “stack smashing attack” in the underground world, and makes up a large percentage of attacks on the buffer.

Prevention methods

Software prevention, stoppage at the source, is the ideal way to eliminate buffer overflows; but it isn’t the easiest or the most practical method.  There are two major ways to prevent attacks using software: the first is at compile time and the second is at run time.

Conception, Implementation and Prevention

One of the reasons it can be such a problem to implement secure software is business-related issues:  The boss wants the program to function properly and he wants it done this week!  In a big project this leaves very little time for the programmer to walk through the many steps it would take in order to properly secure their code.  In addition to the time pressures, it is a commonly accepted practice to sacrifice security and correctness for performance, which might lead to problems down the road.

Thwarting these issues isn’t an easy task, but it is possible.  Using a modern day high-level language which has strong typing makes you much less vulnerable to buffer overflows because the compiler will enforce a smaller variety of operations allowed on the variables included in the code.  “Strong Typing” means that the programming language restricts the procedures that code is allowed to execute on different types of variables, such as trying to put a pointer where a double might be or attempting to call a function defined foo(int) with a string instead of an integer value.  The more checks the compiler and coder do on these types of forced casting, the safer the code will be.

Reviewing code, testing code explicitly for security issues, re-writing older insecure code and replacing code for standard library alternatives which are time-tested and proven will give the best results when it comes to security practices.  The only issue is, the coder needs to actually implement these best practices, or else it doesn’t matter.

Even after bad code is compiled and the exploit code is later prepared, there is still hope.  Thankfully our friends in the operating system and processor worlds combined their knowledge and technologies to create two ingenious ways to foil an attack at the hardware and operating system levels.  The first step to stopping execution of arbitrary code from an un-trusted source is done by leaning on hardware to do some of the heavy-lifting in the form of an NX bit (No eXecute), Intel calls this technology XD (eXecute Disable) and AMD named it Enhanced Virus Protection.  These technologies attempt to automatically recognize when a buffer has overrun its bounds or when an area in memory marked as data storage only attempts to execute.  When either of these actions is found, the NX bit will kill the process responsible for the violation.

The second and third steps are known as DEP (Data Execution Prevention) and ASLR (Address Space Layout Randomization), which is Windows OS terminology.  These same principles are carried over to most all of the modern operating systems including OS X and Linux in almost all flavors.

The DEP protection is simple, both software and hardware work together to prevent any execution from the activation record, the data stack and other places that executable code should never be stored.

Figure 2, DEP settings

ASLR is a little more complicated than DEP and has only recently been introduced in Vista and Windows Server 2008 (although it has been available for various distributions of Linux for several years).  When a non-ASLR protected system boots it loads certain system files and executable programs into a fixed location in memory (such as 0x77AF0000), this makes it very predictable where core resources are going to be and that much easier to find and exploit them.  That would be like someone driving in their car the same exact path at the exact same time every day while travelling to work.  If someone wanted to harm this person, since they are on a very predictable path and schedule, it would be extremely easy for the malicious people to pick them off.

ASLR randomizes the memory address space layout so that each service, program, library or executable integral to the systems performance and security can be randomly placed in up to 256 different locations.  In this way ASLR obfuscates critical memory locations, which will change on every system reboot, so that an attacker has a much smaller chance of writing repeatable programs that hit a code entry point simply by hard referencing the memory address.  It is also important to note that ASLR can be activated for any program, not just core OS executables; given that it is compiled according to Microsoft’s specifications.

The best recommendation, from a security standpoint, that I can make to anyone whether or not you feel like your system is up to date and secure or not, is do not run programs that you do not trust the sources whence it came.

Problems introduced

Problems that are presented when running either DEP and/or ASLR are many, and only a few will be discussed here.  The most important issue is the fact that when either DEP or ASLR catches a piece of software trying to execute a buffer overflow, the default action is to shut the program down.  Let’s pretend that you are Amazon.com running an Apache web server, and you are being subjected to a buffer overflow exploit on Apache via remote intrusion.  What would be the worst thing about automatically shutting down your webserver? One paper says:

Being taken offline persistently can be costly; reports in 2000 show that Amazon loses about $180,000 per hour of downtime.

While it may be true that Amazon would do better with disabled servers than compromised servers — that, in the end, is an economic question — it is, nevertheless, also true that it is difficult to distinguish exploitable vulnerabilities from mere (segfault-inducing) denial of service.  Neither an automated watcher program nor a system administrator working under time pressure can be expected to make the correct determination. (Shacham, Page, Pfaff, Goh, Modadugu, & Boneh, 2004)

The automated watcher program that they are referring to is a crash detection and reaction mechanism which they call a watcher.  A watcher daemon will detect intrusions and take one of two actions, the first is to notify an administrator and ask them to take action and the second would be to take matters into its own hands.  If the settings instruct taking the second option, it will shut down the program in question or attempt to prevent the attack.

This unintended stopping of programs can apply to non-malicious software as well, as Microsoft points out on their Vista website, DEP can have unintended consequences if the security settings are set too high.

A secondary problem with ASLR is memory fragmentation interfering with the effectiveness of the randomization process.  Memory fragmentation can be split into two parts, Internal Fragmentation and External Fragmentation.  Internal fragmentation occurs when a block of memory is requested and allocated to a program, but never used fully by the program.  External fragmentation is when memory is requested and allocated and a very small piece is left over (from the overall memory block) and can’t be effectively used. See Figure 3 below.

Figure 3, External and Internal fragmentation

Security Problems in F/OSS

"Many eyeballs" debunked

There are other security risks that are less prominent (in my mind) but are still applicable in open source software.  If open source software is comprised of a number of different people contributing to the codebase, in theory it would be easy for an anonymous person, who could be trusted by the open source group leader, to program something malicious, such as a back door and take over at some later point.  This is only because strangers are able to submit source code and have it included in the final product.  This is not code that I would want to be executing on my computer.  Anonymous code submission would not be allowed, let alone used, in closed source shops.  Also, many open source projects specifically exclude themselves from any sort of liability, such as Mozilla’s legalese:

5. LIMITATION OF LIABILITY. EXCEPT AS REQUIRED BY LAW, MOZILLA AND ITS DISTRIBUTORS, DIRECTORS, LICENSORS, CONTRIBUTORS AND AGENTS (COLLECTIVELY, THE "MOZILLA GROUP") WILL NOT BE LIABLE FOR ANY INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES ARISING OUT OF OR IN ANY WAY RELATING TO THIS AGREEMENT OR THE USE OF OR INABILITY TO USE THE PRODUCT, INCLUDING WITHOUT LIMITATION DAMAGES FOR LOSS OF GOODWILL, WORK STOPPAGE, LOST PROFITS, LOSS OF DATA, AND COMPUTER FAILURE OR MALFUNCTION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES AND REGARDLESS OF THE THEORY (CONTRACT, TORT OR OTHERWISE) UPON WHICH SUCH CLAIM IS BASED.

Works Cited