Fuzzing can be dangerous. After all, you’re trying to break things.
In fuzzing, you deliver deliberately malformed inputs to software to see if the software fails. If it does, you’ve located a vulnerability and can go back to the code and fix it. It’s an excellent, proactive method for software development organizations to fix security weaknesses.
And it should be no surprise that fuzzing is also the preferred method for attackers who want to locate zero-day vulnerabilities. Fuzzing, by design, tries to make software fail.
Guidelines for fuzzing
Here are the standard guidelines for fuzzing:
- Don’t fuzz production targets. Fuzzing can cause mild discomfort in targets, such as increased resource usage. It can also cause complete failure. You should not point your fuzzer at any target used by real people for real work.
- Put your fuzzer close to your target. Try to eliminate, or at least minimize, the number of systems through which fuzz test cases must pass in order to reach your target. An intermediate system might modify the test cases, drop them, or fail itself.
Our recommendation is to perform all fuzz testing in a controlled, isolated laboratory environment. Ideally, your fuzzer should be directly connected to the target, with no intervening systems—not even a switch.
The lab environment should be isolated from the rest of your network and the internet in case the tests are unexpectedly broadcast, amplified, or relayed.
The virtues of virtualization
Virtual machines are a good way to put fuzz testing in a controlled environment. Depending on configuration, the virtual machine has a very limited ability to affect its host environment, and the virtual network can be closely controlled.
In some cases, it might be possible to put both the fuzzer (such as Defensics®) and the target software in the same virtual machine, which would keep all fuzzing within a closely bounded environment.
Alternately, you could use a fuzzer on a different virtual machine, or in the host system, to test a target contained in its own virtual machine.
Containerization offers another opportunity. For testing application software that runs on a Linux kernel, containerization offers the control and isolation of virtual machines at a fraction of the resource consumption.
Using Docker containers as fuzzing targets
Aside from the controlled environment and isolation provided by containerization, specifying configuration and setup as code (in a Dockerfile) can help ensure repeatable, consistent testing and results.
Our example: mosquitto
Let’s say, for example, that you want to run the open source project mosquitto, an MQTT broker, in a container. We’ll walk through different scenarios of how you could use a container to do fuzz testing.
In the simplest case, you grab the binary artifact of your application and run it.
For mosquitto, we can simulate this by simply installing via apt-get. Here is a Dockerfile that makes it happen:
1 2 3 4 5 6 |
FROM ubuntu
RUN apt-get update RUN apt-get install -y mosquitto
ENTRYPOINT mosquitto |
client_send("hello",5)
client_send("hello",5)
I’ve been using a simple directory structure to keep things manageable. Each time I create a container as a fuzzing target, I use the following files:
- Dockerfile is the file that defines the container image
- build.sh actually creates the image from the Dockerfile
- name.sh simply defines the image name and is used from both build.sh and run.sh
- run.sh instantiates the container image and exposes network ports
For this simple example, name.sh is as follows:
1
|
IMAGE=mosquitto-box
|
The build.sh, script simply calls name.sh, then creates the container image:
1 2
|
source name.sh docker build -t ${IMAGE} -f Dockerfile .
|
Finally, the run.sh script makes an instance of the image and exposes the MQTT port 1883.
1 2 3 4 5 6 7 8
|
source name.sh docker run \ -it \ --rm \ -p 1883:1883 \ --hostname ${IMAGE} \ --name ${IMAGE} \ ${IMAGE}
|
You don’t have to copy and paste this. All the code in this article is available here:
https://github.com/jknudsen-synopsys/blog-box
This first simple example is mbox01.
To use it, just create the image with build.sh and run it with run.sh. Inside the container, mosquitto runs and waits for connections on port 1883. Because we’ve opened up port 1883, you can run Defensics (or any other MQTT test tools) and interact with mosquitto on localhost port 1883.
Building from source
If you’re integrating fuzz testing into your development process (and who wouldn’t want to do that?), you want to build your application from source so you can test the freshest code.
One of the very nice things about using containers as a testing target is consistency. Once you get the container image configuration correct, it’s very easy to get repeatable, consistent results with exactly how you’re building the application.
Again, we’ll use mosquitto as an example application. We prepare our container image with the dependencies of the mosquitto build, then pull the source code from the git repository and build it.
In the first example, mosquitto 1.6.9 got installed. When building from source, the current version is 2.0.6. Because the configuration is a little more secure, our Dockerfile has to make some changes to ensure mosquitto is listening on its external network and is willing to accept anonymous connections.
In the example code repository for this article, this is mbox02. This is what the Dockerfile looks like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
|
FROM ubuntu
RUN apt-get update RUN apt-get install -y build-essential git \ libssl-dev libc-ares-dev uuid-dev libcjson-dev
RUN git clone https://github.com/eclipse/mosquitto.git && cd mosquitto && \ make WITH_DOCS=no
RUN sed -i 's/#listener/listener 1883 0.0.0.0/g' /mosquitto/mosquitto.conf RUN sed -i 's/#allow_anonymous false/allow_anonymous true/g' /mosquitto/mosquitto.conf
RUN useradd mosquitto
ENTRYPOINT /mosquitto/src/mosquitto -c /mosquitto/mosquitto.conf
|
Juice it up with AddressSanitizer
You have already read how to use AddressSanitizer in conjunction with the Defensics Agent Instrumentation Framework to monitor memory leaks in fuzzing targets.
Setting this up in a Docker container requires a little more work but is well worth the effort. In the example code, this one is mbox03. However, it won’t work immediately until you put the Defensics Agent Instrumentation Framework agent server (agent_linux_amd64) in the files directory. Once it’s there, run build.sh as usual and it will get copied into the container image.
The Dockerfile takes care of copying over the agent server and also compiling mosquitto with AddressSanitizer enabled. In addition, it starts the agent server instead of starting mosquitto directly:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
|
FROM ubuntu
RUN apt-get update RUN apt-get install -y build-essential git \ libssl-dev libc-ares-dev uuid-dev libcjson-dev
RUN git clone https://github.com/eclipse/mosquitto.git && cd mosquitto && \ make CFLAGS=-fsanitize=address LDFLAGS=-fsanitize=address WITH_DOCS=no
RUN sed -i 's/#listener/listener 1883 0.0.0.0/g' /mosquitto/mosquitto.conf RUN sed -i 's/#allow_anonymous false/allow_anonymous true/g' /mosquitto/mosquitto.conf
RUN useradd mosquitto
COPY files /files
ENTRYPOINT /files/agent_linux_amd64 server –insecure
|
When you run this container, you must also expose port 12345 so that the agent server is reachable. Here is the updated run.sh:
1 2 3 4 5 6 7 8 9
|
source name.sh docker run \ -it \ --rm \ -p 1883:1883 \ -p 12345:12345 \ --hostname ${IMAGE} \ --name ${IMAGE} \ ${IMAGE}
|
Once the container is running, you just need to configure a Defensics SanitizerProcessMonitorAgent to run mosquitto, using the command /mosquitto/src/mosquitto -c /mosquitto/mosquitto.conf.
For more details, refer to the blog post “Find more bugs by detecting failure better: An introduction to SanitizerProcessMonitorAgent.”
A place for everything, and everything in its place
Hopefully this post has given you some ideas about the possibilities of running fuzzing targets in containers. The advantages of this approach are significant:
- Consistent configurations
- Easy setup and teardown
- Isolation of network traffic
- Consistent builds
Go forth and fuzz!
Have questions about fuzz testing?
Continue Reading
