Subscribe
All application development teams face the same fundamental questions, from the selection of third-party components to the processes and tools that ensure resilience and security. This article describes how the ISC development team addresses security in the BIND 9 application, one of the foundational applications of the modern internet.
About BIND 9 and ISC
BIND 9 is a name server, an application that translates human-readable internet names (so-called domain names) into their numerical equivalents. For example, if I type isc.org into my browser, my computer asks a name server for the network address corresponding to isc.org. My computer and the name server communicate using a network protocol called the Domain Name System or DNS. As a conversation, it looks like this:
My computer: Hey, what is the address for isc.org?
Name server: Ok, isc.org is at 149.20.1.66.
Now that my computer has the address, it can connect directly and ask for the ISC webpage.
The full capabilities of BIND 9 are far more complex and include caching, request forwarding, load balancing, multiple security protocols, and plugins.
BIND 9 is used in a large number of the world’s name servers. Because of this scale, and because listening for incoming network queries is inherently dangerous, security is critically important in BIND 9.
BIND 9 was developed by Internet Systems Consortium (ISC), a non-profit organization founded in 1994 for the express purpose of continuing work on BIND. BIND 9 is open source software available under the MPL 2.0 license.
One of the challenges faced by the ISC team is that BIND 9 is just plain old—in human years, it’s about 24 years old, which in internet years might as well be a million. Much of BIND 9’s 683,000 lines of code were written in a simpler time, when the internet was a friendlier place and adversaries were less skilled or plentiful. And with BIND being open source and freely downloadable anonymously, nobody knows how many sites are running it and on what hardware and software, how they are using it, and therefore what features or configurations should be tested. All of these factors make it imperative that the BIND 9 development team make maximum use of automated test tools.
Holistic security
Making BIND 9 a secure and resilient bedrock for DNS requires infusing security into every phase of development. With a development team of just nine people, three of whom are devoted to quality assurance, this is a significant challenge. As an open source project, however, BIND 9 can take advantage of valuable community resources.
The BIND 9 team runs testing based on two cadences of the development process: incoming changes to the source code (which happen as merge requests) and nightly pipelines.
Merge requests happen throughout each part of the development cycle. Typically, a developer creates a new git branch to implement a feature or fix a bug, then when they are done with their work, they submit a merge request to integrate their changes back into a main release branch. For each merge request, the BIND 9 continuous integration (CI) system runs a variety of tools, unit tests, and system tests—more than 70 jobs altogether.
Nightly testing includes longer-running items.
Static analysis
One powerful tool for security testing is static analysis, which encompasses any tools that examine the source code of an application to report on potential trouble. This category is often called static application security testing (SAST) and includes a spectrum of solutions. Very simple tools provide simple pattern-matching, while more sophisticated options build an abstract syntax tree for the code and can report on inter-procedural logic issues and follow data as it flows through the application.
For example, a simple SAST tool (or grep) can tell you if you’ve used an unsafe function like strcpy() when you probably should be using strncpy(). A better SAST tool can tell you if you have unreachable code, off-by-one logic errors, potential deadlocks, or many other types of bugs.
In a way, the compiler itself provides static analysis, as it reports on a variety of issues that might cause trouble in the application. The BIND 9 team leverages the power of the compiler by treating any warnings as compilation errors. Furthermore, the source code is built using two different compilers, gcc and clang. Warnings from either compiler are not allowed to go unaddressed.
Beyond this, the BIND 9 build pipelines incorporate tools that help ensure the hygiene of the source code. Coccinelle, for example, helps ensure consistency of the coding style, whereas Danger enforces conventions in the development process.
Comprehensive SAST capabilities are provided by the Coverity SCAN project, which delivers the full power of Coverity free of charge to open source projects. The BIND 9 team is proactive about addressing findings from Coverity, resulting in an impressively clean set of results: https://scan.coverity.com/projects/bind-v9_18
The BIND 9 team also uses a handful of tools that enforce clean, consistent code that behaves correctly. You might be wondering why they don’t just get one really good SAST tool, and the answer is simply that different tools find different results. Running multiple tools as appropriate, and addressing the results, provides the best protection for the application.
Think of it as preparing for an organ transplant surgery. Doctors and nurses scrub their hands and wear gowns, the surgical instruments are sterilized, the new organ is kept free of contaminants—all of these steps are necessary for the transplant to be a success. Failures at any point can lead to infection.
Testing
One of BIND 9’s strengths is a robust suite of tests, including unit, system, and performance tests. Build pipelines run tests in a variety of configurations, taking full advantage of compiler features such as AddressSanitizer and ThreadSanitizer.
The result is a matrix of testing, with unit tests running on binaries created with gcc and clang for multiple platforms. Unit tests are also run on binaries created with Address Sanitizer enabled, on binaries with Thread Sanitizer enabled, and so forth.
Fuzzing
Functional testing is, for the most part, positive testing, meaning that the tests are set up to supply valid inputs, and the output is compared against a correct or expected value. Functional testing verifies that everything works correctly under normal circumstances. But what about users or systems that send inputs that make no sense or are out of order? What about attackers who are probing the application for weaknesses?
Fuzzing is negative testing and is complementary to functional testing. It delivers deliberately malformed inputs and checks to see if a failure is triggered. This makes especially good sense for internet-facing applications like BIND 9, which are likely to encounter all kinds of garbage inputs in the real world.
BIND 9 is set up to use two open source fuzzers, namely, American Fuzzy Lop (afl) and libFuzzer. Both of these are in-process, coverage-guided function-level fuzzers. This means that they create test cases (inputs), run those inputs into an entry point function in the application, and see if a process crash ensues. They use code coverage to analyze the code pathway that a given test case follows. Test cases are mutated to create new test cases, and when a new pathway is discovered, a new set of mutations is performed.
Last year, the Black Duck Cybersecurity Research Center team performed DNS protocol fuzzing with Defensics during the development effort leading up to BIND 9.17.7 and uncovered a vulnerability. The BIND 9 team has a long history of collaboration with Defensics, going back at least as far as a vulnerability located in BIND 9.10.0 in 2014.
Defensics performs fuzzing very differently from afl and libFuzzer, but the underlying principle is the same. Defensics uses an internal network protocol data model to create test cases. It knows exactly how each field of each message should look according to the protocol specification, which means Defensics can create test cases that are very realistic but still anonymized in specific ways.
Defensics delivers test cases over the network, in effect working precisely on the exposed attack surface of the application. It monitors the tested application for process crashes but can also be configured to look for memory errors, other types of resource consumption, and other indicators of application health.
As with SAST, different fuzzers find different bugs, so running multiple types of fuzzers can help flush out vulnerabilities and reduce overall risk.
One way that the BIND 9 developers can reassure users their software is clear of obvious vulnerabilities is by publishing the results of these ongoing tests publicly, via “badges” posted in the open repository that link to the latest status. Note the badge in Figure 1 indicating a “passing” status with Coverity.
Figure 1: Results of ongoing tests are posted in the open repository with a link to the latest status.
Supply chain from both sides
Software supply chain security is a hot topic today, including everything from the idea for an application to its use in the real world. One important aspect of the supply chain is the concept of a software Bill of Materials (SBOM), a list of the open source components that are part of an application.
BIND 9 is somewhat unusual in that it does not have any other open source components built in. That said, it does have some dynamic dependencies in the software components that it invokes at runtime, including libuv, libcrypto, openssl, and a few others. As dynamic dependencies, these come from the runtime environment in which BIND 9 is deployed, and for organizations running BIND 9, the deployment environment is just as much a part of the supply chain as anything else.
For downstream consumers of BIND 9, the development team recently added SPDX license tags to its source files. This means that in addition to distributing the text of the BIND 9 license with the source code, each source file also includes an unambiguous SPDX-License-Identifier in a comment, like this:
1
|
/*<br/> * Copyright (C) Internet Systems Consortium, Inc. ("ISC")<br/> *<br/> * SPDX-License-Identifier: MPL-2.0<br/> *<br/> * This Source Code Form is subject to the terms of the Mozilla Public<br/> * License, v. 2.0. If a copy of the MPL was not distributed with this<br/> * file, you can obtain one at https://mozilla.org/MPL/2.0/.<br/> *<br/> * See the COPYRIGHT file distributed with this work for additional<br/> * information regarding copyright ownership.<br/> */
|
The license tag MPL-2.0 aids downstream consumers in identifying the source code licensing of BIND 9 and is easily detectable by automated tools. Although BIND 9 is distributed in source code form, more and more users are skipping the job of compiling it themselves, opting instead to install and deploy prepackaged images from various sources, including ISC packages, “volunteers” posting packages on the internet, and commercial application providers. This is precisely the sort of scenario the SBOM is designed for—in this case, BIND 9 would be one component listed in the application.
Security best practices for BIND 9
The BIND 9 team achieves a high level of robustness and security by using a process suffused with functional testing and security best practices:
- Solid code and process hygiene is built into the development process and enforced with multiple tools.
- Powerful SAST testing is regularly performed using Coverity SCAN and all results are addressed.
- A comprehensive unit and system test suite is run across a matrix of compilers and sanitizers.
- Multiple fuzzers barrage the application with badly formed and invalid inputs.
- Testing and security testing are automated in the development process.
All of these best practices will benefit almost every development team, but other types of security testing are appropriate as well. For example, many applications make heavy use of open source software components, in which case, software composition analysis (SCA) tools are crucial. For web applications, interactive application security testing (IAST) is an important dynamic testing technology.
Moving into the future, the BIND 9 team will continue to build upon its strong foundation of robustness and security by automating more tasks, doing more tests, and incorporating tools as they are available.
Continue Reading
