Subscribe
While the software industry celebrates a decade of DevOps, there’s an increasing drive toward adopting DevSecOps and making security a part of software from early on. Building secure software while keeping up with the speed and scale requirements of the market is a paradox for modern IT organizations. Companies often face a common set of challenges when moving from DevOps to DevSecOps, and they can be addressed by employing these DevSecOps best practices.
1. Foster a DevSecOps culture and mindset
There are several definitions of DevSecOps, but the one that stands out universally is collaboration, automation, learning, measurements, and sharing (CALMS), which was coined by Jez Humble and adopted further by Synopsys’s very own Meera Rao. At its core, DevSecOps thrives on a culture and a mindset in which various cross-functional teams share a single goal of continuous software security.
To embed a culture of DevSecOps, it’s best to start with a few self-motivated and committed teams that are aligned to the goals of strategic DevSecOps initiatives. The strategic initiatives act as guiderails for these teams while they work to ingrain DevSecOps culture into day-to-day functions, balancing security, speed, and scale. Once the pilot teams adopt DevSecOps and start showing visible benefits, they become examples to other teams that could follow their footsteps.
The key to fostering a DevSecOps culture and mindset is to operate in iterations and work upward from individual project teams to the entire organization.
2. Enable teams to build security in
While it sounds perfectly logical to “build security in,” it’s easier said than done. One of the key challenges that teams face is a lack of understanding and tooling or processes to help build security into their software. Enabling teams to achieve this goal is vital to ensuring that they are able to build secure software.
Ensuring that software is secure starts even before writing code for it. Security activities such as threat modeling and architecture reviews can help set the course for the security requirements and controls to be implemented during the software development life cycle (SDLC). When implementing the requirements and controls, giving development teams enough training on how to write secure code and fix security issues is of utmost importance.
Ensuring visibility into security vulnerabilities also helps create awareness and much-needed feedback loops in identifying and fixing those vulnerabilities. For example, one way to give immediate feedback on the code is to use IDE-based scanners to identify unsecure code right in developer’s workstation. Such tooling enables developers to code securely and fix vulnerabilities early.
3. Choose When first, not Which
Every organization that wants to integrate security into its DevOps workflows is likely to be torn between decisions about which security activities are needed and which type of tooling to buy. The key is to think first about when a security activity is performed in an SDLC. Each organization works in its own unique way when adopting DevSecOps, driven by its industry, maturity, and culture. The placement of security checkpoints will be unique as well.
For example, when developers have adequate training about coding securely, they often find it useful to perform security testing before code commits happen. That prevents developers from checking in unsecure code. For other organizations, the earliest starting point for security scanning could be in their central integration pipelines, which are normally triggered right after source code gets merged from developer branches to the main branch.
After determining when to perform security activities, each checkpoint can be used to indicate which security activity and tool is most applicable. In the example above, a pre-commit security scan or an IDE-based scan could be implemented to shift security testing further left in the development stage. Additionally, the central integration pipelines could have more security checkpoints implemented, with activities such as deeper static application security testing (SAST), software composition analysis (SCA), dynamic application security testing (DAST)/interactive application security testing (IAST), or penetration testing.
4. Automate tools and processes
Automation is key when balancing security integrations with speed and scale. The adoption of DevOps already focuses on automation, and the same holds true for DevSecOps. Automating security tools and processes ensures teams are following DevSecOps best practices.
Automation ensures that tools and processes are used in a consistent, repeatable, and reliable manner. It’s important to identify which security activities and processes can be completely automated and which require some manual intervention. For example, running a SAST tool in a pipeline can be automated entirely; however, threat modeling and penetration testing require manual efforts so they cannot be automated. The same is true for processes. Sending feedback to stakeholders can be automated in a pipeline; however, a security sign-off requires some amount of manual intervention.
A successful automation strategy also depends on the tools and technology being used. One of the considerations in automation is whether a tool has enough interfaces to allow its integration with other subsystems. For example, to enable developers to do IDE scans, look for a SAST tool to have support for common IDE software. Similarly, to integrate a tool in a pipeline, review if the tool offers APIs or Webhooks or CLI interfaces that can be used to trigger scans and request reports. Another consideration is a tool’s ability to be used “as code,” including configuration as code or pipeline as code, which can determine the level of automation that can be achieved throughout the SDLC. As an example, a containerized AST tool can be deployed and run easily in an automated environment, due to its infrastructure as code capability. Similarly, CI systems, like Jenkins CI, allow defining global libraries to provide pipelines as code feature, which enables global AST integrations across large number of CI jobs.
5. Start early and start small
Often when organizations or teams start integrating security activities and scanners in a DevSecOps pipeline, they tend to enable an overwhelming scope of rulesets and scan configurations. This hampers DevSecOps adoption in two ways. First, development teams suddenly see many security findings in their queues, which makes it impossible for them to address them all over a short sprint, and that causes reluctance to fix security findings. Second, that loss of support and acceptance from development teams can threaten the entire DevSecOps culture.
It is key, therefore, to start small and early. Security testing should begin as far left in the SDLC as possible and should be done with a gradually increasing scope. For example, instead of enabling full scans or scans with the entire ruleset for a pre-commit security checkpoint, teams should consider keeping the ruleset limited to its top five vulnerabilities. The security activities that occur later in the SDLC can include deeper scans and reviews for prerelease security assurance.
6. Tie in the out-of-band
While automation plays an important role in adopting DevSecOps, there are certain types of security activities that must be done out-of-band and manually. Usually these activities are performed on a predefined schedule, perhaps quarterly. But this can result in either overdoing or underdoing those activities.
Triggering out-of-band activities based on events in an automated pipeline helps balance the requirement. For example, results of SAST scans that are anomalous or cross a threshold could trigger a deeper manual source code review. Similarly, if a critical vulnerability is detected in a third-party component, a penetration test could be scheduled to assess its exploitability.
7. Treat security vulnerabilities as software defects
Security vulnerabilities are commonly reported differently than quality and functional defects. Often, organizations maintain the two types of findings—security and quality—in two different locations. This reduces the visibility each team and role has when they look at the overall security posture of their project.
Maintaining security and quality findings in one place helps teams treat both types of issues in the same manner and with the same importance. In reality, security findings, especially ones from automated scanning tools, can potentially be a false positive. It becomes challenging, in such cases, to ask developers to review and fix those problems. One solution is to tune the security tooling over time by analyzing historical findings and application information, and by applying filters and custom rulesets to report only critical issues.
8. Measure every step
Adhering to DevSecOps best practices means measuring success or failure. This is done by collecting data about the various factors and attributes contributing to DevSecOps and retrieving useful metrics from them. A comprehensive metrics program is one that includes people, process, and technology components holistically, and provides insight into success and failures. For instance, metrics should shed light into failures stemming from people not adopting well-defined processes, as well as failures from inefficient use of tools due to lack of defined processes. To achieve this, measuring and collecting relevant data at every stage of the pipeline and security activities is of paramount importance. Automation should also be utilized where possible to continuously and consistently gather the data points required for metrics.
There are numerous factors that can be measured and used to build metrics, such as the number and type of security scans performed throughout SDLC, the number of critical security and quality issues reported by an application, the time taken for pipeline execution, and more.
This data helps teams create metrics to help move toward an overall reduction in defect density, preproduction security defects, and mean time to deployment.
9. Learn from failures
DevSecOps adoptions are always iterative. During continuous integration and measurement, failures are bound to happen from time to time. Ideally these should be used to help teams learn and improve. Any DevSecOps pipeline that doesn’t report a failure or take an action on one is not truly embracing DevSecOps.
The “Three Ways” principle of DevOps is also directly applicable to DevSecOps.
- First Way: Never passing a known failure or a known critical security defect into downstream work, thereby ensuring success of the end-to-end system.
- Second way: Shorten the feedback loops by reporting any failure, including security, as soon as it happens.
- Third way: Build a more secure development culture by learning from past failures, iterating and tackling new failures/defects just as outlined in the first and second ways.
10. Pursue scalable governance
Traditional governance models significantly impede delivery velocity and are incompatible with the fundamental goal of DevSecOps—to ensure fast, safe, and secure delivery of software. As a result, along with security testing, governance activities should also be automated where possible.
Governance as code should be used to implement checks across the software delivery pipeline, and it should include required triggers for manual intervention to handle escalations, exceptions, and implementing compensating controls. As an example of governance, consider sign-off gates that an organization typically has before their releases. In many cases, the sign-off gates are the direct implementation of controls a project wants to have, so that state of security can be assessed before marking an important milestone in SDLC, a success. Collaborating, obtaining buy-ins, and enabling development and operations teams is key to ensuring that the governance model is inclusive and has the required adoption. Such enablement can be achieved using various feedback mechanisms, which include, but are not limited to:
- Pausing the pipeline builds
- Sending notifications
- Creating defects and tracking them centrally
- Breaking/stopping the pipeline build completely
Getting started
Transforming your organization to DevSecOps is a huge undertaking with many known and unknown challenges. It’s important to keep in mind that DevSecOps is not an off-the-shelf tool or a golden pipeline—it will take a roadmap to become a reality for any organization. Follow these DevSecOps best practices to lay the foundation. After all, Rome wasn’t built in a day.
Continue Reading
