Why Security is Important for Developer Operations

After working at my current gig for about a year now (internship plus full-time), I wanted to take some time to outline my experiences transitioning to more of a developer operations role from my existing security background. The transition has been fairly straight-forward, as much of the work I do currently touches security in some aspect. However, the biggest point I've noticed is how much security considerations are involved when developing tools for a traditional developer operations requirement.

While I am a big proponent of at least some security training for everyone in a technical role, it remains to be seen just how much is "enough." Obviously, as the realm of technical fields continues to expand at its current pace, having every member of a technical team fully trained in the security of the applications they are developing is impossible. Yet it is also imperative that they at least understand the risks associated with these applications; doing so should be a requirement of a good developer.

As a "Developer Operations Engineer" (a role whose title is still rather undefined and unstandardized), I generally focus on several categories of projects: infrastructure development, monitoring and incident response, and deployments and the application lifecycle. While this is a highly compressed view of my role, each of these has a dizzying array of security concerns. While some companies offload much of these concerns to a security team, smaller companies need to remain vigilant of their impact.

In most modern startup environments, infrastructure development typically refers to everyone's favorite buzzword: the cloud. Amazon Web Services, Microsoft Azure, Google's App Engine, the list goes on. The security concerns associated with the cloud are not the focus of this post, but I do want to highlight places where security is especially important. Almost every one of these services has security enhancements that are not used as often as they should be. For example, AWS's Virtual Private Cloud is not only free, but can also greatly improve security when used properly. Yet quickly starting instances from EC2 still requires less hassle and so remains the more common choice. Another example is the use of security groups. AWS's security groups are infinitely customizable, yet simply opening a port to the world (0.0.0.0) is a tempting simpler option. While hosted infrastructure providers like Amazon and Google abstract a lot of security work away from the customer, good security practices still require active participation.

Monitoring and incident response is perhaps the area in which a lack of security can have the biggest impact. While many "DevOps" engineers view monitoring in terms of system performance, monitoring must also cover system security. Disk space, CPU utilization, and memory usage are all important indicators of a healthy system, but so too are login attempts, changes to file permissions, and unauthorized outgoing network connections. A good monitoring platform must also monitor for security events that could signal an intrusion or potential breach of security. In the same light, the response to a security event should also contain a viable plan for mitigating the same risk in the future.

Finally, the deployment of applications is a critical component of the security of an infrastructure. Because the main goal of deployments from a developer operation's standpoint is automation, any security bugs introduced once tend to be replicated. For this reason, it is imperative that the deployment process and any actors in it (Jenkins, AWS S3, etc.) are fully secured and audited often. Vulnerabilities that are present here can expand exponentially when deployments are pushed.

The role of developer operations or server engineering is rapidly changing and expanding. While it does, it is important, if not necessary, to include security in the expansion and ensure that those building a company's most critical technological parts are also trained in protecting them.