Automation is one of the key principles of a strong DevOps culture because it has such a high return on investment. How can you be confident in your software unless you’ve had a real person validate it? We continue the DevOps Culture series with discussing how automation takes your team’s confidence beyond the commit and into production.
In our last blog post, DevOps Culture: Empowering Your Teams, we discussed how DevOps is more of an IT culture shift than a change in policy or tooling.

It Starts With Your Team

Your team is diverse, it likely includes some developers, maybe a cloud engineer, some management-types and some support staff. This entire group is affected by automating your deployments and testing. To solidify that fact, ask yourself some questions about your team dynamics:

  • Do your engineers introduce change manually?
  • Have your customers found defects before you did?
  • Are your deployments an “event” (i.e. late night, long hours & high stress)?

If you answered “yes” to any of these questions, then it’s time to look at some cultural shifts in how you automate your software. It’s very easy to get into a pattern of doing things by hand and “quick fixing” issues as they arise because you always promise yourself you’ll go back later and fix that hack. Yeah right. There’s never time to go back and refactor.
Here are some considerations with regard to automation that will help your teams start off on the right foot and avoid those late night stress parties.

Automate-First Mentality

An automation first mentality begins with accepting a new fact about your codebase: If it’s not committed to source control, it doesn’t exist. The strategy here is more of a mind game because the goal is to automate everything. By identifying this fact, we can place a clear and concise way of dictating this without allowing mental gymnastics to justify otherwise.
You can start right now with your next task. Whether a colleague regularly asks you to run a security scan or you pick up tasks from a kanban board — you can automate the work. Instead of shelling into that server, generating a report and emailing it over — why not write a script to do it? Your first bit of automation has set you up with some added bonuses:

  • Self-Service: Your colleague can now run their own reports
  • Repeatability: The report output will always be the same and can be easily improved

Uneventful Deployments

Software deployments are a normal part of the development lifecycle. Much like writing the code itself, it should be happening all around you without much attention being paid to it. For most of us, Continuous Delivery as I’ve just described it is our goal. It’s the key components we lack to get started and be successful. Getting to where deployments happen multiple times a day without a big commotion starts with Infrastructure-as-code and a culture of shared responsibility.

Infrastructure as Code

A core tenet of automation is infrastructure-as-code, especially when we’re deploying into the cloud. It’s nigh impossible to have fast repeatable resource deployments when it’s all done by hand. It’s also something that many organizations do, but not necessarily well enough to be completely successful.
Your infrastructure code:

  • Lives in the same repository as the application(s) it supports  — Put your automation code, deployment scripts and templates alongside your application code. Avoid hiding it in some “devops” folder and make it visible to everyone
  • Is managed by the software developers and cloud engineers alike — Don’t draw lines in the sand. Part of embracing the culture of devops is breaking down the walls between your team. Everyone is empowered to introduce change into the codebase.
  • Is just-as important as the software you’re deploying — Automation code is no different than your application code, it is your application code. It deserves your full attention to detail — unit tests, integration tests, security hardening and code quality standards.

In the beginning, it’s natural to make that divide in your teams hard to dissolve. Your cloud engineers want to retain control of their systems and your developers want to keep playing mad scientist. This is a great opportunity to introduce collaboration methods like pair programming and peer review to bridge that gap in initial adoption. Let your teams work together to address these challenges and they will self organize and let the automation do the hard work.

Test Automation

We’ve all been in the position where our code “runs on my machine” but is a disaster in production. The reasons are many for why this happens — sometimes it’s an integration issue and sometimes it’s because you forgot to run the tests before you deployed. Oooops!
Automating your tests is a quick win for improving the stability of your software and is a firm step forward in the direction of Continuous Integration. Continuous Integration, very simply, is everyone’s code being committed to a shared repository as frequently as possible and using an automated build to detect issues early. It reduces risk by quickly identifying integration problems, the so called: “Integration Hell”, by reducing the feedback loop to your development team. Successful continuously integrated projects are powered by automation — especially automated testing.
Before you even start writing your software, get the automation in place to create a build and to run your tests. There are tools out there that make this easy, or you can build your own. If your test automation is some shell script that is run by Jenkins on every commit — great! In fact, this is a fairly common starting point that often gets under appreciated.
In order for your test automation to work for you, it needs to:

  • Run quickly — All your build / test automation happens immediately upon commit. Close the feedback loop as much as possible.
  • Send feedback — Sound a klaxon in the office, post to a slack channel, or trigger an email. Do something to inform your team that there’s a problem with a build.
  • Fail the build and fail fast — Don’t be afraid to stop the build as soon as there’s a defect detected. Why wait? Allow the team to swarm on the issue and get it resolved.

Finally, in order for test automation to provide maximum value, your changes need to be small and incremental. Long gone are the days of 2,000 line code commits. Build features incrementally and deploy them to production as soon as possible. If this concept is new to you, consider Test Driven Development (TDD). TDD is a methodology to software development by which you translate acceptance criteria into unit and integration tests then write the supporting code. Your goal shifts from writing some large complex program to working on small chunks of the system that will naturally fit together — and as a bonus have great test coverage.

Legacy Code: It’s not too late

“DevOps is a ‘start from where you are today’ proposition. Do what you can to move the needle, and build momentum as you go.” – Tony Bradley, TechSpective.net

The reality is most projects are “legacy code” and greenfield projects are more like unicorns. These cultural shifts apply to teams working on either type of project because the project or tooling doesn’t matter. The goal here is to change how your team approaches problems and implements their solutions.
For example, let’s take automation and apply it to an existing in-production application. It would be very challenging to be successful to fully refactor an application and automate it in one fell swoop. Instead, implement an “automate-first” mentality with your project. The next time you find yourself manually performing some work — write a script to do it instead. It will feel great and that feeling is infectious.

Conclusion: Success Sells

“Don’t Let the Perfect be the Enemy of the Good”

The great thing about tackling challenging cultural shifts is that they are very contagious. It all starts with the individual buy-in on these best practices then it will spread rapidly. Your leadership can be focused on one person (or one small team) at a time and still be quite effective at creating a collaborative culture in your enterprise.
In our next post in this series we will discuss self-service and how it can further solidify the value in automation for your customers and the development team.

Stelligent Amazon Pollycast
Voiced by Amazon Polly