To set the right expectations about tools, culture is more important in DevOps than tools. For a successful DevOps program you need to take more a holistic CAMS (Culture, Automation, Measure and Share) approach as you read in my previous article on DevOps. Also, before you decide on DevOps tools, building a business case might help you in securing the budget for the tools. While there is no simple way to build a business case for DevOps due to the complex and multi-faceted nature of these programs, the 2016 DevOps report from Puppet Labs is a good start to understand the financial reasons to adopt DevOps. You may specifically like to have a glance at, Chapter 7 - Understanding the ROI of DevOp, within this report.
Coming back to the topic of this article, let’s go through the tools used for DevOps as per various phase of the SDLC:-
Historically, centralised version control systems like CVS, SubVersion etc have been the standard for version control. The most obvious drawback with these systems is the single point of failure that the central servers represents. Whenever you have the entire history of the project in a single place, you risk losing everything. This is the reason distributed version control systems (DVCS) like Git, Mercurial etc have become very popular now days. Clients don’t just check out the latest snapshot of the files, they fully mirror the repository. Thus if any server dies, any of the client repositories can be copied back up to the server to restore it. These systems are typically used with commercial hosts like Github or Bitbucket and helps with pull requests, branching strategies, permissions and scalability.
Git, which is the most popular DVCS has a sophisticated branching model. Git allows multiple local branches that can be entirely independent of each other. The creation, merging, and deletion of these branches is very fast. With Git, nearly all operations are performed locally making is blazing fast. The data model used by Git ensures the cryptographic integrity of every bit of your project. Lastly, it’s completely free and open source.
What really sets Git apart are feature branches promoting collaborative coding. The fact that each feature is on a separate branch make it much easier to do code review and generally to collaborate on code. Rather than waiting for change approvals before deploying to production, you can improve code quality and throughput with peer reviews done via pull requests. Pull requests tell your team about changes you’ve pushed to a development branch in your repository. The code reviewer can then review the proposed changes and discuss modifications before integrating them into the main code line.
Continuous integration is the practice of checking in code to a shared repository frequently and running automated tests each time. This allows you to detect problems early, fix them when they’re cheapest to fix. With branch-and-merge workflows getting popular by using tools like Git, tools that take the pain out of running CI in a multi-branch environment help you to maintain the testing rigor without sacrificing dev speed. You should look for tools that automatically run your tests for secondary branches, and give you the option to push the code to master branch when secondary branch builds are successful. The most popular tool for CI is Jenkins though Bamboo is more popular for environments where JIRA is used due its tight integration with JIRA.
Unlike manual tests, automated tests are executed faithfully and with the same rigor every time. They also help in identifying the risk areas by producing reports and trend graphs. Automated testing is especially useful in an agile environment where the amount of regression testing exponentially increases as you move to later sprints. It really helps to speed up your development and testing cycles for long running agile projects. Depending on the type of testing various tools can be used e.g. JUnit and NUnit for unit tests, Selenium for GUI testing, SoapUI for API testing, Jmeter and Loadrunner for performance testing, Cucumber and JBehave for behavior driven testing, Jenkins and Bamboo for test orchestration etc. On a different note test driven development inverts the SDLC by starting the development lifecycle with test specs at each stage. Please refer to my article for a more details on agile testing techniques.
Besides the actual work of developing your application to respond appropriately in each environment, you may also face issues with tracking down dependencies, scaling your application, and updating individual components without affecting the entire application. Containers attempt to solve many of these problems. Containers are a way to virtualize operating system that allow you to run an application and its dependencies in resource-isolated processes. Containers allow you to easily package an application's code, configurations, and dependencies into units that deliver environmental consistency, developer productivity, operational efficiency, and version control. Containers help ensure that applications deploy quickly, reliably, and consistently regardless of deployment environment. Docker is the most popular containerization software in use today. These are some of the main benefits of Docker containerization:-
· Isolated lightweight execution environment
· Abstracting the host from the containerized application
· Simplified versioning and dependency management
· Composability and Predictability – Infrastructure as code
Kubernetes is a tool for orchestrating and managing Docker containers. While Docker provides the lifecycle management of containers, Kubernetes takes it to the next level by providing orchestration and managing clusters of containers. With Kubernetes, administrators can create Pods, which are logical collections of containers that belong to an application. Kubernetes and Docker deliver the promise of PaaS through a simplified mechanism. Once the system administrators configure and deploy Kubernetes on a specific infrastructure, developers can start pushing the code into the clusters. This hides the complexity of dealing with the command line tools, APIs and dashboards of specific IaaS providers. Developers can define the application stack in a declarative form and Kubernetes will use that information to provision and manage the pods. If the code, the container or the VM experience disruption, Kubernetes will replace that entity with a healthy one.
Taking the concept of continuous integration one step further, code is frequently deployed to a production like environment which helps in catching bugs and issues before they go to production. It also helps us to ship the code faster to our customers enabling us to provide more feature releases over a period of time. Start with automating deployments for your lowest-level environment first then replicate that all the way up to production. The most popular tool for CD as well is Jenkins though Bamboo is more popular for environments where JIRA is used due its tight integration with JIRA. These tools orchestrates complex deployments step by step and provides visibility into each environment’s history.
Infrastructure as Code
Provisioning tools like Puppet and Chef reduce the pain in standardizing environments and they can integrate with cloud-based platforms such as Amazon EC2, Google Cloud Platform, OpenStack, SoftLayer, Microsoft Azure and Rackspace to automatically provision and configure new machines. Though these are branded as configuration management tools but these are also Infrastructure as Code tools (IaC). IaC is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. IaC, which is more popular with cloud based systems helps with Cost reduction, faster execution speed and reducing the risk by removing errors and security violations while provisioning environments.
The tools discussed in this article is not a complete list and you should do your own research before deciding on the DevOps toolset. I hope my article helps you to embark on your DevOps tools journey. Please share your thoughts and comments.