An approach to reduce human intervention in operation and making the operations more agile

DevOps is an approach to IT delivery that combines people, practices and tools to break down silos between development and operations teams. DevOps teams accelerate the development of applications and services and, with a more responsive approach to management of the IT infrastructure, can deploy and update IT products at the speed of the modern marketplace.

DevOps bridges the gap between “dev” and “ops” — in other words, software development, where the code behind applications is created, and IT operations, where those applications are put into production, available to end users, and maintained. DevOps emerged from two earlier trends: The agile development movement and lean manufacturing principles. The former emphasizes short sprints of work and rapid iteration to create a more responsive IT development organization, and the latter minimizes waste and maximizes productivity in factories.

DevOps is the efficient integration of software development, quality assurance and IT operations, which were traditionally siloed teams.

DevOps solves a bottleneck problem associated with agile development. If agile developers are producing new software or code updates at a higher frequency, then traditional operations teams will struggle to get the software tested and live in a timely manner, and the actual value of rapid development is consequently lost. Ultimately, while the agile movement made the design and building of software more iterative and flexible, that approach did not extend through the full software development lifecycle (SDLC) into deployment.

As a culture or philosophical approach, DevOps is dedicated to continuous improvement, collaboration and transparency. DevOps sees IT operations holistically in terms of value. Its goal is not to focus on individual work silos, but on the entire flow from initial idea to available product or functionality — optimizing everything in between, with an eye toward achieving greater business value at a faster pace. High-performing DevOps teams see not only faster code iterations and deployments, but overall shorter time to market for new ideas, fewer bugs and a more stable infrastructure.

Below we’ll survey key DevOps concepts, and identify entry points for adopting a DevOps culture in your own organization.

DevOps: Development and Operations

Agile software development began with the “Manifesto for Agile Development,” a brief document published in 2001 by 17 developers. It promulgated four essential principles:

• Individuals and interactions over processes and tools.
• Working software over comprehensive documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan.

These principles challenged traditional waterfall development, which is rigidly sequential and exhaustively documented, and consists of sharply defined roles, tools and processes. But while agile development teams began to iterate software products more rapidly — responding nimbly to changing requirements and other circumstances — IT operations teams were not necessarily ready to move at this new pace.

In 2008, Patrick Debois of Belgium and American Andrew Clay Shafer began discussing the challenges of bringing “agility” to IT infrastructure. A year later, Debois held the first “devopsdays” event in Ghent, Belgium. From this event, the term DevOps entered the IT mainstream.

Three essential principles of DevOps, first distilled by author and DevOps proponent Gene Kim, are:

• Systems thinking: A more holistic approach to IT that encompasses developers, testers, infrastructure managers and consumers of code, or software and “emphasizes the performance of the entire system, as opposed to the performance of a specific silo of work or department.”
• Amplify feedback loops: Kim observes that “the goal of almost any process improvement initiative is to shorten and amplify feedback loops so necessary corrections can be continually made.”
• Continual experimentation and learning: As the Agile Manifesto values responsiveness and collaboration, DevOps values pursuit of continual improvement — finding new, better ways to work. Kim writes of the need to create “a culture that fosters two things: continual experimentation, taking risks and learning from failure; and understanding that repetition and practice is the prerequisite to mastery.”

A DevOps culture is founded on multidisciplinary teams continuously building and deploying new features and services. In DevOps, developers are responsible for maintaining the code they write, meaning they’ll resolve incidents more efficiently, as well as care more about baking reliability into everything they build. Furthermore, a DevOps team will focus on the lifecycle of an entire product, rather than focus on individually assigned projects, such as creating a new feature or designing a new web component. This product mindset (sometimes expressed as “products not projects”) helps IT better align with business value, prioritizing work that serves the business and the end user — driving better adoption and usage, deeper customer loyalty and higher business revenue.

What is the difference between DevOps and agile?

DevOps is the extension of agile principles from software development to software deployment. Where the evolution triggered by the Agile Manifesto addressed better ways to create software, DevOps applies similar principles and philosophies to the full software development lifecycle, carrying agile development through to release.

What is continuous development in DevOps?

Continuous development encompasses several DevOps concepts: continuous integration and continuous delivery (CI/CD), and continuous deployment. By constantly making, releasing and measuring small improvements rather than confining themselves to single huge releases annually (or less often), IT teams can constantly improve their offerings. Whether that offering is the core product of a software-based business like Netflix or Instagram, or an improvement to a website or IT service, this allows the business to remain competitive in a marketplace where innovative upstarts routinely disrupt market leaders — and entire markets.

What is continuous integration in DevOps?

Continuous integration is the practice of regularly incorporating new code into the main source code as individual tasks are finished. New code is checked into a central, shared repository, where an automated build will test and validate the changes. This surfaces problems quickly, gives immediate feedback to developers and lets them tackle necessary changes right away.

What is continuous delivery in DevOps?

Continuous delivery is the practice of testing new code as it’s integrated, extending the velocity of continuous integration. Continuous delivery automates the testing and staging of new code to prepare it for deployment.

The process may include unit, integration, functional and regression testing. When code is approved, it’s automatically staged for deployment. However, continuous software delivery does not deploy the vetted code automatically. That’s the purview of continuous deployment.

What is continuous deployment in DevOps?

Continuous deployment is the process of automating the release of new code as it’s integrated and approved through continuous integration and continuous delivery. Code that’s successfully tested through the automated delivery processes is staged and released into the production environment, making the new feature immediately available to end users.

Because the code passes from developer to end user without manual intervention, there are risks associated with continuous deployment. Generally, continuous deployment is best for lower-risk products or features. Where sensitive data, high security risk, regulatory obligation or great financial risk is involved, DevOps teams are less likely to employ continuous deployment. Outside of such considerations, continuous deployment is generally seen as a key goal of DevOps, because it provides maximum velocity and time to value.

Continuous deployment and continuous delivery, both frequently abbreviated “CD,” are often confused. But where continuous delivery delivers software that’s ready for release, only continuous deployment (or manual deployment) actually puts the updates into production for end users.

What is DevOps architecture?

DevOps architecture generally refers to the infrastructure that facilitates the goals of your DevOps culture; there is no single, prescriptive architecture for DevOps, and IT organizations don’t necessarily need to scrap their existing architectures to enable it. There are, however, core principles or best practices around architecture in DevOps organizations.

The rise of DevOps coincides with the rise of cloud platforms, and cloud and other virtualization technologies contribute significantly to DevOps success. Common technologies underlying or facilitating DevOps organizations include cloud platforms, virtualization and microservices.

• Cloud: Cloud platforms allow IT to provision resources in virtual environments (the cloud) instead of on-premises. These platforms allow users to access resources quickly and with great flexibility. (See “What is the role of cloud in DevOps” below.)
• Virtualization: Before virtualization, a server was a real piece of hardware, and if you needed a second server, you had to go buy one. Virtualization allows the creation of virtual servers (or desktops, etc.) so that rather than provisioning a new piece of hardware, software teams can create virtual environments on existing on-premises or cloud servers. This removes a considerable bottleneck for teams working at an agile pace, and automated provisioning increases the speed to the point where developers can access needed resources in minutes.
• Microservices: A microservice architecture is made up of loosely coupled units of finite functionality, which come together to compose a complete system. Larger systems or applications are easier to build and modify when composed of these single-function, reusable modules. This independent component approach lets individual DevOps teams deploy updates with fewer dependencies, meaning they can move faster with less risk of a change creating disruption. In their book, “DevOps: A Software Architect’s Perspective,” authors Len Bass, Ingo Weber and Liming Zhu note that “the microservice architecture style is designed to solve many of the problems that motivated DevOps.”

The architecture for a DevOps organization will employ cloud, virtualization and microservices. It will also seek to optimize testing automation, monitoring and security practices to enable DevOps workflows. In general, automation is a key element of DevOps and just as there is a cloud-first bias, there is an “automate it” bias. These biases or best practices will shape the architecture of the DevOps organization.

What is the role of cloud in DevOps?

Cloud computing is an essential enabler of DevOps culture. The speed, scale and efficiency of the cloud allows agile and DevOps teams to increase the speed and quality of their work. SaaS (software as a service) tools have brought both efficiency and economy to software teams — and SaaS software itself is an example of a constantly iterating product that all but requires a DevOps approach.

While you might not put every application into the cloud, a good DevOps practice is to think cloud-first for every new application. The question to ask when planning a new application or service is not, “should we put this in the cloud?” but “is there any reason not to put this in the cloud?” There will be reasons, sometimes — around security and regulatory needs, or based on your overall infrastructure — but cloud should be a default for a DevOps organization, not an afterthought.

What is DevSecOps?

DevSecOps integrates security practices into the DevOps process, making everyone accountable for security, rather than merely passing code to a separate security team at the end of the process. DevSecOps aims to deliver secure software faster, with fewer late-stage defects and less rework.

Generally, three fundamental principles of DevSecOps are:

• Introduce security thinking early in the development process.
  
• Introduce security thinking as a core development responsibility, not a separate team’s concern.
  
• Automate security processes to maintain the efficient flow of DevOp

A DevSecOps Manifesto posted in 2012 defines in greater detail the principles of DevSecOps in greater detail.

What are DevOps best practices?

DevOps best practices center on culture, describing an approach that is collaborative, flexible and focused on efficiency, agility and the delivery of end value. However, there is no single set of rules or best practices for putting this broad philosophy into action. Best practices and the value of DevOps will vary by industry, while the needs of a software developer will vary from those in retail, manufacturing, healthcare, financial services, etc.

Early DevOps advocates Damon Edwards and John Willis introduced an acronym, CAMS, to embody key development principles. It stands for culture, automation, measurement and sharing.

Additionally, the DevOps Agile Skills Association puts forth six principles for DevOps, summarized below:

• Customer-centric action: Short feedback loops to understand and incorporate customers’ perspective.
  
• Create with the end in mind: This drives organizations to think less in terms of each person’s strictly defined role, and more about how the overall organization creates IT products for internal or external customers.
  
• End-to-end responsibility: Rather than have each team member only responsible for their own role (giving developers no stake in infrastructure failures, and operations teams no accountability for bad code), DevOps seeks to eliminate those silos and make successful results the focus and responsibility of the entire team.
  
• Cross-functional autonomous teams: For each IT “product,” the team needs to have all the skills and responsibility to see the product through from idea to availability, and through cycles of improvement until the product is retired.
  
• Continuous improvement: DevOps teams always look to improve speed and quality and eliminate waste.
  
• Automate everything you can: To work faster, more efficiently and more effectively, it's crucial to leverage IT automation as much as possible, to free up team members to do more valuable work and reduce opportunities for human error.

What are DevOps tools?

DevOps tools help organizations implement DevOps principles by enabling the management of code development, testing, and infrastructure. DevOps is primarily about the culture around creating and delivering software products (see “getting started,” below), but the automation tools and other products are essential to bringing that cultural evolution to life.

Numerous tools exist that are expressly designed to facilitate DevOps cultural practices. There are tools for source control, configuration and release management, monitoring and more.

What is the best DevOps tool?

The best DevOps tools are the ones that serve the processes and people that form your DevOps culture. DevOps is not a product you can buy and implement and say, “Now we’ve got DevOps.” DevOps technologist Alex Honor published a post in the early days of DevOps titled “People Over Process Over Tools,” crystalizing the movement’s emphasis on culture over tooling.

That said, there are a number of notable products that help DevOps teams get the job done. A non-exhaustive list of prominent tools, both open-source and proprietary, includes:

• Source code management: Git (GitLab, GitHub), Bitbucket
• Configuration management: Puppet, Chef, Ansible, CFEngine
• Release management: Jenkins, Travis, CircleCl, TeamCity, Gradle, Bamboo
• Orchestration: Mesos, Zookeeper, Kubernetes
  
• Monitoring, virtualization, containerization: Nagios, Icignia, Monit, OpenStack, Vagrant, AWS, Docker, Kubernetes
  
• Log and application lifecycle analytics: Splunk is a leading log management tool ideal for DevOps.
  

How do you get started in DevOps?

To get started in DevOps, first consider your current culture. Identify the silos and bottlenecks that prevent rapid development, deployment, feedback and constant iteration to understand where to improve.

Andi Mann, author and chief technology advocate at Splunk, advises DevOps newbies to “decide on your path to DevOps based on your unique organization, but make sure to hit the critical waypoints that are key indicators for success.” Some guiding principles: Remove friction, eliminate bottlenecks. Embrace failure (and learning). Measure. Evolve continually.

The State of DevOps report, authored by Puppet, a leading DevOps software maker, and Splunk, breaks the evolution of DevOps practice into a five-stage journey. It measures progress according to the CAMS model — culture, automation, measurement and sharing. The report identified five foundational practices essential to DevOps adoption and success:

1. Monitoring and alerting are configurable by the team operating the service.
2. Deployment patterns for building applications or services are reused.
3. Testing patterns for building applications or services are reused.
   
4. Teams contribute improvements to tooling provided by other teams.
   

DevOps KPIs

Key KPIs include:

• Frequency of deployments: With the goal to do smaller deployments more often.
• Frequency of failed deployments: Aim for fewer.
  
• Features released per quarter: Doesn’t have to be quarterly, but that’s the timeframe most business leaders are thinking in.
  
• Mean time to discovery (MTTD): When things go wrong, how long until you notice?
  
• Mean time to recovery (MTTR): When things go wrong, how long until they’re fixed?
  
• Mean lead time (MLT): How long it takes from the time a piece of code is requested until it is implemented.
  
• Up time: Track availability, measuring both scheduled maintenance downtime and unplanned outages
  
• Defect escape rates: How many defects are making it into production versus being caught by your QA teams?
  
• Application performance: How does an application perform after you make a change versus before?
  

Once you identify your challenges and how you’ll measure progress, you can begin reshaping your culture. Start breaking down silos between developers, testers, operations teams and security analysts. Include business analysts in the conversation. Begin educating everyone that their goal is not only to clear out their own to-do lists, but to share responsibility for the overall speed and quality of software creation and delivery. Combine ice-breaker educational opportunities with changes to process and infrastructure.

And finally, it’s crucial to remember that DevOps is not something you’ll perfect in six months or a year. In the words of DevOps proponent Jez Humble, “DevOps is not a goal, but a never-ending process of continual improvement.”

DevOps is the future

DevOps is not a fad. And increasingly, it’s not an option. The idea of “aligning IT with the business” has been prominent, from board rooms on down, for well over a decade, and the DevOps philosophy and related practices has been well-established as a way to deliver better IT faster. While prominent “unicorn” organizations like Netflix, LinkedIn, Facebook, Amazon and Google have had spectacular success with DevOps, it is increasingly mainstream for more traditional organizations.

There’s no business today that isn’t trying to be more responsive in the marketplace. There’s no public-sector organization that isn’t trying to improve its services. Amid rising customer expectations and a proliferation of data, IT cannot sit still. And these days, nothing moves like DevOps.

What is the first step to approach in automation?

What is operating model transformation?

What is an operating model framework?

What is the product operating model?