Understanding DevOps! A complete breakdown.
“If Ops goes to lunch with Ops and Dev goes to lunch with Dev, a low level of efficiency is a certainty.”
Hence to remove the certainty we need to have Dev and Ops working together and that is something taken into account in DevOps. Actually, it is a lot more than just about Dev and Ops. Before we dive into what and why of DevOps let’s recap what we were following before its arrival.
Traditional Software Development Process or waterfall model was getting quite inflexible with the passing of time, in fact, today, there might be only a few who can wait for months to see their product’s first version with a good chance of not meeting the expectations as by the time the requirements might have changed but due to the rigid structure of development It was very hard to adopt the changes.
Something was needed to be adapted or a transformation was in much need. Many things evolved over time like scrum, pair development.
All this lead to the formation of Agile and its manifesto which was very flexible and has the capability of adapting new requirements in the middle of the development cycle at the same time
giving the customer something to see with each iteration and they are in sync with how their product is being built. Organizations, people working in it started adopting agile methodology
following the different practices and they got benefited as well.
So the problem is resolved right!! Wait It is not about only developing the application, until it is deployed somewhere and accessible to the end-users it’s of no use.
And that is something taken care of by the Operation(Ops) team in the organization.
So the next hustle was the unwanted delays due to the separate working environment of the Dev team and Ops team and the differences between them.
Over a decade ago, development teams had already adopted mainstream agile methods and were releasing smaller software increments faster and more frequently,
while operations – upholding their corporate constraints around application availability and compliance – became a lucid bottleneck within the process.
To keep software development cycles fluid and deliver updates to the end-user at the speed of the business, operations had to follow this same agile trend.
Hence We were not able to benefit to the fullest even after adopting Agile methodology.
A brief history of DevOps:
Patrick Debois who coined the term for the first time was interested in learning IT from every perspective, and in 2007, he began working on a large data center migration where he was in charge of testing.
During this project, he realized that the frustrations experienced in projects like these are from the constant switching back and forth between the teams during the development lifecycle.
He recognized that plenty of time and energy was wasted navigating the project between these two worlds, but the divide between them seemed too wide to bridge.
To deal with this The idea slowly began to spread and after the DevOpsDays event held in Belgium in 2009, it became quite a buzzword.
What is DevOps?
- DevOps is a set of practices that combines software development and IT operations.It aims to shorten the systems development life cycle and provide continuous delivery with high software quality.
- In simple terms, DevOps is about removing the barriers between traditionally siloed teams, development and operations. Under a DevOps model, development and operations teams work together across the entire software application life cycle, from development and test through deployment to operations.
- DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes.
All the definitions mean similar in a sense and are absolutely fine, but the keynote here I would like to add that DevOps is not only about automating things to fasten the process,
or following some predefined best practices and using some tools to remove the barrier between dev and ops. Rather it is a culture and a mindset that needs to be adopted at each level of organization and the more we understand the more will get benefited from this culture.
Before going in details of this new trending culture lets see what devops can do for you or why you should follow it?
Well, the first reason is that it’s been shown to be effective in improving business outcomes. The Puppet Labs State of DevOps survey indicated the teams using DevOps practices deployed changes 30 times more frequently, with 200 times shorter lead times. And instead of that, resulting in quality issues, they had 60 times fewer failures and recovered from issues 168 times faster than other organizations. Those are huge benefits. And the survey also showed that these results help across different sizes and types of business.
The second reason is that it makes your daily life easier. High Tech is a very interrupt-driven high-pressure exercise in firefighting that can often lead to personal and professional burnout. It has been found that the DevOps approach reduces unplanned work. It increases friendly relationships between coworkers, and it reduces stress on the job.
Benefits of DevOps:
- Faster delivery time: One of the major benefits that DevOps provide is the faster delivery of quality products to the customers. It speeds up the pace at which you deliver your software features and modifications through automated testing and integration. It makes the developers and operations guys to keep an eye with continuous feedback and on the product throughout its entire life cycle for any software updates or bugs. This decreases the time to monitor, locate, and fix bugs, which accelerates your time to market.
- High collaboration between teams: We get a more effective team in the culture which emphasizes values such as ownership and accountability. Dev and Ops teams collaborate closely, share many responsibilities, and combine their workflows. This along with faster feedback system reduces inefficiencies and saves time (e.g. reduced handover periods between developers and operations, writing code that takes into account the environment in which it is run).
- Greater customer experiences: Ensure the quality of application updates and infrastructure changes so you can reliably deliver at a more rapid pace while maintaining a positive experience for end-users. Using practices like continuous integration and continuous delivery to test that each change is functional and safe. Monitoring and logging practices help you stay informed of performance in real-time.
- Continuous Release and Deployment: It benefits us by delivering the releasable artifacts at each iteration rapidly using an automated delivery pipeline. Disparate tools and processes increase operating costs, lead to context switching, and can slow down momentum. Yet with tools that drive automation and new processes, teams can increase productivity and release more frequently with fewer hiccups. Teams that practice DevOps release deliverables more frequently, with higher quality and stability. In fact, the DORA “2019 State of DevOps” report found that elite teams deploy 208 times more frequently and 106 times faster than low-performing teams.
- Agility: This is one of the most important benefits which compliments the Agile methodology of software development as it is very flexible and agile to adopt new requirements and see it in action and also get reviewed/tested directly by the end-users quickly. The feedback loop plays the key role here, notifying developers and other teams about a new requirement or issues/ bugs that arose through the help of continuous monitoring and tracking. The agile nature of this culture not only eases the task for dev or ops teams but also helps in meeting customer’s expectations even with frequent changes very rapidly.
- Competitive advantage: DevOps implementation can be the trump card in providing that competitive advantage. The major differentiating factors that help a business to be competitive are quicker software releases, high-quality features, continuous feedback incorporation, and maximized ROI. The benefits reaped through DevOps tick all of the above boxes. This results in a powerful competitive advantage over those who haven’t embraced DevOps.
- Cost Reduction: Following monitoring tools can help to track and locate the issues that arose and fixing them up quickly even before reaching an end-user thereby preventing downtime to a great extent. Making the repetitive tasks flow through an automated process in DevOps helps in cutting down some manpower costs as well. Additionally, Microservice applications are built on modular code, which is a DevOps best practice. It results in increased elasticity, which improves how your applications respond to varying workloads. That’s how it will help you reduce your infrastructure costs by a huge margin.
- Innovative mindset: We’ve seen how DevOps allows you to deliver your software products rapidly. Such rapid software delivery frees up some of your developers’ time, so they can instead spend it experimenting with any additional features or improving the effectiveness of the existing ones. Developers can validate the feasibility of such ideas proceeding according to the results with minimal disruption to the current project.
Aren’t these benefits great? Before going ahead let’s have a look at some common misconceptions about devops.
Common devops misconceptions:
- DevOps=Dev + Ops: Well it’s getting as a default understanding, especially for beginners that working together of Dev and Ops team coherently is what we say as DevOps. This not only involves Dev and Ops team but other teams as well like QA, Security, and other stakeholders who all are involved in the software development life cycle directly or indirectly.
- Tools: DevOps requires a cultural shift beyond simply implementing a new lineup of tools. Each department, technical or not, needs to understand the cultural shift behind DevOps. It’s one that emphasizes empathy and better collaboration. No doubt we use tools to fasten the process as per the needs in the organization but that is not only the main aspect of DevOps.
- DevOps is Not a technology: It is not something which anyone can learn like any programming language or any other technical thing, and implement the next day. DevOps is the correlation of people, processes, and products to enable continuous delivery of value to end-users. The outcomes are tightly connected to allow for frequent releases and at the same time to keep the same level of quality.
I hope by now you have understood what and why of devops. Next is values of devops.
The CALMS Model:
- Culture is a choice made by the people within an organization. One decided and having a common goal set requires people to start following it with learning, sharing, and experimentation. Even with the best tools, DevOps is just another buzzword if you don’t have the right culture. The primary characteristic of DevOps culture is increased collaboration between the roles of different teams working in an organization.
- Next is A and That’s Automation. Automate where you can. Automate where it provides value. Through automation, you can reduce human error and increase speed. Let automation do repetitive tasks. This will maximize the availability for people to focus on tasks that require thought. Automation is a critical part of our DevOps journey. Once you begin to understand your culture, you can create a fabric of automation that allows you to control your systems and your applications. Automation is that accelerator that’s going to get you all the other benefits of DevOps.
- The L in CALMS is Lean. The fundamental philosophy of lean is about recognizing which activities you and your organization perform that add value to the product or service you produce, and which do not. Anything that does not create value for our customers is a waste. Lean gives us guidance to review our process. Above all, unused features should be removed Lean encourages teams to deliver fast by managing flow, limiting the amount of WIP (work-in-process) to reduce context switching and improve focus. The Puppet Labs State of DevOps Report identifies lean management as one of the two key practices that contribute most to organizational success. DevOps can use the benefits of Lean methodologies into operations by reducing the barriers to delivering more value to the customer (Lean) and aligning with the business. This results in a better running development team, smoother operations, and ultimately happier customers.
- The M in CAMS stands for Measurement. One of the keys to a rational approach to our systems is the ability to measure them. All right, there are two major pitfalls in metrics. First, sometimes we choose the wrong metrics to watch. And second, sometimes we fail to incentivize them properly. Because of this DevOps strongly advises you to measure key metrics across the organization. Look for things like MTTR, the Mean Time to Recovery or cycle time, look for costs, revenue, even something like employee satisfaction. All of these are part of generating a holistic insight across your system. These metrics help engage the team and the overall goal. Data-driven metrics helps organizations make smart decisions.
- Last is S for Sharing. Sharing ideas and problems is the heart of collaboration. And it’s also really the heart of DevOps. Sharing forms a feedback loop that helps continuous improvement leading to a better quality environment and product to deliver.
All right so this was CALMS, Culture, Automation, Lean, Measurement, and Sharing.
So we understood the core values of DevOps. There is not a definite path to follow DevOps. As this is a culture, it depends on organizational needs and how they are going to adopt the changes.
Some key practices in DevOps:
- CICD: Continuous Integration and Continuous Delivery is basically an automated process through which the developer can deliver the code changes more frequently followed up by automated testing and integration forming up a pipeline to deliver the quality product very quickly. The main idea behind CI is a consistent and automated way to build, package, and test applications. With consistency in the integration process in place, teams are more likely to commit code changes more frequently, which leads to better collaboration and software quality. While CD automates the delivery of applications to selected infrastructure environments (development, testing, QA, production).
- Automated Testing: Testing is something unavoidable and as this is a repetitive task, multiple times during the development cycle, we can automate this to fasten the process. Test Automation software is the best way to increase the effectiveness, efficiency, and coverage of your software testing. There are various tools for automating and having a different type of tests like Unit and component testing, functional testing, regression testing, and others in our cicd pipeline.
- DevSecOps: This brings Security as key aspect from beginning. In the course of rapid delivery, we can’t compromise on Security. Hence DevSecOps focuses on having a mutual understanding of security aspects from start and automating some security gates thereby preventing the DevOps process from slowing down.
- Infrastructure as Code(IaC): The process of managing and provisioning of infrastructure through some scripts(code) using various IaC tools and development techniques, thereby automating this process is known to be as Infrastructure as code. This helps managing servers, operating systems, storage, and other infrastructure elements without physical intervention and provides a greater speed, less risk, and reduced cost.
- GitOps: GitOps is an extension of Infrastructure as code. It leverages Git as the single source of truth, and control mechanism for creating and updating system architecture by storing the desired state of the system in version control so that one can have a view on the audit trail of changes.
- Continuous Monitoring: It helps organizations to track, locate understand key metrics, and also resolve application or infrastructure issues in almost realtime. It helps to maintain the health check of the overall system. The respective developers or other stakeholders can be notified even before happening of some major fall down by setting up some thresholds up to which the system can resist and action can be taken accordingly.
- Microservices: Microservices is an architectural style that structures an application as a collection of services that are. Highly maintainable and testable. Loosely coupled. Independently deployable. Organized around business capabilities. The microservice architecture enables the rapid, frequent, and reliable delivery of large, complex applications, and it’s an excellent idea, in DevOps to follow this architecture especially in the case of complex applications.
Some famous tools being used in DevOps:
- Git: Git is a distributed version control system tool that helps to track the progress of development workflow. It allows rolling back to any previous version whenever needed.This is widely used as the SCM tool across many organizations. GitLab, Github and Bitbucket are remote Git hosting services.
- Jenkins: Jenkins is the most famous CI/CD tool. The main reason for its popularity is having great support of libraries which helps in building and automating the delivery pipeline and integrating other tools easily with it.
- Docker: Docker is the most widely used for containerized applications. It made containerization very popular in the market like quite often people think of containers as docker only. Docker containers are OS independent and hence portable to use anywhere.
- Kubernetes: It is a container orchestration tool for deploying, scaling, and managing container-based applications across a cluster of systems. Docker Swarm is another container orchestration tool.
- Selenium: Selenium is a portable software testing framework for web applications. It provides an easy interface for developing automated tests.
- Puppet: Puppet is a cross-platform configuration management tool. It helps to manage and automate software inspection, delivery, and operation
- Chef: Chef is another powerful configuration management automation tool using which you can transform infrastructure into code. One major difference between Chef and Puppet is that Puppet uses a declarative method(JSON or XML) while Chef goes in an imperative way. Ansible which is another famous configuration management tool also uses the declarative way.
- Nagios: Nagios is one of the most famous DevOps monitoring tool. Nagios offers monitoring and alerting services for servers, switches, applications, and services.
DevOps engineer, what all things one do?
DevOps brings with it one new position in the organization for a person who has a good understanding of this culture and can help to bring the transformation.
A person who has the skills that span both development and operations, as well as interpersonal skills to help bridge divides between siloed teams can be a DevOps engineer.
Somebody who understands the Software Development Lifecycle and who can automate the process, building up the pipeline right from development to delivery using various tools out there.
They are either developers who get curious about deployment and network operations or sysadmins who have a passion for scripting and coding and enter the other side where they can improve the design of test and deployment.
They look through code releases and deployments through Continuous Integration and Continuous Development/Delivery (CICD). Infrastructure as code, automated processes, scalable, robust, and tailored to your needs solutions are their expertise areas.
Some core responsibilities of a DevOps engineer:
- Understanding customer’s requirements.
- Project planning and designing the roadmap of the development lifecycle.
- Understanding and hands-on over various development. testing and infrastructure automation tools.
- Setting up a CICD pipeline.
- Troubleshooting and fixing bugs arose during the process.
- Monitoring the process, creating required feedbacks or notifications based on that.
- Encouraging and try to automate wherever possible.
- Excellent coordination and communication within the team and with customers as well as required.
- Managing stakeholders.
- Suggesting and setting up required tools for automation process.
- Sharing knowledge and mentoring team members wherever required.
- Managing periodic reports to the management, stakeholders as well as customers. This also can be automated.
All right, So this was a brief overview of the trending Culture which has been successfully embraced by many organizations getting huge benefits from it, while many are in line to adopt it.
At the end DevOps is something which helps meeting customer’s expectation in the fast going world where requirements keep on changing frequently and at the same time providing
a quality product rapidly with great customer satisfaction, At the same time, it is not everything.
Thanks for having your time.
Happy learning 🙂