Before getting into this post, for better understanding, I would recommend the readers to run-through my previous posts in the below order:
- Defining DevOps
- Continuous Integration and Continuous Delivery (CI / CD)
- Dev Vs Ops
- Continuous Testing and Continuous Monitoring
- Infrastructure as Code
- Continuous Deployment Vs Continuous Delivery
- IT Environments Management
Start with “Why?”
What does it mean to ‘adopt’ or ‘implement’ DevOps? What is a ‘DevOps Solution’?
- NOT a product.
- NOT a process.
- It is a philosophy that includes Principles and Practices effecting People, Processes and Tools.
Hence, implementing DevOps is not just about adopting a product or a process. It is about undergoing Transformational Change.
Organizations want to create innovative applications and features. They create these applications or services to solve a business problem. Either for themselves or for their customers/users. Whether they are a startup with an innovative Mobile App or a large financial institution with their complex transactional systems or a Military contractor building a real-time battlefield management system, they are all looking for a way to make what they build ‘better’.
‘Better’ can mean many things
- For the startup, better may be the number of new features they can get out to their users, fast. Quality may not be their highest priority.
- For the financial institution, better may be minimizing downtime for their systems every time they push out a new capability. The number of new features they release may be lower on their list.
- For the military contractor, better may be ensuring close to 100% reliability of their system. Innovative features may be trumped by quality and reliability for them.
So, the reason to adopt DevOps will vary considerably from organization to organization and even from team to team within the same organization. The team building the customer website may have very different priorities than the team deploying the new HR system. All reasons are good reasons, as long as it is not “it’s the buzzword the Senior Executive heard at the last conference.”
Some reasons for an organization to adopt DevOps:
- Time to value
- Speed of deployment
- Reduce cost/time to deliver
- Reduce time/cost to test
- Increase test coverage
- Increase environment utilization
- Minimize deployment related downtime
- Minimize deployment time issues (to avoid weekend long deployment marathons)
- Minimize roll-backs of deployed Apps
- Increase the ability to reproduce and fix defects
- Minimize ‘mean-time-to-resolution’ (MTTR) of production issues
- Reduce defect cycle time
- Reduce challenges related to Dev and Ops collaboration
In a nutshell, the reason(s) to adopt DevOps has to be a reason that provides value to the organization. The goals for DevOps an organization has will determine the practices and principles of DevOps to adopt.
I have worked with organizations whose biggest challenge was the limited ability to test often and thoroughly. Their QA environments took too long to be ‘refreshed’ for every test cycle, extending their test cycles to unacceptable lengths.
- What they needed to do was virtualize their test environments with ‘production-like’ systems, which could be provisioned and populated with ‘fresh’ test data in minutes/hours, rather than days it took them currently.
- For them, adopting Continuous Delivery to Test, by building a Delivery Pipeline that would allow them to automate the cycle from a developer kicking off a build to tests being run in a ‘freshly’ provisioned environment, would add tremendous value. This is where they should begin and limit their DevOps adoption too.
- Once they see value from this one capability, they can examine other areas of challenges for them that can be addressed thru DevOps.
Not Easy/Cheap as Said/Seen:
Implementing DevOps is NOT easy or for that matter cheap. Not in terms of cost of tools, people, etc., but in terms of the effort it takes. It would be a multi-month, or depending upon your existing maturity, even a multi-year project. The good news is that “if done right”, starting and progressing down the path can produce value pretty early.
The main reason ‘it is NOT easy’ is because of the transformational change the organization will need to undergo. The People part of the equation will take more effort than the Process or Tools.
So, assess the areas of improvement in the current delivery cycle. If you do not have the skills in-house, hire a consultant/consulting organization to conduct such an assessment. The assessment would help determine which areas in the DevOps spectrum of capabilities would provide you the highest ROI. That would address the key pain-points you have. Then create a roadmap for implementation.
The Need for Organizational Change
One of the key goals of DevOps is to reduce the existing gap between Dev and Ops. Whether you are a small organization with a small team of overworked Dev and Ops practitioners or a large organization where Dev and Ops are massive teams under different management chains altogether, this gap exists.
- For the small shop, it may be just getting Dev and Ops to find the time to talk outside of just deployment time.
- For the large organization, it would be creating communication channels outside the management hierarchies that separate Dev and Ops.
Either way, transforming how these two organizations collaborate and communicate is a requisite component of DevOps implementation.
- This does not mean the creation of a new cross-organizational ‘DevOps’ team.
- It means transforming how these teams interact, collaborate and communicate with each other.
- It means transforming how these teams are managed and potentially even how they are compensated.
Dev Vs Ops?
Most organizations have struggled with this Dev-Ops relationship/alignment. The reason for that is very much understandable.
- Dev is tasked with the job of producing innovative capabilities and automating business processes for speed and efficiency. Ops on the other hand is tasked with making sure the systems these applications run on are stable, available and responsive.
- Dev is measured on the change they create. Ops is measured on the stability they provide.
- Dev is ‘dinged’ for not producing change on time or not producing enough change. Ops is ‘dinged’ for instability or downtime of the systems or unexpected change.
- Dev wants to introduce new capabilities and features as soon as possible. Ops wants to keep stability at all times. Reasons enough to butt heads.
These differences are reconcilable. After all, their individual goals are just two faces of the same coin. Both Dev and Ops are there to provide a system that adds value to their customers and/or users. They just have to communicate and collaborate more to make sure they align and work together to make this happen.
Work as a team and Communicate enough to ensure
- Dev brings about innovation keeping Ops in the loop on the changes Ops will need to make to enable those innovations, and
- Ops need to get more involved in the processes of Dev and get more visibility into what Dev is building so they can preemptively make the required changes to deploy what Dev produces.
One of the reasons this is a challenge is that in most large companies Dev and Ops have been separated by a pretty large organizational ‘wall’. They are typically in separate management chains – Dev being in a CTO or VP of Dev’s organization and Ops in a COO or VP of operations’ organization. This results in hindered communications.
In outsourced organizations, Dev and Ops maybe totally different sets of (multiple) vendors who never talk outside of a change management tool and contract based negotiations of responsibility!
I recently met with a company where a project we were looking at had three different vendors responsible for development, QA and infrastructure respectively. This is a normal arrangement for the whole company, which has less than 20% of its IT staff made up of employees. Not much Dev-Ops collaboration going on there.
So, how does one do it? Adopting DevOps takes considerable transformational change in the organization. There are a few key principles in the DevOps space that must be used to form the basis of this transformational change.
- Do you see an effort to align Dev and Ops going on in your organization?
- What kind of Organization Change is going on?
- Does the way Dev and Ops teams are measured and compensated need to be re-aligned too?
Aligning the Dev and Ops Teams
DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. This is more important when adopting DevOps than any other capability or tool.
In my earlier post on the need to better align Dev and Ops and the challenges that such organizational change would address. This post discusses key guiding principles on which this Dev-Ops alignment should be based. They are designed to improve collaboration between these organizations and to break down the legendary ‘wall’ when it comes to the relationship between the Dev and Ops teams.
1. Shift Left:
Organizationally the goals of DevOps are to bring Dev and Ops closer. Not just at deployment time, as they may do already, but all thru the development cycle. It requires Ops to allow Developers to take more responsibility of the operational characteristics of the applications they are building. In turn, it requires Developers to keep Operational considerations in mind while building their applications. This is referred to in the DevOps world as ‘Shift Left’. As in shifting towards the left, some Ops responsibilities shifting earlier in the software delivery lifecycle towards Dev.
2. Collaborate across all teams:
The Dev and Ops teams typically use different tools to manage their projects, for change management, and outside of email different collaboration tools. In order to better collaborate across the organizational boundaries, these teams should start using the same Change Management and Work Item Management tools or use tools that are integrated. Thus allowing seamless visibility across tools and traceability between respective Change Requests. Real-time collaboration using a common tool is the best scenario.
3. Build ‘Application Aware’ Environments
As we move towards Software Defined Environments, we have the ability to build, version and manage complex environments, all as code. All of the benefits of this are debatable if the environments are not a perfect fit for the applications and the changes to the applications being delivered. The goal is hence to build Environments that are ‘Application Aware’ or are fine-tuned for the applications they are designed to run. More importantly, one needs to ensure that the environments are architected in a manner to allow for the evolution of the applications, both as they are developed, and as they are projected to change or as they may evolve in the future. This obviously would require close collaboration between Dev and Ops.
Development Architects and Product Managers need to work with the IT Architects and Operations Managers to Architect Environments and their projected evolution to align with that of the application. Not only that, but also allow enough resilience in the environments to allow for unexpected change.
For example, massive change caused by a super successful Application. Eg., Instagram had to keep changing their server environment almost daily as millions joined their service.
4. Environment Sprints?
Agile Development Principles prescribe that at the end of every Sprint, developers have a build. An executable that runs, even if does not do much in terms of functionality. It has been proposed that this concept be extended to environments.
This means that at the end of each Sprint, the Dev team would have an executable AND the Ops team would have an environment, a production-like environment built that the executable can be deployed on. This would allow the build to be tested in a production-like environment. This would also provide immediate feedback to the Ops teams on the behavior of the application in their environment and use that feedback to improve the environment.
This also provides an opportunity for the Ops and Dev teams to align better. They will both have to use a single Sprint plan for releases – of their Builds and Environments respectively and will provide feedback to both at the end of every Sprint, using which Dev can enhance their Apps to improve functionality and performance and Ops can enhance the environment for the same.
The Human factor
These principles are designed to foster Dev and Ops alignment from a teaming perspective, from a people perspective. These however do not replace the need for good old team building. Whether it is through face-to-face get-togethers or in today’s distributed worlds, through regular virtual meetings and online collaboration in real time. The best teams are the ones where the people know each other, trust each other, look out for each other and when there are challenges, know who to pick up the phone and talk to or (Video) chat through instant messenger.
Adopting Continuous Deployment
When adopting DevOps, Continuous Delivery is a core capability required. Continuous Deployment to Production is an optional capability that you may or may not adopt, based on your needs and constraints.
So, what does one ‘deliver’ when adopting Continuous Delivery (or Deployment)? Let’s look at it from the perspective of People, Process and Technology.
DevOps, at its core is a cultural movement. The people aspect of devOps is where it all started. When adopting Continuous Delivery, the most important part is the culture of continuous delivery – continuous collaboration between ALL stakeholders who need to be onboard to enable and consume continuous delivery. This includes Dev and Ops but also stakeholders across the SDLC including, LoB Owners, Product Owners, Analysts, Architecture & Design, Security, QA and Management.
It is important to create a culture where all these stakeholders not only contribute to the Continuous Delivery, but also consume the feedback that comes from the Continuous Delivery, at every stage.
- Dev uses the feedback to determine what to work on for the next Sprint
- QA uses it to test and validate functionality, integrations and performance
- Ops determines how the environment performed and where it needs enhancement
- Project and executive management to manage their project and release plans, and so on.
Above all, by taking on Continuous Delivery, you Continuously Deliver and iterate on culture – a culture of continuous collaboration, communication, trust and working towards common business goals.
When continuously delivering software, one is not only validating the functionality and performance of the software being delivered and the environments it is being delivered to, but also the process of deploying the software itself.
Deployment of the software is not as simple as copying some binaries over FTP. It involves file transfers, to multiple locations on potentially a complex set of nodes, but also involves configuration changes to OS, databases and middleware. It also involves an orchestration of steps.
One cannot simply do deployment steps in a mechanical linear manner. Middleware processes may need to be restarted after configuration changes. Services may need to be stopped before file transfers and then restarted, all in a coordinated orchestrated manner. Hence, continuous delivery allows for these processes to be tested and refined to ensure that when it comes to the final deployment to production, it is not the first time the team is executing the processes. They are tested and proven to work.
Remember, these deployment processes also includes the processes to deploy the environments, not just the applications.
What does one deploy?
- Deploying anything from simple configuration changes
- incremental code changes towards a new feature
- Database schema changes
- changes to the environment.
They all need to be deployed. These require a tool or set of tools that can automate the deployments and ensure continuous delivery of all changes from one environment in the SDLC to the next – to Dev, to SIT, to QA, to Perf, UAT and Prod. Tools like UrbanCode Deploy, ElectricFlow, and GO falls into this space, with its myriad plug-ins for application deployments, middleware configuration, databases and for environment deployments and configuration.
The key here is to start continuous delivery right from the start of the project – from “Sprint 0” – all the way through the project. In the beginning, the deployments may be simple, but much more complex orchestrated deployments come later in the project.
Continuous delivering changes – application, middleware, configuration, data and environment – in small pieces, using the right automation tools, reduces risk by validating the automation, the deployment processes, the configuration changes, the environments being deployed to and of course, the application being deployed.
Continuously Deploy – from Sprint 0
Last but not the least, revisit the statement “deploy right from Sprint 0”.
What does one deploy in Sprint 0?
There is no code yet. That’s easy, you just deploy the environment. Get the Unix / Linux / Windows distribution and install it.