Continuous Delivery Foundation cdCon 2021, 6 Jul 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Sponsored Lightning Talk: How Capital One Improved the Developer CI/CD Experience, Roderick Randolph

Description

Sponsored Lightning Talk: How Capital One Improved the Developer CI/CD Experience, Roderick Randolph, Capital One
Speakers: Roderick Randolph
At Capital One, we've built an innersource CI/CD pipeline framework using open source best practices that empowers developers to deliver software through automation. As the framework scaled over time to support thousands of applications with hundreds of contributors from many teams, the overall developer experience also needed to evolve. Today we can build and deploy faster, identify impediments using agile practices, and create better outcomes for development and operations. During this talk you will learn how we scaled our CI/CD framework and adapted our developer experience to improve efficiency and productivity, and you will walk away with tips to delight your developer's CI/CD experience to unlock greater developer productivity.

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

A

Hi, my name is roderick randolph and I'm excited to talk today about how capital one improved the developer, ci cd experience, but first, let's meet mary mary as a software engineer, who has developed a business application and wants to quickly and easily deploy her code into production.

A

When mary commits her code, she fully expects her ci cd pipeline to automatically compile her code report. Any security findings execute her tests and safely deploy her code into production. Mary also expects the pipeline to give her feedback. When something unexpectedly happens, mary becomes impatient when the pipeline is slow or she's unable to easily find or troubleshoot an issue when something goes wrong.

A

The more time mary spends trying to get her software out to customers the less time she's spending developing new business features.

A

How happy or satisfied mary is about her ci cd pipeline is part of her developer experience, and this is what I want to talk about today. How can we improve the developers cicd experience at capital, one? We built our own internal, centrally managed ci cd pipeline framework built on top of open source software.

A

Our framework started small in a remote office in toronto and over the past several years it has grown to support the entire enterprise. Personally, watching the framework grow and scale at capital, one has been mind-blowing.

A

We built the pipeline framework to make it really easy for any developer, to be able to build tests and deploy their application into aws through automation and with the least amount of friction as possible.

A

We've made our ci cd pipeline framework available as an inner source. Product intersource means that any developer within our company's walls is able to submit a github issue or open a pull request to get a bug fix or to add a shiny new feature to the framework over time. As the number of inner source contributions increase and the framework adoption grew, the developer experience needed to evolve and improve as well.

A

We noticed that pipeline execution times were creeping up. Error messages were sometimes difficult to decipher pipeline logs were too verbose and so forth. Cicd pipelines are just like any software application. They do collect technical debts and need tender, loving care too. So recognizing an opportunity to improve the developer experience. We sprung into action.

A

Our approach consisted of the following high level: steps step, 1, solicit, developer feedback, step, 2, prioritize, an area of focus, step, 3, implement a solution, step, 4 track and measure progress and step 5, rinse and repeat: let's go over each step, starting with step number one soliciting developer feedback at capital, one we treat our pipeline framework like a product and our developers as internal customers.

A

We are product owners and product managers for our framework and to get feedback from developers. We utilize a number of different forums to engage with the community. We have empathy interviews, we have public slack channels, we have jam board sessions, developers can submit github issues, we conduct surveys and so forth. We gain invaluable insights about how develop how developers use the pipeline through these forms by learning what works well and what could be better.

A

So the key takeaway from this step is adopt a product thinking, mindset and getting develo and getting feedback from developers is going to be invaluable and understanding the developer experience and how to improve it.

A

Once we receive the feedback, we then move on to step number two prioritize, an area of focus.

A

We won't be able to solve all the challenges at once: developer experience spans the entire software delivery, life cycle and multiple aspects of ci cd processes. So we want to focus on areas where we can have a significant and immediate impact.

A

Improving pipeline execution, speed and error handling were the two initial area focus for us. If we could reduce pipeline execution times and reduce the amount of time developers spent diagnosing error messages, we knew it would be a huge productivity boost, plus it would enable even faster release cycles for new business features.

A

So the key tip here is not to try to solve all the feedback or developer experience challenges at once, but identify one or two key areas. To start now, step number three implement a solution.

A

This is the fun part. This is where we get to actually implement something that will have a real change on the developer experience. So diving deeper into one of our focus areas around improving error handling. We wanted to implement a solution where we could make error messages more actionable before a developer would have seen an error message like this.

A

This error message informs the developer that their application target group already exists and caused their pipeline deployment to fail, but it leaves the developer hanging on what exactly do I do next, but now we've improved the error message by providing additional context with clear instructions on what to do next and adding something something as simple as emojis to the message just adds that extra touch to the developer experience, so the key takeaway here is you, don't necessarily need to start from scratch or create some elaborate design to improve the developer experience, something as simple as enhancing error messages to be more action.

A

Oriented is just as or even more effective.

A

Moving on to step, four track and measure progress in our ci cd pipeline framework, we have implemented a logging and analytics capability that enables us to observe how our framework is being used and its performance.

A

This enables so much insight and observability because we're tracking every pipeline stage and every task within the stage if it moves we track it. Remember when I said we wanted to reduce the pipeline execution times well through our analytics, we're able to see in the code where much of the execution time was being spent.

A

It helped us identify sections in the code that were inefficient or were just no longer necessary. We love it when we can remove code and speed up the pipeline. In some cases we were able to reduce the end-to-end pipeline execution time by 80 percent.

A

So you might ask what happens when the pipeline is faster? What are developers doing well through our analytics and tracking? We saw developers pushing even more changes to the pipeline, which means more releases are going out the door, so the key tip here is to simply log and track everything, so you can see your progress finally, step 5 is rinse and repeat, in other words, keep it going solicit more feedback from developers prioritize the next area of focus, implement those solutions, measure the progress and so on and so forth.

A

In closing the best part about ci cd to me is the continuous part improving the developer. Experience is not a simple check-to-box task. It's a journey filled with continuous improvements. We never stop improving.

A

Circling back to the beginning with mary, our developer. She is counting on us and those improvements every single day, and with that my name is roderick randolph, I'm a distinguished engineer at capital, one feel free to connect with me on linkedin. Thank you for listening and enjoy the rest of your conference.