GitHub GitHub Galaxy 2023, 11 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Harness your organization's productivity from day one with enterprise-ready tools

Description

What gets in the way of developer productivity? For many organizations, things like tech sprawl, lack of visibility from siloes, and time-consuming manual processes can prevent developers from focusing on what matters most: shipping new and innovative features. Learn from Mark Huber, Senior Director of Engineering Enablement at Cox Auto, Andrew Foster, Director of DevOps Engineering at Humana, and Joyce Wang, Director of Software Engineering at LinkedIn, on how shipping new features and product innovations faster helped them meet business goals. Mark, Andrew and Joyce will cover how they removed roadblocks and increased developer productivity in their organizations, and how they’re deepening their usage of their developer platform by building an innersource culture.

Stay up-to-date on all things GitHub by subscribing and following us at:
YouTube: http://bit.ly/subgithub
Blog: https://github.blog
Twitter: https://twitter.com/github
LinkedIn: https://linkedin.com/company/github
Instagram: https://www.instagram.com/github
TikTok: https://www.tiktok.com/@github
Facebook: http://fb.com/github

About GitHub
It’s where over 100 million developers create, share, and ship the best code possible. It’s a place for anyone, from anywhere, to build anything—it’s where the world builds software. https://github.com

A

B

A

Looks like we are live well, welcome. Everybody, I, wanna, say good morning good afternoon good evening, depending on where you are and when you're watching this. uh My name is Dave burnison I'm, a senior Enterprise advocate here at GitHub and today, we've got this fireside chat. Talking about harnessing your organization's productivity from day, one with Enterprise ready tools, so I'm coming to you from Redmond Washington, uh please drop and chat. Let us know where you're coming from so a little bit about me, how did I land to GitHub well I've been a devops junkie.

A

Most all of my life I've gone from managing devops teams at a software company to Consulting on devops culture processes and tools, to selling devops tools to becoming an Enterprise Advocate working as a link between our customers and the product group.

A

So a couple of housekeeping things before we get started here. If you have a question for our speakers today, please be sure to drop it in the Q, a tab and also watch for new questions from others and vote on the questions. You'd like to see our panel answer today, we'll be addressing your questions throughout the event. We've got a few questions to get us started here, but then yeah. We definitely want to make this Interactive.

A

uh Some tips on the the viewing experience be sure to join us from your desktop. We recommend using Chrome, Edge or Safari Firefox will work as long as you're using the current version, and if you aren't able to see the stream try, refreshing the page and, if you're having trouble and are on a VPN, try disabling the VPN and refreshing the screen or try logging in by dropping the event link into an incognito window and finally for troubleshooting.

A

If you need help with issues on the platform, please invent go over to the docs tab on the far right to find some help documents or you can also email our partner team at goldcast all right. Well, we have had a fantastic three four days of content here. You know it started out during the keynote with Thomas in ball, Allison and Mario, talking about where we're at today with GitHub and where we're going in the future.

A

There's spend lots of talk and enthusiasm about GitHub co-pilot and during the remove developer roadblocks and streamline productivity with GitHub Kevin allwell did a great overall demo uh on GitHub and we had Insight from uh other Enterprises using GitHub on how Native security delivers customer value and optimizes the developer. Experience really encourage you to go check that out uh when, when the videos become available next week now this afternoon we've got increasing developer velocity with GitHub Enterprise.

A

My colleague, John Peck will be hosting and moderating that session and, like we said all of these sessions will be available next week.

A

So watch for an email for us where you can come back and watch the recording again share them with your colleagues and and such all right so joining me today we have Mark Huber uh from Cox Automotive, the senior director of engineering, enablement, Andrew Foster from Humana director of devops engineering and Joyce Wang LinkedIn, director of software engineering, all right I'm going to stop sharing my screen, so we can get right to the chat and everything so uh mark. Would you like to introduce yourself tell us a little bit about sure a bit about yourself.

C

Thanks Dave, uh like Dave, said, my name is Mark Huber uh I am the senior director of the engineering enablement organization here at Cox Automotive a little bit about Cox Automotive. um There may not be a name that you're familiar with, but you probably are familiar with a number of Our Brands. uh If you've heard of AutoTrader or Kelly Blue Book uh we're a huge part of the automotive industry, there's also dozens of Brands within the automotive industry that you may have never heard of.

C

uh But groups like Manheim Auto Auctions runs uh some of the largest wholesale auto auctions in the world.

C

uh Our dealer track platforms serve the uh financing uh practices within dealerships and basically all all parts of the the automotive life cycle is part of the the Cox Automotive family's Mission, and our mission really is to change the way the world buys sells, owns and uses Transportation uh in in our world, um and so I've been uh very privileged to to lead an organization uh who provides for the the needs, the practices, the tools, the techniques of delivering product in that space.

C

Our organization uh of about 40 Engineers supports over 500 product development teams, uh delivering software and solutions and all of those spaces, uh and uh we are uh big users and fans of uh of the GitHub ecosystem. In doing so,.

A

Awesome awesome. Thank you Andrew. You want to introduce yourself and tell us a little bit about what you do sure.

B

Dave, thanks for the opportunity uh like Dave, said, I'm Andrew, Foster I lead uh the devops organization at Humana, um similar to Mark. uh My organization's name is actually Enterprise engineering enablement as well and uh Mark. We have never talked before so uh just interesting. Naming um I uh Humana is a fortune 41 uh health and wellness company we're more than just traditional insurance.

B

We focus on what we call human care, so caring kind of for the whole person we're the fifth largest Health insurer in the United States, and we serve around 5.1 million individual Medicare Advantage members, as well as many of our active duty, military families and Veterans. um So that's kind of a little bit about humana's business. My team supports 5000 plus Engineers across humana's various lines of business, um and we run everything from um you know: kind of your traditional open systems, devops as well to Mainframe and Salesforce devops as well.

B

So Ximena has a very diverse technology ecosystem that we care for save.

A

B

A

And, of course, last but not least, Joyce, please introduce yourself and tell us what you do.

D

Hey y'all, um so I'm based in California um I, lead the developer ux team at LinkedIn, um so my team actually owns many of the uis that are entry points into our developer tools and so right now our main initiatives are actually migrating. Linkedin to GitHub and we've been working very closely with the GitHub folks on doing this. um It helps all being under Microsoft and we're also building a developer UI platform on top of uh spotify's backstage, which is essentially an open platform for building developer portals.

D

um And since my team owns many of these front door, uis our focus is to make sure that users can navigate through our tools. You know find what they need and they won't follow up. This recommended path that we have for them, which we actually call The, Golden workflow pet golden workflow path.

D

um Yeah back to you, Dave.

A

Oh great, very good, very good, all right well I'm, going to start with a question for each of you and that is well two-part question. How long have you been using GitHub? How many developers do you have using GitHub today and- and actually you know, do you? How many do you hope to have in the future?

A

So we'll start with we'll start with Mark.

C

Sure um so the instance we use get GitHub Enterprise server uh here at Cox Automotive and in the main instruments that we we use collectively in our organization, is about eight years old, uh but we have actually had uh several Grassroots installations of it. uh Probably going back about 12 years, um we actually went through a an interesting internal migration to bring everybody together onto a common instance, because we were seeing the friction of different instances. uh People not able to collaborate together effectively right. So you know a few hundred users on one instance.

C

Another couple hundred users on another instance uh today, we've Consolidated that- and we are just shy of 5 000 developers on that instance- we're right at 49.70 when I checked last night, so I'm just waiting for that ticker to count over five thousand.

A

Awesome, that's awesome! How about you Joyce.

D

So we're actually at uh 7 700 developers-ish, uh climbing up to 8 000. We haven't completed our migration. Yet um in terms of how long we actually started our first batch of migrations, maybe about a year and a half ago in August, 2021 um and but the team has been really working on analyzing GitHub and like its offerings, you know putting together migration strategy since late 2019.. So it's been a couple years at this point.

A

Awesome awesome, Andrew.

B

Yeah, so a Ximena, we have uh right at 1700 users in our GitHub platform. Today um we also have a and we've been actually GitHub customers. Since around 2015., um we actually have a fairly sizable Azure devops instance, as well with around 12 000 users. So um you know we work to integrate both platforms together and take advantage of the the cool capabilities there.

A

Awesome awesome all right now this next question Joyce is uh we'll start with you. Can you comment on the process of adopting GitHub and migrating from other tools to GitHub.

D

Yes, um it's a fairly long journey like I said, but um I will try to summarize uh so prior to GitHub. uh We were actually on review board and Garrett, and our high level strategy was to keep um RCI CD workflows, as is you know, while moving just the code review and uh code collaboration workflows to GitHub, um so you know to start.

D

Obviously we did a requirements analysis just to make sure that GitHub satisfied all of linkedin's requirements, and you know our needs and then, after that, we actually worked very closely with uh the GitHub folks to ensure that we had a test instance provision and configured uh so that we could actually start the process of you know figuring out how GitHub apps should be designed and implemented, um and in parallel we started putting together.

D

You know mocks of how these apps would look like, so that we could actually just get this in front of a wide variety of users um across our different personas at LinkedIn, um just to get early feedback on these new workflows and eventually you know all of this came together and that's when we began migrating repos and teams onto uh arcithub instance and in terms of the migration strategy.

D

We chose to migrate, our team's internal repost first just so, we could play around with that and then, following that, we actually chose to migrate our Flagship teams, um and it was an interesting strategy because they were the teams that owned the largest and highest traffic repos, and this was intentional so that we could actually detect um you know potential scalability issues and really focus on.

D

You know a single use case at a time, so that really helped us uncover some early problems, and you know once we got past that we continued our migration in a couple different ways. um You know we're still continuing now, almost there at 100 um we're actually at 92 percent of the way there so yeah I. Guess that's our that's our journey. um If you want to know more, we actually have like an article that we wrote up uh with more details about the journey that I can send to you all.

A

I think we have that link. Yes, I had that link right here. Let me copy that and crop that into the chat there. You go all right, so would would you uh did your teams? How did your teams feel about that whole process of migrating? Did they did they welcome the opportunity to do adopt GitHub you know. Was it was it or was it perceived as a forced adoption? You know that carrot versus a stick kind of a thing.

D

Yeah, so actually the majority of our teams really welcome the opportunity to adopt GitHub. um There were a lot of teams that were quite impatient, I would say, um and those teams were the ones that signed up to be our early pilot users, um and the agreement was that if they were early pilot users, they would have to proactively give us feedback about um their experience.

D

um However, we obviously know there's a balance that we need to play. We have so many repos um and so many users at the company that there's no way you know everyone would migrate to GitHub um without uh you know, without us kind of forcing them, and so we did have a period of a few months where we opened up GitHub for self-serve migration.

D

But you know, as that was slowing down um we're currently in a phase of forcing uh we post to migrate over just so we can, you know, move away from a review review board and Gary. um We don't want to support um you know so. Many systems.

A

Very good, very good, all right now would you say and I'll start with Andrew on this next one. Would you say that your development teams are more effective and efficient now that they're using GitHub and if so, why.

B

I would say we we definitely see benefits from GitHub um when it comes to kind of the social coding aspects or intersourcing.

B

um You know, Humana has a what I would call fit for purpose strategy around our devops platforms, and um certainly you know when we look to um teams needing to collaborate in a highly efficient way around their source code. um Github is the easily the platform of choice there. um Given the you know, advancements uh and better interface, around pull requests and just the ability to collaborate around source code. It's uh We've definitely seen benefit there. Absolutely.

A

Awesome awesome Joyce. You have any thoughts on that. Would you say you're more effective and efficient. Now.

D

I I would definitely say so um so with GitHub uh we were able to shift left I'm sure many of you have heard this term to ensure that developers can find out about.

D

You know the actions they need to take all the errors that um they need to address as early as possible, and you know one simple example: publisher is, um you know folks wanting to know whether or not their code change got the right approvals and is now ready uh to be merged, um because we focus on usability here we felt like it was very important to surface this information in a PR.

D

um You know right away, and so this way users could know you know why their PR was blocked. uh You know what actions you know, who else they have to reach out to to get. um You know further reviews.

D

um We actually didn't have this previously, so users would, you know, attempt to merge um and then find out after the fact that their code change actually failed to be merged and go back and you know start over and uh you know have to hunt for the right reviews.

D

um The other quick feature I'll talk about is a draft PRS. um They were really helpful in encouraging um our developers to put up change early and often for collaboration rather than waiting for you.

A

D

Change to be perfect, and this is something that we encourage just constant iteration.

A

Awesome awesome all right, let me see here um actually Richard. Can you go ahead and drop that first poll in there? um This is talking about um you know which of these tools are integral parts of your development process. Things like planning and tracking cloud-based development environments required code review fully automated builds and deployments automated tests, package management and an AI based code suggestions.

A

um Now Andrew, are you using GitHub as your only tool set or and and how important is that first class integration with other tools to Humana and you.

B

A

A couple of things earlier.

B

Yeah I kind of hinted at uh ximena's, Azure devops instance as well, um so with with GitHub we've integrated several other popular devops tool.

B

um You know pieces of the devops tool chain out there like sonar Cube and artifactory, um even things like terraform Cloud, um and so you know I think when you think about the devops tool chain holistically.

B

um You know there there isn't yet one platform that can do it all, um and so you know we we've taken to integrating kind of special where there's specialized needs around like code quality, for example, with sonar Cube or or you know, infrastructures code with terraform, you've integrated those tools into our developer, workflows and within both GitHub and Azure devops, and also say you know. You know it's been been really neat um for me to see the maturation of the Integrations between GitHub and Azure devops.

B

You know, Ximena heavily uses Azure boards, for example, for our agile planning and tracking and um it the ability to integrate and tie like Azure boards work items to GitHub pull requests for example, or you know, build an uh Azure devops pipeline against the GitHub repo. Those types of Integrations has have helped us uh use, the I would say the social coding aspects and the intersourcing ability, with of GitHub with our existing.

B

You know, AK existing devops platform with Azure boards and and Azure pipelines, and so that's given us a lot of flexibility um in our tool chain to be able to provide an experience for engineers to kind of meet them where they are right.

B

So we definitely provide you know similar to Joyce uh kind of a golden path if you will to production, but we also are able to allow some flexibility for the engineers to work, how they want to work and be in the platforms that they want to be in, so that you know we don't have to have developers, context, switching or jumping from you know, one platform to another.

B

It's important for us to keep developers within their within that sweet spot right within the tooling that they are most passionate about and most effective in, and so that ability to integrate and for for devs to get that feedback. There we've you know within their, you know, within their Azure pipeline or within their GitHub action.

B

um It helps to We believe it helps productivity quite a bit by keeping them focused around their tooling.

A

Awesome, yes, thanks for bringing up the poll yeah, so yeah, we see uh required code review fully automated build yeah, pretty even split across most of these things, only a few using uh cloud-based development environments. At this point- and there was a question in Q a about that- uh throw this out there to uh any of you, uh how are you doing local development environments? Are you using Dev containers or code spaces? That's from oh I! Don't wanna.

A

My apologies, if I'm not saying that name correctly,.

A

C

Want to comment code yeah.

A

Completely and.

C

So I I can say: there's there's a there's a lot of interest in it. um You know, prior to the existence of products like code spaces, um you know, we've seen a huge push in the containers and and the that really that value proposition around containerized Dev environments. We see a lot of teams, sort of Home growing those Solutions um to make more reliable and repeatable processes as they develop and deliver software.

C

um We have teams that have uh built full Solutions predating uh code spaces to really standardize remote development environments. You know uh based within like ec2 instances, and uh you know, tmux type solutions to allow developers not just to collaborate inside GitHub but to literally pair program remotely virtually uh you know together within their Ides.

C

um It was the uh you know. That's been the evidence that there's desire for that kind of way of working and that kind of collaboration. So for us you know this is one. That's on our list um that we're very interested to see. uh You know how we can bring. You know a full Suite of products to teams because there's clearly a desire for it.

B

Dave Optum in on it too and add on.

C

B

Mark was saying um you know, we we've definitely um been looking at code spaces and the dev container concept as well, um and it's it's interesting. We we found that it uh has helped uh with developer onboarding quite a bit, um so you know we went from maybe a couple weeks to you know you have someone coding, maybe day one or day two right, and so it's helped tremendously, but we've also seen, at least in the Humana context, that it's forcing a different way to think about development.

B

So you know, Dev containers won't have access back to maybe you know endpoints in our data center, for example, or maybe even in our cloud, and so developers need to think about how they engineer software differently and use like patterns like mocking, for example, and some of these concepts are, you know somewhat new, at least in the Humana context. So you know I think that's a challenge that we're seeing, but definitely we've seen the benefit from Dev container, certainly yeah.

A

B

A

Right, let me get back over here.

A

So kind of kind of along that, with that we have a another poll here, Richard you want to go ahead and pop that one in there, how do your developers feel about their current tool set so just kind of get a sense for um yeah? We can kind of Pop. We can share that here and kind of see what the responses are.

A

um Oh actually, we want to see the next one but yeah how how do your developers feel about the current tool set and I'll kind of throw that out to excellent? Thank you, I'll kind of throw that out to the the team here. So you know how? How did your developers feel do they feel better now, with uh GitHub and and the Integrations that are out there with uh whether it's the planning and tracking tools? I know. Somebody also mentioned jira yeah.

A

If you go look in the GitHub Marketplace jira is one of the most installed uh app Integrations with uh GitHub as well, but uh yeah. If you want to comment on that shift, hopefully in uh going from frustrated to hopefully loving what they're doing, but the tools they're using.

C

My experience has been that you know no matter how satisfied they are they're, always looking for something more, um and so we have our developer communities uh pretty active in getting you know giving feedback, and um you know wanting to see the tighter Integrations one of the real benefits of having an organization who really stewards the environment with a strategy means we can do things like integrate, hold new tool Suites to every repo right, the apis. Let us you know, deploy a set of web hooks or whatever the need is for a new tool integration.

C

You know uh from zero to 50 000 repos in a in a in a single day, um and so all those sort of value-add tools, sometimes even ones, that they don't know that they uh need it's it it uh to get those instrumentations and those extra Integrations um for free by a team. You know who's providing that on their behalf within a within a large Enterprise.

C

um That gets some uh very satisfied, but it also brings them always wanting more um and there seems to be a sort of a never-ending backlog of new tools that that teams are discovering and looking at and wanting to see integrated into the into the suite, and so that ability to extend and integrate is, is one of the things we find really valuable, um whether it's with third-party software or whether it's with solutions for processes we create internally.

A

D

Good yeah I can I can comment quickly too, um so we actually have an initiative going on to unify our engineering experience, um and this is just to provide a way. A consistent way for developers to you know, create and manage um their new uh software, whether it's like in our own data, centers or uh or in Azure and as a part of this work.

D

One of our guiding principles is to really ensure that we're adopting industry, best standards and practices and just not adding too much on what we call LinkedIn sauce on top of some of these standards, which we have done in the past, um and this way um you know our end goal is to be able to decrease the learning curve for for new hires and people who have been in industry in for uh for a while.

A

I, like the way you mentioned the LinkedIn sauce there, uh you know I I came to GitHub by way of Microsoft and I know: Microsoft has their own one engineering system, which or uh which is their Center of Excellence for software development and yeah back in the day, there were internal tools like Source Depot and that, but we now standardize on uh Visual Studio code, Visual, Studio, Azure, devops, GitHub and uh and all of that and yeah there's there's a lot of things.

A

Layer well, I shouldn't say a lot there's several things: layered on top of GitHub and Azure devops for, for example, but those things also Lee sometimes lead to what ends up in the products themselves kind of a thing. uh So yeah we're very fortunate. You know for that. You know for for things that you know if you're building on top of GitHub, please go out there to our feedback uh discussions and say: hey it'd, be great.

A

If we could see this added to GitHub, you know and and things like that, because yeah we definitely do want to learn from customers like you and from our audience. You know. How can we take some of that LinkedIn sauce if appropriate, and you know build it into the systems for everybody's benefit foreign just a little bit here, um Richard, let's go ahead and put up that next poll. uh So the question is: how does intersource form part of your collaboration strategy and what business outcomes have you seen improve through increased collaboration?

A

Joyce, you want to start with that. One.

D

uh Sure um so, yeah I guess intersources an area that we're currently looking to improve at LinkedIn. um We feel like reuse of code is not you know easy for consumers or uh producers um and, as you know mentioned in my intro, we've been investing in a developer, UI uh built on top of Backstage and the goal is there is to improve discoverability as one of the main goals um and to be able to build a Marketplace so that you know producers and consumers can come come in, find their projects source code.

D

um You know undocs that are relevant to them without needing to have that tribal knowledge which um you know historically, has been a problem uh here at LinkedIn foreign.

B

I'll jump in on that question too. um You know, I, would say similar to Joyce intersourcing as a methodology that we're trying to improve on at Humana as well.

B

um We actually recently, actually, this year, we we began measuring our developer experience just internally through surveys um and one of the consistent pieces of feedback that we've heard um since we've started measuring is that developers are seeking. You know more um standard libraries, standard pipelines, easy to consume, documentation, those kinds of things, and so um you know we believe that intersourcing can play a big part in improving that and having the overall community help. You know help improve.

B

uh Even so, it's not just kind of one team's responsibility, but it's like all of ours responsibility right to improve those things. So um you know if we can be a conduit for that through intersourcing methodologies. That's that's the approach that we're going to attempt to take, um but, but definitely um you know similar to Joyce I- think it's something that Humana recognizes as something we need to improve on and that we are focused on the yeah in the next couple of years.

A

Oh, this is awesome I'm, seeing the poll results here, we're up to 42, saying they have uh both open source and intersource programs that that's fantastic to see because you know going back three five years ago. You know I I would expect that to be a a lot lower so great to see that kind of adoption.

C

Yeah yep yeah, are you sure, for you know, Cox Automotive, you know, intersource has been an interesting conversation.

C

um We found that there's a real balance between uh you know, hundreds thousands of developers, you know, depending on some critical library or or pipeline, that they can use and to make that open and visible. So people can understand how it works be able to contribute back to it, but also know at the end of the day that there's someone who owns and is responsible for that thing that they depend on right.

C

So having a pretty intentional strategy about how you know what things we promote, how we promote them and make sure there's a sort of strong business. Strategic ownership behind those assets is important. Usually when we talk about intersourcing, we think about libraries and code. One of the more interesting and important success stories for us has been the inner sourcing of decisions.

C

um We have taken a strategy with uh you know in our organization, our architecture organization, but even the broader organization. Around architectural decision records we actually document and develop those decisions in the open, uh through through markdown files and GitHub, with pull requests and very active discussions. uh Almost like a request for comment kind of process to allow everyone to not only understand the decisions that are that are made, but the conversations and the context that led to those decisions right and we opened those repos up for anyone to to make and propose decisions.

C

And so we can work through them collaboratively and that you know, may not be the the classic definition of intersource. But uh it's been one of the most interesting and valuable propositions for that sort of collaborative ownership through GitHub um that we've we've had a lot of success with.

A

Awesome awesome all right, one more one, more question here and then I think we'll probably uh shift over to uh some of the audience questions. um So there have been numerous Innovations on GitHub over the last few years. Things like uh GitHub, Advanced, security and code spaces, GitHub actions, GitHub co-pilot.

A

uh Can you talk about how adopting one or more of these Innovations have benefited your organization? uh Mark I'll start with you.

C

Sure uh actions has really been where it's at um for us for the last 13 months or so um we started a uh the for the first time ever believe it or not in Cox Automotive, uh an effort to bring together an enterprise-wide offering of uh continuous integration. Tooling. Historically, we We've Come Together as an engineering org over over a year, many years from different Acquisitions different mergers. Different groups coming together in the spaces of cicd, was a a always a locally owned concern.

C

um Actions gave us an option right. It gave us a real platform that we could um come together on. We were able to not provide just not just the actions features but actually provide uh an enterprise-wide Suite of Runners um that any team can access, uh and you know what we say is you know it used to be that teams would build Jenkins servers built-in City servers they spend. You know how many days and weeks setting them up how much ongoing time patching and maintaining them.

C

Now someone can bring a yaml file and be deploying to to an AWS environment and in under five minutes, uh and that's been an absolute game changer for us. uh The adoption rate of it has has been Wildfire, um and so that has been really where all of our focus and all our activity has been, and we've been not just turning on the platform, but we've been integrating it with all of the rest of our processes and workflows. uh You know from every end of the of the fplc.

A

Very good, very good, Andrew Joyce any comments.

B

I can just chime in and talk a little bit about um a git Ops methodology that Humana is choosing to to roll out on GitHub. So you know as we're moving into the cloud um and and looking at kubernetes workloads and shifting our application. You know runtime more so to kubernetes.

B

um You know. One thing that we found value in is is github's flexibility in terms of driving different ways of working, um and so you know where the past four years or so you know- we've been really driving towards I would say: push-based deployments.

B

um We found that using um you know, git Ops methodologies and uh continuous delivery operators to pull those changes in uh you know to the runtime has been uh very: it's been a shift in mindset for our engineering organizations, but it's been one that actually provides a more structured, workflow and actually kind of enables teams to iterate faster right.

B

When you you are operating more so just around your your repo and and so um definitely a shift for us um thinking wise, but we found that sometimes that's good right to kind of bust up the way, you're, you're, working and think differently. It helps to helps to the teams to kind of grow their maturity of their process. So that's one way that you know we've kind of built on github's flexibility to to drive that I would say innovation in the way that we work.

A

D

Yeah I guess I.

A

D

A little bit also um yeah I guess maybe my answer won't be as in-depth, because we're we're actually super early in some of these GitHub offerings. um So obviously we want to get onto GitHub first, which is our very first step, um but I think after that, it's going to open a lot of uh doors to these other uh GitHub offerings, and so there are other teams within my organization that are um you know, working towards migrating RCI workflows over to GitHub actions.

D

um You know this will allow us to align with industry Trends. You know, improve our open sourcing workflow and just simply reduce our operational costs, and then we also have a code experience team looking into um actually they're running a co-pilot early adopters program and we have about. We have a lot of interest there about 400 users onboarded.

D

um We have about 300 sites on GitHub pages and our security team is looking into um integrating Advanced security, and you know remote development team is looking into code spaces and so, like I said many of these are super early. um So we'll have to report back in uh on the outcome in some time, but I'm sure there'll be a lot of benefits to our developer, with just having a lot of these offerings. Centralized in one place, which is GitHub.

A

Awesome awesome, yeah I just want to emphasize what Richard popped into chat there yeah please go over and upvote the questions in the Q a panel and actually I'm going to pop over there now and what I see as the most voted on question from I hope. I'm saying that right, how are you structuring the repo is that mono, repo or individual repo at domain level? Can you give some heads up on mono repo's, pros and cons, Andrew I know you said you could provide some thoughts there.

B

Yeah I can jump in on that. So, and this is a very interesting debate, I know lots of people fall kind of on different sides of it, I would say for Humana.

B

Our strategy has been to draft towards more componentized repositories, so we've taken great pains and actually just a little bit of backstory kind of why we went down this route ximena's, obviously operating in the healthcare industry, um so we're highly regulated highly audited in terms of our sdlc processes, and so for us, it's very important to understand um what Source Code maps back to which component, in our you know, kind of application, portfolio management system and even within our change management database or cmdb.

B

So we have to really keep all those records really tightly coupled so that when Auditors come in they can, we can kind of, say Yep. This is what we release. This is why we released it. This was the funding source Etc, and so um we actually took to um creating what we call an asset yaml file and this concept lives at the source.

B

You know root of every source code repository at Ximena and it has identifying information that Maps back to our APM record um and that's been kind of a game changer for us because it allowed us to both drive a componentized structure within our repository.

B

So you know, teams can't release unless they're tying to their repo to one component, so it kind of forces that methodology of you know one component to one repo to one pipeline, um and so it's kind of that was a technological check that we put in place, but it also just helps us inventory our source code. You know now we can. You know whenever there's a some kind of security incident.

B

You know, hopefully that never happens, but if there is, we can really quickly say: okay, here's exactly where this horse code is at so having that inventory has been a game changer for us. So obviously, you guys can tell like I'm very passionate about kind of smaller repo size, not only from like a governance standpoint, but also just you know, smaller the code bases the easier it is to release and the more often you can release if you're, making small incremental changes.

B

So you know that's kind of the way that we looked at it for for how we implemented at Humana.

A

Awesome Joyce mark.

D

Yeah I can I can add a bit um so actually LinkedIn started off with um so this was 2007, so I wasn't even at LinkedIn, but I'm just I hear of these stories. um We started off with a monolithic code base uh back then, and you know it was. It was good for a few Engineers I think we barely hit a couple hundred. um You know back when this was like early startup um in early startup and you know, changes were deployed every two to three weeks. uh It was super manual.

D

There was like tedious testing, um even deployments, you know had site downtime required and just deployment in general, the Cycles were super slow, um and so actually there was this period of uh time called uh inversion. Where we decided to um you know, really invest in our tooling, uh because we decided that you know we can't. We could no longer handle all this pain anymore um and.

C

A

D

That point, if, if we decide or if we saw that trunk was not in a Deployable State at any time, Engineers had to you know, stop all forward development and everyone was like, you know all hands on deck, um trying to get the trunk fix um so that yeah that did not not work well, um So. Eventually, we switched over to uh this model. What uh that we currently go by, um we created this.

D

uh This term called multi-products that or this concept called multi-products um and the key is uh to be able to with multi-products to be able to um they're smaller repos, essentially, and um so we were no longer uh in kind of this mono repo uh in this monorepeople world, and the key is to use these multi-products to manage dependencies between each one, and so you know, let's say for uh multi-product a if that depends on you know some version of B, then all a has to do is include that version of B in its metadata, and so you know with um that's just a super quick summary about it.

D

um But you know some of the benefits that we found were um you know better dependency management. um It was uh just we built generic tooling. um On top of this uh to be able to help us manage multi-products um multi-products help with greater ownership um at LinkedIn. uh You know, teams could just own their own and not worry about. Oh, my, uh my chains broke someone else's and lots of finger. Pointing um happened, and some of the other benefits were just uh committed to deploy.

D

Time decreased and you know we know we no longer had these ordered deployments necessary.

A

Very good, very good I, don't know Mark did you have anything to add or should I move on.

C

Yeah, just just quickly, I'll I'll uh we've seen a pretty strong shift towards the smaller repos um there's a couple examples of mono repo patterns, but you know we're you know as Cox Auto we're across. You know we're presenting hundreds thousands of products, but you know uh probably six to seven thousand Deployable units, um and so we see uh I wanted to kind of just point back to what Andrew said.

C

It's funny, it's a very similar situation where we need to even understand the inventory of our landscape right um and so those individualized repos uh tying them up to what we call our workload catalog um and then being able to to then cross-reference that we're introducing yaml files to uh to make the associations back we're kind of walking backwards into the situation from years of development, um but then to be able to instrument that to say what technologies are in use in my catalog, what technologies are being deprecated um who's, making changes when, where is the most active development happening um as a large organization and I?

C

Imagine many of the people listening today are um the ability to just gather intelligence about what thousands of developers are doing it in itself is valuable and is a product that you can bring and deliver um metadata and relationships and organization of information and relating to your organizational structure, your company's org structure, which may not always match your GitHub org structure um that uh is really aided by the smaller, more discrete repos. It's easier to get a much better understanding of what's going on, and so for.

C

A lot of reasons, including just like Joyce, said, the reliability and frequency or release we're seeing a pretty strong shift uh towards the individualized repos. The only kind of counter guidance we ever give to that is is if you have extremely strong coupling, if it must be packaged together, then ship it together and keep it together and version it together, but often that's sort of a tends to be more of an anti-pattern than anything else.

A

Very good, very good. All right I, don't see too many votes, so I'm going to go back to well. Actually we do have an upvote for this one from uh Taurus. uh Maybe you can tell tell me how you do continuous deployments for databases.

A

Does anybody have any tips or insights there on managing database updates.

B

That's a tricky one I think maybe we're all quiet, because it's tricky um I would say I'll just share from a Humana perspective. um This is something that we're working on actually this year. um You know we obviously put more effort into like devops for for applications, but I think we all know. Databases are a critical component of most apps, um we're we're actually looking at a third-party product and again back to the integration question right, we're looking um at liquibase to solve this problem. For us we have other Solutions.

B

You know that we've tried um internally, and none of them seem to scale quite as well and handle kind of vast number of different Technologies. So, but you know this is perhaps something that is is you know not an easy problem to solve, and it's something that we're actively working on uh this year, just within my team, so yeah, that's a very transparent answer for you guys.

C

I was furiously Google, you know over here for the title of a very old book uh on this topic and I cannot find it. um So let me ask chat GPT real quick, um so uh you know for us the the there is no Silver Bullet answer, but but a couple key principles stand out from what Andrew just said right um that change that that way of of managing the structure of your database, it's got to be driven from code right. So what, regardless of tool set?

C

You choose whether it's like a base, whether it's um uh tools from companies like Redgate, um even some of the more interesting open source Frameworks out there um roundhouse, let's celebrate one um there is it's got to be driven from code um and it's got to be in that manner of uh you know forward forward iterative, non-breaking changes right um and so uh there's a lot of documentation technique about there about how to how to introduce columns how to iterate uh on database table schemas uh in ways that don't introduce breaking changes.

C

um If I find that book title I'll drop it in the chat, but uh that whole concept- it's been, you know it's probably 10 or 15 years old. Now at this point, and still we our own teams, sort of struggle to find that sort of working, Cadence and right combo of way of thinking and tool set to achieve it um but yeah that is, uh that is a not an easy question.

A

All right, I did drop in a link there having come from Microsoft. uh You know, there's a great article there, where the Azure devops team talks about how they took team Foundation server, which is a client, a classic client server application to migrating it to the cloud. So there were lots of Lessons Learned there and a great article specifically on versioning database updates and versioning apis, and things like that. So you can get the bits out there on the server and then flip.

A

The configuration to say now use this version uh in in things like that. So great great article there all right. Let me go back and see. What do we have now is uh uh upvoted question here. Well, I will go back to uh the most recent questions. I, don't see many votes there for intersource. How are you managing access versus security uh looking to prevent code theft? This is from Jay West.

D

C

I guess I'll give a.

D

Quick answer um so actually I would say at LinkedIn our repos are pretty open, like we have a pretty open culture of like hey. If you want to contribute to to my repo like it's open for you as long as we approve it, um your change at the end, um you know so I will say with, um but I will say that you know if there's something that um needs to be locked down.

D

um We have capabilities of being able to do that right now, um with with apples, um so I I feel like with with intersource. Maybe it's a bit too early. We haven't really thought about that, um but we do have other mechanisms um to be able to do that at the moment.

A

I'm, assuming you you leverage like protected branches right, nobody can. Nobody can uh commit directly to like your main branch and and things like that. So there's always some level of verification before code hits the main branch.

D

Yes, I guess: I guess something similar along along those lines um yeah and we we also have capabilities of like if we really have to like in code search right, lockdown um locked down to repo, um so that people can't see it.

B

D

Know we have permissions to be able to control all that, because I think one thing that um one thing that LinkedIn um really cares about is ensuring that only people who have like, if it's a super, sensitive repo, only people who have um the need to view it or you know, have access to it um like those are the only folks that should should have that power. So um we have pretty good granular controls, kind of scattered uh throughout our tooling I would say.

A

Very good, very good Mark, any thoughts.

C

Yeah, certainly, we have taken a strong preference to you know developing in the open within the organization. The the you know.

C

We encourage public uh repositories within our ghe instance um and that you know the benefits of that for the organization as a whole is far outweighed any of these concerns right, but the concerns are real, um and so it's actually been an active area and get investigation about not so much how to prevent uh the code theft concern, but even the the accidents right, the unintentionals um where you know code may be pushed to an inappropriate location, um and so those are uh something that you know.

C

We have a whole organizations of people who are quietly thinking about it in the background about technical techniques to identify and Trace the movement of of our intellectual property.

C

um But I'll be honest, it's a spot that um you know we'd love to see more from uh from from GitHub into in tooling and utilities to be able to um track and follow it, because it's not an easy problem.

A

Right right, well that that is an area where you know. Last couple of years, we've made advancements in security, whether it's bringing uh fourth Enterprise managed users, so you can tie directly into your identity provider. uh Providing you know, custom roles that you can create. You know to create finer grain, uh access controls and things like that, but yeah that that Journey has not ended. Yes, we're still looking at you know, providing more and whether it's providing more in audit logs or providing additional controls, and things like that so yeah that that Journey definitely continues.

A

All right coming over to the other questions, the next one that I see with the most up votes from Kevin. um How about automated code quality checks, uh any any check stats, any tips! What do you? What do you guys use.

A

Thoughts there.

B

Yeah I can jump in on that, um so Humana leverages the sonar Cube platform um for code quality from a and that's integrated um and actually enforced uh to and required to pass, um at least for new code um on all of our deployments via pipeline, um whether that's GitHub actions or Azure pipelines.

B

um As you guys can imagine, ximena's um got a lot of a lot of code bases. We have 25 000, plus repos in our ecosystem and there are varying states of maturity. Some have been around for 20 years in terms of source code, and so um you know we have to help the the Enterprise move forward and and care about things like unit testing and code smells, and um you know ensure that our code quality is is up to par with with releases going forward.

B

So you know we don't have the luxury of starting from scratch. Essentially, and so you know, we've taken to um integrating specifically on new code, entering kind of tighter policies, around code quality, so enforcing you know, good unit testing practices, ensuring there's clean, commits going in and that's a that's a Mansa and uh that we've been drop really been driving with her engineering community. Over the past couple years and I would say that we've definitely seen a positive trend there for sure, um and a lot of it I think is just developer awareness.

B

um You know. Sometimes um you know, devs are running the scans, but not really caring about the output of the scans, and so you know we found by putting a a hard gate in the process that says hey this. That scan must actually pass right, a certain quality gate thresholds. um It's really helped improve uh the overall source code. Health and also you know, bringing leader awareness in to a certain extent has helped um so yeah. We uh that's kind of a long-winded answer, but a short short way to say yeah.

B

We use sonar Cube for our code, quality measures, yeah.

A

Very good any other any other thoughts there. Okay get an eye on time here all right. Moving over to our other questions, then, um let's go with this question from Ken. uh How complex were your CI CD pipelines prior to actions and how did that impact your approach to migration.

D

I guess I can go um cool.

A

B

D

I guess our pipeline prior to action. Well we're still on we're still using those pipelines because we're very early interactions but um I would say they're very rigid and not flexible. um We actually allowed, for you know, actions like it's very customizable. It is truly a platform where people could come in and, like you know, add the stuff that they want um to their workflows.

D

um We actually didn't really allow that um so we claimed we had, or we have a workflow engine that that uh pretty much was um had the steps hard-coded in them so uh and and they were, it- was very uh deployment, uh see like deployment, Centric, I guess, um and so, if a new Step needed to be added, um it was a huge effort for for the team like we would. We.

B

D

Get user or like feature requests and the team would have to um you know, figure out how to get this stuff added into kind of the overall workflow. um So yeah, it was very limited. Customization I would say compared to actions.

C

The the complexity of pipelines for us has always been a um uh kind of a signal. um We've tried to keep pipeline deployment patterns um which have been, in most cases, locally development, locally developed by teams all across the organization um and sort of straightforward. As simple as possible.

C

um Repeat, uh you know, item potent style patterns where we can run the pipelines over and over again and get uh clear outcomes, um and so those types of pipelines, those you know, deployment pipelines- are have.

D

C

uh Very good and very straightforward: the transition over to actions where there's a lot of interest and investigation is the is the larger concepts of orchestration right after the code hits the servers, testing smoke testing services watching for key metrics uh triggering automatic rollbacks. um You know if a metric is violated in a certain threshold. Those types of patterns- those are that's where our you know biggest interest and uh most active conversation is around right now.

C

I, don't know that we've found the the really the The Sweet Spot yet on how to model those types of pipelines within actions.

A

All right, we've got just a couple: minutes left I'm going to go to the other question. It has a couple of votes here, that's from Melinda. uh What about prioritization? How are you leveraging automation? You know which tools? uh How is that working with GitHub coordination or not anyone played with GitHub projects yet so I'm assuming Melinda you're you're talking about prioritization of uh tasks, user stories, Etc.

C

I can say you know we're not we're not using GitHub projects for sort of our official product delivery.

C

um uh You know tracking and management or agile life cycle management, um but uh I do see a lot of uses, for you know, teams using GitHub projects in our Organization for all the other things that they need to track right. All of our internal processes. I mentioned those. uh You know the the process of of documenting architectural decisions uh through adrs in in repos there's a little bit of a process and life cycle for proposing and reviewing the sign and defining them.

C

We use GitHub projects as a way to keep track and Steward uh Steward, those through and I've, seen dozens of other examples of teams finding localized uses for projects, because they need a simple lightweight way to you, know: sort of track the progress of of activities that they have to do separate. You know from their Sprint work and uh often that is something to do.

C

uh You know with what they're doing in GitHub, and so that's been just a very natural thing for teams to to gravitate towards so lots of little less official use cases of it. All over our organization.

A

That's great to see that's great, to see Joyce Andrew.

B

D

No I'm good I, don't actually don't have much to say on this one. Okay,.

A

Well, real quick one last thing then, and then, and then we'll wrap it up here. So do you have any parting tips for our audience? You know thoughts about uh you know, GitHub, adoption.

B

As we both came off being at the same time, Joyce um I'll just say I think um don't limit yourself. I guess is what I would say. I think you can adopt uh devops practices with almost any technology stack um with a platform like GitHub and Azure devops. For that matter, so um you know I would say just keep your mind. Open, doesn't really matter what technology stack you're working in you can do the devops with with any of them so yeah. That's that's my mindset around it.

D

Yeah and I would say um you know once you're, if even if you're evaluating whether or not you should adopt it, um just try it out right, get a couple. um Devs, uh you know on your team. Have them give you feedback early feedback, we learned is always it's a good thing um can help you iterate or or even pivot, directions. Right, if you, um if uh you know you're, not getting great feedback about uh the thing that you're trying.

C

Yeah and I love this point about feedback. It's so critical um in any in any sizable organization, uh who's working on multiple products, or has you know uh any significant number of teams? There is no one-size-fits all you have to solution for all right, um give them the flexibility to give them the choice, don't constrain them in the ways that they want to work.

C

um The platform this platform GitHub, is, is well suited to supporting many ways of working uh and yet still letting a large Enterprise.

C

um You know, do what they need to do from uh requirements from a governance standpoint, um and so uh you know make make the you know remember that the developers they are your customers. They are your community, uh but so is the organization that you serve um and engineer accordingly right. It is an engineering problem to bring a tool set like this to just so many Developers, and so don't underestimate the value of your time spent uh really honing and crafting the offering of GitHub within your organization.

C

It will pay massive dividends to the business on the back side,.

A

Awesome awesome, I, wanna, I, wanna, I wanna. Thank our panelists today, here Joyce Andrew, Mark love to hear your insights and everything. Thank you. So much I also want to remind everybody. We will be sending out an email with uh links to all of the recorded sessions so watch for that next week and please take a moment to fill out our our survey. If you go to the uh docs tab at the right of your screen and in resources or in chat, we would love to get your feedback.

A

We designed this program for you and we want to make sure you're getting the most value out of your out of your time here, and we do use this feedback and our learnings to improve our future programs and continue to raise the bar for you. So please don't be shy. Tell us what you think and with that I want to thank everybody for being here today and we hope to see you at the next one thanks. Everybody.