GitHub Satellite 2016, 13 Sep 2015

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Social Coding at SAP - GitHub Satellite 2016

Description

Learn how we have introduced and scaled GitHub Enterprise at SAP and why it plays a crutial role in SAPs cloud transformation. We will showcase how different teams are using GitHub in their delivery pipeline and how SAP is adopting open source style workflows internally.

Thomas Jansen is a Product Owner at SAP focusing on improving the life of developers by providing tools and services for their daily work. Besides creating and running internal and open source apps and services, he helps establishing products like GitHub and Travis-CI at SAP.

About Satellite:
GitHub Satellite 2016 was a celebration of the latest and greatest in development. Watch the recordings to learn how engineers, founders, activists, and beyond tackle the latest and most difficult challenges in software.

For more, join us in San Francisco on September 13th-15th for GitHub Universe, three days filled with the creativity and curiosity of the largest software community in the world.

For more information, go to githubuniverse.com.

A

Hi, my name is Thomas, and today I would like to like to tell you how we have introduced github at sa p, how different teams are using github on a day to day basis and how open source style, workflows and practices are actually slowly more adopted internally.

A

So this will be basically the story of what our team has been doing for the last two and a half years roughly, and in addition to that, you will also learn that the term of github satellite, what was actually invented at sa p for describing satellite applications so applications integrating into the github API.

A

So going back in time, two and a half years, our team doing internal development was challenged with the task to improve the overall developer experience at sa p. When we first heard that that sounded a bit like we were supposed to save the world, um it also sounded like a good opportunity, but what made this especially challenging is that we have traditionally a very integrated stack where teams are actually developing their applications.

A

Don't know if you're aware of that, but it's integrated in a sense that you have one server where you would write your coding. You would have an integrated version, control system, you have an integrated deployment mechanism and so on and so forth, and why is that straight? Why? That system is strengths and weaknesses?

A

That is a bit of a challenge and but on the other hi aside, we also have like the other half of the company embracing more open technologies and stacks, so we're using a lot of Java for application development, but also Ruby's with nodes and stuff, like that, so that's becoming more and more prominent, like C++ for our database platform, so that is slowly catching up and we said: okay, let's go into that.

A

Let's see what we can do, so we basically started doing some interviews, some research and, despite from the number of increasing technologies which we need to support, we also observed that actually more and more responsibility was shifting to the individual teams. This is nothing new.

A

We just heard it in the talk before generally claimed with the Deaf orbs password, but this is actually happening, and teams are getting more and more empowered, basically also being able to choose their own technology and their own stack, and this is mainly driven by the fact that we are no longer just developing and creating software products, but shifting more towards operating, maintaining cloud services which is internally claimed as the cloud transformation.

A

So teams actually need the ability to control a larger part of the deployment pipeline and also be able to own that and the talk from brand if you've seen that, like with the cherubs integration, those things really come in handy. If you want to go for that.

A

So, given this trends and our challenge which we are facing, we came up with a tool strategy for our department that we want to enable continuous integration and continuous delivery on a technical level and on the same side, we want to also actually enable collaboration on a more cultural level. So, in a sense that teams can collaborate in the best possible way, not only inside of their teams but also across teams and across organizational boundaries.

A

So, given that it was pretty obvious that github seemed to be a very natural choice for that first, it's perfect for collaboration. It's the de facto standard for the open source community to host their projects. It's very well established also for people joining SI p. They already know what they're very familiar with the different workflows and things which you're experiencing.

A

So the idea was really to establish github as a strong foundation, so a strong platform on the level of source code management and collaboration, and then in order to be able to extend that platform with different tools, integrating into github with the idea that those tools, ideally are what we've always claimed them to be loosely coupled yet highly integrated.

A

So this is actually the time when the term get of satellite, at least from our perspective, was born, where we describe those applications sitting on top of github enterprise and integrating into the system as satellite applications. So what we actually did is we created our first satellite applications, mainly also because it up enterprise at that point in time was already used by one team internally, so the it was running you could already use it. It was not very established, so it was only like 200 people using it, but we are mainly a tools department.

A

So we wanted to build around that to make the experience greater. So we created PSC hub where you would go in to log in with your github enterprise account and PSC hub is actually a tool. Two men maintain product standard compliance, so whenever you want to ship a product that sa p, you have those compliance checklists where you say that you comply to a certain corporate requirement or not.

A

There is like mitigation and approval steps integrated, and what was nice about that is that this world was like a newer and nicer nicer version of an already existing tool, but we also wanted to integrate that into github in a sense that we wanted to connect developers primarily working on github and on the project with product managers or product product owners who are mainly focusing on that part and to bring that conversation closer together.

A

So what you can see right now here is that actually, all those different requirements are reflected one to one back into a github issues or a conversation we're all so developers can join in and actually be able to fix certain issues with with that regards. So that was like our first proof, point for building a first github satellite application and we also created another application, which is called bridge.

A

It's a very lightweight dashboard for employees to keep tabs on different in tone systems like backlog management, continuous integration system, but also things like ASAP news or famous Chuck Norris quotes and this application. Our was actually built with inner source in mind. So we wanted to really embrace the idea that others, people join, contribute to that project at least internally and then are being able to create custom applications for their local environments or creating new features and in order to bootstrap that we created a couple hackathons internally and our main locations.

A

So we organized one in Palo, Alto, one in Shanghai, one in our headquarters in Waldorf, and also want to Bangalore and are not right now, I think with over 70 contributors to the project and that application is used by actually one third of the complete organization internally, which is pretty cool, and one thing which I remember was a really nice story is that in one of the Texans, actually one team. So we split the complete group into several teams.

A

One team was able to build upon another team's work, even though it was a very competitive environment by just checking out there not yet merged pull requests and building stuff on top of that, and that really showed the power on an open conversation and being able to collaborate, even though they were competing actually. So this was all great, more and more people were joining github as a source code management tool as a platform we're using our satellite applications. So all was good, but there were also more and more discussions on going on.

A

Actually, a garret versus get up since SI p is also a core contributor of garrett. So we're a lot of discussions going on here and apart from those discussions, we also went once that bag and talk to the teams. So what we observed is that there were like two kinds of teams: the one we're very happily using garrett.

A

They really liked the pre-commit workflow that they were doing reviews on a commit by commit basis, so not much to do from our side, because that was already working, but we got also a lot of feedback from teams who actually wanted to switch to github, but we're lacking a more structured code review process. They were kind of accustomed to a bit like how Garrett is working, but they also feel that they need to have more control on how to do code reviews inside of Gita.

A

So what we created and came up with is an open source project review. Ninja review ninja is actually a tool to support that more structured culture of your process. It's deployed internally on our along our github enterprise installation, but it's also available externally for free to in order to be used for all open source projects and meanwhile also a lot of other organizations have approached us and deployed it along their or github enterprise installations as well, because they were actually facing similar challenges and in order to guide you through the workflow real quick.

A

Let's suppose we have this awesome library, which is callate calculating a couple of math operations, and now I would like to contribute another function to that awesome library. So what I'm going to do right now is that I'm going to commit to a new branch and open up a pull request for that specific feature, and then we will see as review injuries acting actually integrated into that repository as an integration.

A

We will get a comment on the pull request and a pending status, so we haven't performed any code review yet so a reviewer now can go into review. Ninja actually look at this specific change and then open up individual issues on different places of the pull request.

A

So in that case might complain about the formatting here that that should be adjusted to match the the other formatting and now, if he creates a comment on that in review ninja, the status is actually changing to a failing status, because we have one open issue which then gives the original contributor the possibility to also view that directly in the conversation on github and then be able to actually fix that.

A

So if we now go back to our feature branch and then into that specific file and then actually change it again to match the format where are just a second, so just copying it there and then changing it to an ad function, actually adding that and then creating a new commit on that same branch.

A

That will automatically update the pull request and if we go back to review ninja, you will also see that here actually now for that specific issue that that reviewer can see that he has created an issue at a certain point in time a new commit was coming in and now you can compare and see if that specific issue was fixed or not.

A

So in that sense you can track individual issues very nicely and then also give a ninja star, which is actually a sign that this pull request is now ready to be merged in so that's the basic workflow.

A

As of now, we have more than 30,000 ninja stars run all over the place, and actually, when we launch that one interesting thing which we then observed is we are getting also, of course, first contributions from the open source community in and we were challenged with yet another process and to handle clas. So I don't know if anybody is aware of clas, maybe if some signs no clas our contributor license agreements and what they basically do is, if you run an open source project, then, and somebody creates a pull request.

A

They basically safeguard the open source project so that they can be really legally okay with integrating that change. It's very established for larger organism, sized organizations and also open source foundations, and in our case it was no not very well integrated into the github flow. So, depending on how that is set up, you would need to sign a document and I, don't know, send it over by email or something which completely breaks the the idea of creating.

A

Maybe I will request for a typo in a readme page yeah, so you don't want to have that friction and the process.

A

So we created our second open source tool, which is called CLA assistant, which actually targets that problem. So as with review ninja, it's integrated into the pull request and basically sets the status on the head, commit of their pull request and gives you a notification if you have not signed the CLA edge so suppose, we now want to link a CLA which is technically stored in a just to our repository.

A

We go into CLA assistant, we say yep, let's do it, let's link it, it will create a web work for us and then it's all set up now, as we have already some open pull requests. We need to recheck our pull request because we only get notifications by web books usually, and then you can see that you now have actually a pending status for your licenses. You can now go to the details and then see the essay p individual contributor, License, Agreement, sign it.

A

It will take you back to the pull request and the status is now ok.

A

So this was also really interesting um because we got also a lot of feedback here, also got in touch with github itself on that problem and discuss it, a big back and back and forth and now I think we are up to three and a half thousand signed clas and also have a couple of self hosted installations, because some organizations choose to run their own instance externally, which is also fine.

A

Meanwhile, internally arm, we enabled the integration into our central build systems, so we created a I would call a middleware Russell, which is called see, I connect which allows you to choose a repository and then activate certain services which are then triggering a creation of Jenkins jobs on some central infrastructure.

A

So that has happened, and we also deployed Travis, CI Enterprise ASAP and have a very growing internal fan base. Ever since, and also the folks from Travis are here, and we are really trying to establish that as as in basically a choice for more def of saurian oriented teams to have more control on on their build environment and deployment pipeline.

A

It was not only us also. Other teams started to basically or embrace the github satellite approach and create their own applications. This is github insights and it's apparently on top of github issues and allows you to have a hierarchy arm.

A

So you can have something like here: epics user stories and backlog items in a hierarchy view and also a ranking of github issues, just as another view, and everything, of course, is still integrated. So you don't need to use it if you're a developer, but you can use it and and work for your sprint planning. For example, we were also able to just deploy externals services as well, of course, on. This is one example: get print which is used by arm one of our training teams in order to create training content.

A

So it takes a couple of markdown files and then generates PDF documents to be handed out to all training participants. And meanwhile we have really a lot of those satellite applications. So it's getting harder and harder to keep track on all of those. So we actually now created an internal site listing all satellite applications so that we can remember them.

A

So with all those tools available, the idea was really here to enable custom workflows for teams using github.

A

So we would have teams using github Jenkins and may be deployed to an essay p back end, and we also have teams using github, Trevor's and maybe Zen hub on top of it and review ninja for code review. So it's really up to the teams to decide what fits best in their local environment and in their technology and to give you some insights on teams actually using it.

A

For example, we have the to go team which is creating a right sharing application, so you can go to to go calm and then check it out, and they are actually were one of the first teams joining our github and for them it was really important to bring the different disciplines together so bringing designers developers and product managers together at one place where they can have a common conversation, not only about code but also about designs, new features or maybe your directions and what they want to go with the project.

A

So that's really exciting. They are also really very heavily pushing the DevOps topic internally at as ap doing daily deployments and really big fans.

A

Then we have medical research insights, they are based out of Potsdam mainly, and they are really into get into like automatic deployment and testing and having a very agile workflow and for them one of the key things was that they actually observed that this can be a good thing for I, mean team collaboration, so they're in this innovation center network internally, whether working across multiple sites on on new technology and that really helped them to communicate internally.

A

So they don't only use it for source code, but also for their complete portfolio planning, actually by using our new board for that and yeah by that, really really helping them to achieve that, then we have another example of the Conqueror platform, which is an acquisition of s AP, and they for them. It's really important that github X, basically as an authentication provider. So that's the same philosophy which I've just mentioned.

A

You have those tools integrating into that one platform where you have that one common user management and then all those tools you can easily replace it, but they all still integrate into that same platform and then add value to the overall solution which is really nice. So where do we stand today?

A

As of today, we have more than 9,000 users, so I guess we are one of the larger enterprise installations, although we don't know for sure we are running the latest version since last weekend and we are at a point where we need to investigate the castle set up. So we're still running on a single machine are, but we will change that very soon.

A

We also have an internal getting started page, which is called a github boot camp. So we would get like the first steps on creating a new user. Getting your repository, creating your first commit a couple of videos, interactive material arm and then also a link to workshops which we are organizing. So we have a trainer network of around 20 trainers internally.

A

So, whenever a team needs some more training on get up, github specifics, we can set that up and we also have meanwhile senior executive support to really bring in new tools into the organization on a technical side, but also to is there social coding as a kind of cultural paradigm.

A

So our lessons learned so first of all, for a large organization to introduce a new source code management system is really a change management process, and we doing that for other tools which we bring in the organization now as well so ideal, you start with a really small group and then you slowly expand over time and take all those learnings. So it's not a big bang.

A

Secondly, if you're really at a point where you want to change how people are working, you will have people complaining about that, and that can actually be a good sign, because then you know you're actually changing something. If nobody's complaining, then yeah. You know you also don't get that feedback and also you should keep on pushing.

A

If you know it's the right topic and last but not least just recently, we experienced that it's really important to use web books instead of pulling github, especially if ever very large installation, as we have as of now so with that I'm, basically through. um Maybe you are now curious to create your own satellite application. What you can do is you can check out my github account. There's a github satellite repository, it's a very lightweight way on getting started and writing your own github satellite, integrating into the github API.

A

It's not perfect, I try to make it running for today, but it's you know it has still some bugs, but maybe you want to contribute to that project and then we can have a very nice get upset light app for others to basically also contribute to that journey and integrating into that platform. Thank you very much.