GitHub CodeConf 2015, 25 May 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: The Guardian: Developing in the Open, Roberto Tyley - CodeConf 2015

Description

The Guardian, a newspaper founded almost 200 years ago, now has a Digital Development department with a handful of small teams, deploying dozens of times a day and developing in the open on GitHub.

Roberto Tyley, a developer at the Guardian, describes their interesting journey down this road. His talk will cover the development of membership.theguardian.com as an example, and talk about some of the nice GitHub integrations they’ve created to help their work, gu:who & Prout.

About CodeConf
CodeConf improves the software community by providing a forum for thought-provoking talks and forging social connections. The third installment of the CodeConf series took place in Nashville in 2015. Attendees came together to discuss open source, best practices, documentation, and community.

For more information on this year's CodeConf, go to:
https://codeconf.com/

A

And so now that he's all miked and coming on up here, I'd like to introduce him. So please give a big round of applause for a Roberto, Tai, Lee hi.

B

Yeah hi, so I am Roberto Tai, Lee and I come from London, England and I'm a lead developer at the Guardian news organization, where we break some really massive stories like last year. We we won the Pulitzer for revealing the extent of the NSA's mass surveillance, that's Edward, Snowden the NSA whistleblower being interviewed by one of our the journalists in Hong Kong, and so just to give a bit of context.

B

The garden is like 200 years old, he's like old person, old anyone recognize the old person, that's Charles Babbage and in 1821, when the garden was being founded. He was in charge of it's just like a mathematician who was checking this discrepancy in navigational tables, which told sailors where they were on the earth, the longitude by the Stars, and it was just then when he was realizing. There were lots of little discrepancies and errors in that he thought in themselves.

B

There must be a better way of doing this and eventually, like maybe like an automated mechanical machine, and that became the difference engine which became the computers that we all have in our pockets and I. Think that's really cool I garden is pretty much as old as the concept of computing, but it was actually founded in response to the Peterloo massacre, which was a horrific event.

B

There were 60,000 people all peacefully protesting for political reform in the north of England in a square, and the troops were sent in to dissipate them sabers drawn and they slaughtered lots of people.

B

Many lives were lost, lots of injuries and the Guardian was formed in response to that to have a voice to report on the abuses of power, and so that holding power to account is something that the garden has been doing for a long time, basically being a pain in the ass for many governments and what things about being accountable about political count is that you need to be accountable, and so you need to be open and held to account yourself, and so one of the ways in which we're open is that if you go to our site, if you have a look at a nice responsive site with which really hard on actually all the code behind that is open source.

B

So we actually open source a lot of stuff I checked on our github org. We have over 700 repositories and more than half of those are open source, and these are some of our most popular ones. There's the Guardian front-end, which is like the front end of the site- and this has you know, got a 92 thousand stars and you can see the entire source code for the Guardian calm and the next one down the scribe. It actually came out of our homegrown CMS system for the articles on the Guardian.

B

We use a rich text editor and scribe as a polyfill for the constant edibles editable spec, so it allows cross browser which Texas's portland's actually is really widely and hey guess what open source actually means?

B

You get contributions back from community and that doesn't happen for a lot of our open projects, not not for like all of the money chances, but we do get contributions and some projects, especially ones which, like involved JavaScript to get, do, do get a lot of contributions so I'll to talk about some apart from project I'm gonna talk about a couple of the tools that we've developed as well.

B

So yes, there's the obvious nice things about open source like you get kudos right. This is like people on Twitter say: oh I got garden, you figure this thing out and then I can read your source code and I. Think I can do it too. That's awesome, Thank, You guardian, so that's nice, but there are also pragmatic. Things like self-interested things like hiring like hiring is is hard right.

B

It's something that we're all trying to do all the time and we like not only a developer's interested, more interested in being able to show off the work they do. They love being open source, actually just even just doing the hiring process, if someone's like coming in and they're, not sure if they want to work with you or not, then you can show them. This is the sort of stuff we do this. Is you know these are the pull requests that someone else who's been on? Your team has been doing you know.

B

Maybe you have some ideas about them, but it's like a talking point for the interview and it's a we strike quite strongly motivating or you know, they're not interested in that sort of stuff. Then they don't have to waste your time, others by like an infant interview. So it's it's really good for that.

B

So if you've recently been to the Guardian site, I don't know how many people here have like even heard of the Guardian before I arrived on stage like some people have yeah, okay. Well, if you've been to the guy site with an ad blocker, you might have seen this little banner up here and it's like we notice. You've got an ad blocker switched on perhaps you'd like to support the Guardian in another way, and this feeds into what I've been working on for the past year, which is our membership site.

B

It's it's a new sort of a microsite that allows members to support our journalism, because there are lots of people out there who feel that our journalism Reid doesn't need to be preserved.

B

When you know what challenged by so many governments, trying to kick us in the ass so allows them to support through membership fees, but it also allows them to get closer to going through. Like live events stuff like that, this was quite a speculative site when we did this.

B

This is quite an original idea for a news organization to do this and we were dealing with a new part, a new part of the Guardian, and these were people who weren't used to the way that we've been working in the past rapid, abstract development being agile, and so they were use to waterfall style developments. They were used to long release times like not not listing anything until it's been like a tuba QA process, and so what you need, then, is to build trust, and you need to communicate.

B

Those people and being open is really good. For that you can, you can system what we did was be engaged with those people we went and sat with them. We start showing them the small incremental improvements we could make and we were able to share the code with them. Shouldn't pull requests, getting engaged, get them involved in making text changes because that's actually words of their business. Quite a lot of times.

B

They're quite enjoy doing that, and so we want to get the point where a business that was kind of dictating how we should work was persuaded that this was a better way of working and that we were able to use continuous deployment and to iterate more rapidly. So that's really good. So we were continuous to deploying from master to production, which leads onto one of the pots that we pretty quickly realized that we had to write. So this tool is called Prout and they're.

B

Only French speakers in the audience: no that's a relief because apparently proud in French means breaking wind. I didn't realize that, but so its actual name comes from. Is your pull request out? Is your pull request out and it just makes it much more visible as to how your continuous deployment pipeline is going, because, obviously your pipeline will have many stages right, it will have the CI server will have employment, it will have the actual servers, spinning up and becoming ready and maybe have a CD on top of that.

B

So sometimes that breaks- and there are a lot of things to check like pipelines explode and what you really want is a check that goes from the start of the pipeline, which is a pull request and just cares about the very ends bit, which is your life site serving the stuff you want so I mean this is a conceptual map of what your deployment pipeline looks like right and you start with a pull request, and then it goes into the magic mountain where the army of crazy zombie squirrels construct the deployable artifact by whatever may be used.

B

Jenkins I don't know, and then you deploy it by some of the means. We've riff-raff at the Guardian, another open source project and then that gets deployed to the live service and eventually, like a person, can hit a URL and they get the new artifact.

B

So the thing about practice, it doesn't really care about what happens in Magic Mountain. It doesn't care about the route to your pipeline takes. All he cares about is the thorah quest and the life site as long as the live site bakes in it's a git commit ID that was created with then crowd can read like a tiny bit of configuration, Jason which says ok.

B

This is where my site is and if the pull request is not in 14 minutes counselor over dude, and then it will close down your entire git repository and it will figure out so on. The left is like your get positive history and then, if it's looking, what prep we'll do is we'll grab the commit ID from your life site and I'll, say: ok, so that little world symbol, that's that's the commit that I can see on the life site.

B

So that means all the green things they have been actually deployed, but the red thing for some reason that that that pull request there has not been deployed to production, even though it's merged onto master. So that is a bad situation. The Python is broken somewhere and you need to get them to fixing that, so it will label the pending pull request.

B

It will label the deployed purpose, and if progress is overdue, then it will also label that red you can it's very easy from a visibility point of view again communicating with the wider business, it's communicating what is deployed and how how half-hour features have made it. So that's one of the tools we thought the God and we because we're coming so dependent on the get up infrastructure. We realized that.

B

Actually, that's um it's really great thing, but we had so many users in our organization that we didn't know who the hell they were so those like, we had two reducers. They had really cryptic usernames and no names associate with them, and so we were like why.

B

Why are these people in our organization? We don't really know because the audit log doesn't go back that far and why aren't they using 2fa ain't, two-factor authentication? If someone can merge a pull request with a github account, they can change.

B

What's deploy to our site, so I get uh breakouts are pretty crucial, so we need them to be properly secured, so goo, who is a little part which runs off a tiny lot repository that you stick in into your organization and it's just a you, a user stop text file and it says these will be users through my organization and anyone who, failing in some regard some secure regard like they're, not identifiable, they don't have the full name or they don't have to fax worth turned on.

B

It will highlight those and you can look at the issues again. Everybody abuses, github issues right. This is how we're abusing it and you can filter down to say. Oh, who are the people who haven't, got two factor authentication tonight we have a hundred percent two-factor authentication use at the Guardian, because if you don't turn it on, we just automatically boot you off the organization again, so we feel fairly confident. At least the accounts are secure, so yeah insecurity in general.

B

If you are open, if your repositories are public, that encourages a much better mindset around security, you're, not you're, not tempted to try obscurity, you're, not tempted, put passwords or the credentials into a repository, and you secure your endpoints.

B

So you might be thinking like if I can't necessarily open a git repository. There's got stuff in the gets history so because get maintains all history. How do how do I get rid of those passwords, even if I delete them from my Lex commit? What I'd do that? So the is a talk will get fall to branch which will kind of rewrite this for you, but we've actually at the garden, we were another tool called The BFG, which is actually much more friendly experience.

B

It's much faster as well like a thousand times faster, even writes your history. It strips out passwords credentials, big files and it's very easy to use really fast. It's that quite a useful tool when you're open sourcing a project so.

B

Yeah, in summary, being open is good for hiring communication, cost security, quality trust immediacy and improving the total sum of human knowledge, which is a fantastic thing, so I say: do it and develop a neo.

B