Cloud Native Computing Foundation Cloud Native DevX Day North America 2021, 30 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Continuous Documentation Manifesto - Omer Rosenbaum, Swimm

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Continuous Documentation Manifesto - Omer Rosenbaum, Swimm

We have continuous integration/delivery but why not continuous documentation? This would particularly help onboard new members to code bases. The current status quo for sharing engineering knowledge within teams is broken. Code documentation requires new methods and tools to improve team efficiency and productivity. Continuous Documentation is a new paradigm for creating and maintaining code documentation that involves incorporating it into the normal development workflow. The principles of Continuous Documentation require that documentation be always up-to-date; be created at the most opportune time; and be tightly coupled with the code. By practicing Continuous Documentation we can get high-quality documentation that actually works for developers, guiding them through the code base, while speeding up development and improving agility.

A

Hello, everyone. Thank you for joining this session. My name is omer, and I'm really happy to be talking to you today about a topic that is close to my heart, that I've been researching and working on for the past years called continuous documentation, but before we jump into continuous documentation. Let me take you just a few years. Back a few years ago, I was leading an international training program focused on cyber security and data science. We were extremely privileged to have amazing students in this program.

A

They were all brilliant people, highly qualified graduates of top universities from around the globe and came with relevant job experience. It was a real pleasure to teach them and, after the program finished, I would keep in touch with those people. I would talk to them and hear their experience as they started their new jobs job that they would eventually become leaders in while talking to them.

A

I realize they keep hearing the same stories over and over again, I hear them explaining how hard it is to get onboarded to a new code base or new code bases. It's so many lines of code. They said, and all that logic accumulating over time by people that some of them are still there on the team and some are no longer there.

A

There is complex logic to get grasp on with lots of decisions that have been made along the way it took time and effort from them and also from people who were with them on the team and got to know the code base better, as they explained the code to the new hires, even for those brilliant skilled and experienced people. The experience of being onboarded to a new code base is hard. It takes time and effort, and I kept hearing similar stories from other people around the industry in many different companies.

A

So getting onboarded to new code is hard, especially when you start a new job, but does it end there? Is it the only time when you need to understand new code well as developers? We know that not at all understanding code happens all the time it happens to us on a daily basis. We always get tasks that involve looking at code that someone else has written. It could be in a code part that is new to us.

A

It could be, in a code part that we already contributed to but has changed since the last time we touched it. We don't necessarily understand the motivation for the code, that's there or all the details that were involved by when implementing it. Sometimes we need to understand code right now. For example, there is a bug it's affecting production, it's affecting our clients. We need to find a solution right now and for that we're trying to look around we're reading the code, we're asking people.

A

Maybe someone knows this part of the code and sometimes in those magical times there is something there that helps us and that's a document and even a great document.

A

This document is there and it can help us in many ways it can describe the big picture, for example, telling us why this code is there and why it was constructed the way it is. It can also provide us with some practical, specific implementation details that will help us a lot while we implement the solution, but, as we well know, this reality doesn't always happen. This document rarely exists, a document that is relevant, rarely exists.

A

If it does exist, we won't necessarily find it, and sometimes we find a document, that's supposed to help us just to find out that it's outdated, that the code has evolved and changed so much since this document was written that it's just no longer helpful.

A

I would like to argue that developers need and deserve good documentation when such documentation exists. It helps in so many ways. It helps developers on board to new code bases, for example, when they start a new job or switch between teams. It helps teams be more flexible and inclusive, as different team members can quickly get into new code parts and contribute to them. It can also help you on those times when you need to find a solution right now, for example, if there is a bug affecting production, so it sounds easy right.

A

If we agree the developers and developer teams need and deserve good documentation sounds like. There is an easy solution. We can do, let's all document and all will be better well in theory. Maybe, but practically this is not the case. It sounds like a fantasy world and I would like to think why I mean, after all, documentation can help in so many ways. So why don't? We have good documentation and when asking why, I think, as developers we've all been there, we've all started new jobs.

A

We've all been on the side, that's supposed to create documentation, and we don't always follow that. So why does that happen?

A

I believe the problem is that current practices and tools for creating and maintaining documentation simply don't serve developers as a result. Documentation is lacking or often outdated, and therefore developers may not rely on the documentation at all and not bother to create it and actually with the current tools it practices, it kind of makes sense. I mean creating documentation, takes a lot of time and effort, it's hard to know where to start, and it just takes a lot of your time and then let's say you took that time and you wrote the documentation.

A

Making sure that it is up to date is nearly impossible and you may stop and ask yourself: okay, I'm writing this document now, but it's going to become obsolete in a few hours weeks month. Why even bother and spend all this time writing something that is going to become obsolete so fast, so we agree, developers need good documentation, developers deserve good documentation, but the current tools and practices are broken and don't serve developers.

A

So I'd like to argue that code documentation requires new methods and tools, and I describe those methods in an article. I wrote about continuous documentation that was published in info queue and got the name. The continuous documentation, manifesto and I'd like to share with you its basic principles right now.

A

Continuous documentation calls for creating and maintaining documentation within the development workflow, in other words, treating your documentation as a crucial part of your development workflow like tests or the code itself.

A

It consists of three basic principles, so, according to continuous documentation, the documentation should be code coupled always up to date and created when best. Let me just explain each one of these code. Couple documentation means documentation that explicitly references parts of the code base.

A

Let's examine a simplified tutorial of how to add a command to git's code base, so here, for example, we explain how to add a command to git. In this example, we show the command add that is gited, so the document explains that every command has a file with a corresponding name within the built-in folder. In this example, it's add.c this file includes the definition of the command in this case cmd for command.

A

Add, as we can see here within the code of add.c, it goes on to explain that the function must be declared within built-in dot age, and here we can see its declaration within built-in.age.

A

In addition to make git aware of the add command, it needs to be registered by adding a cmd struct to the comments array, as we can see right here within git dot c. What makes this document code coupled? Well, it includes text just like any other document, but it also includes explicit references to the code. For example, here we can see a snippet actual lines of code from the codebase.

A

It can also include a path, for example, here it's a folder. It can also be a file name. It can also include names, for example. Here we can see a function's name or it could be a variable's name all in all. It's references of parts of the codebase that are embedded within the text and within the document as a whole code coupled documentation is great for many reasons.

A

It provides a standard to what documentation should look like what practical documentation for developers should look like and for me, as a developer, I'm really happy. When I see this kind of document or tutorial. When I see it, I know it's going to be practical for me. It is going to help me understand, what's happening in the code, why it was implemented that way and what the actual state of the code is.

A

It is also much easier to create documentation that is code coupled as someone trying to create documentation. That is practical. It's easier to start with the code base or specific part of the code base and then explain what those parts do it's much easier than starting with say a blank page, trying to think about what's right and then filling it up with words.

A

Code coupled documentation also enables many other things for the, for example, it's easier to find documentation this way and also it enables making sure the documents are always up to date, which leads me to the second principle of continuous documentation.

A

The documentation should always be up to date. That means that we need to make sure that the current state of the documentation matches the current state of the codebase as developers. We know that code changes it changes frequently. It changes a lot in gradual steps, so sometimes there could be minor changes and maybe a file name that you referenced in one document could be changed, so the file has been renamed, or perhaps there was a small refactoring to some part of a function, and sometimes the whole picture is different. The entire logic has been changed.

A

People added something removed, something made a very big change to the actual implementation to make sure the documentation is always up to date. We need to do it continuously, for example, on every pull request. In this way, developers can know that the documentation they read is indeed up to date, and then documentation is something you can trust. You can read a document and know that it is true, it is accurate and it explains and describes the state of the code as it is right now to make sure the documentation is always up to date.

A

What we need to do is to check the documentation and the code and if the code changed we're going to need to update the documentation accordingly to reflect the code's new state without code coupled documentation, this would be impractical and almost impossible. It would mean you have to read all of your documents, which consists of basic text, understand, what's happening there and then look at the code and look for inconsistencies or changes thanks to code coupled documentation, this task becomes easier.

A

It means that what we need to do for every document is look at the code, part that it references and see if they changed. Of course, tools that do this automatically can ease up the process a lot.

A

We should do this continuously, just like continuous testing when we know that the test pass, for example, after every commit or before merging a pull request. Just the same, we can do with documentation make sure the documentation is up to date before every pull request is merged, for example, by embedding relevant tools into your continuous integration system that way or another by making sure the documentation is always up to date, we make sure documentation is relevant and the developers can trust it.

A

The third principle of continuous documentation calls for creating the documentation when best and by this women, not when the ship has sailed, but as part of the development workflow, a classic approach to creating documentation would include focusing the creation on concentrated sprints.

A

This usually happens when you understand that there are a bunch of new people coming to the team and there is going to be so much they need to understand and isn't currently covered by the documentation that you want to write a lot of documents, or you just find out that currently current team members don't understand so many things and the documentation is lacking or when you find out that a lot of it is already outdated and you need to read through it, update it and create new documentation on the way.

A

The downside to this approach is that it makes creating documentation really hard. Let's say you want to document some part of the code that you implemented, but you implemented it a long while ago, let's say a few months ago now you want to document this part, but all the relevant information isn't fresh on your mind, you're, not sure why you did every little thing you did there you're, not sure what implementation details are still crucial and to explain and what implementation details aren't so crucial to explain.

A

Maybe the code is even changed by other people, and now you need to understand what they added to the code in addition to what you did originally, but that's even the easy case. On some cases we need to document some parts of the code, but the most relevant person for writing.

A

That documentation is no longer a part of the team that person has left or is currently unavailable, and so we need to find someone who doesn't know the code well to understand it completely from scratch and then document it, which would never be the same as the person who really understood the code. While writing it in the first place, the alternative would be creating documentation continuously as part of the development workflow, for example, just after finishing implementing a very important feature or after fixing a crucial bug.

A

If you try to create documentation just when the information is fresh. On your mind, the process is much easier. You remember the motivation to why you implemented the feature to all the decisions you took while deciding how to implement it and to all the small specific details that you need to describe.

A

Creating documentation continuously makes the process so much easier. To do that, you could follow different approaches. You could decide that big pull requests or important pull requests include the documentation within them. You can decide to dedicate some time after every sprint for creating documentation.

A

You can use tools that will help you and suggest where new code has been added that you might need to document it. Regardless of the approach you choose, creating documentation continuously rather than on concentrated sprints will make it much easier and by following this approach and created documentation in gradual steps, you will slowly increase the coverage of documentation of your code.

A

To recap, we agree. Developers deserve and need good high quality documentation. Now we can be more specific developers and developers, teams need and deserve practical code coupled and always up-to-date documentation that they can trust. Unfortunately, current tools and practices are too hard and don't serve developers. So this is rarely a reality code. Documentation requires new methods and tools.

A

We described continuous documentation which calls for creating and maintaining documentation in a way that incorporates it into the development workflow. In other words, we treat our documentation as we treat other crucial parts of the workflow, for example, tests or the code itself.

A

According to continuous documentation, the documentation should be code coupled always up to date and created when best. I believe the way we share knowledge and code has to change. If I go back to my students from the story, I shared with you in the beginning of this talk, I would like to hear different stories from them and from other people joining their new jobs and starting to onboard your new code base.

A

I want to hear that it's an interesting and fun experience to look around so many lines of code that people have written over time, but with the documents that help you that guide you through the code that explain the big picture, as well as the small details they need to know in that case, even if the relevant people who created that code are no longer part of the team, it shouldn't be so intimidating when such documentation exists. It helps in so many ways it eases up.

A

Onboarding, it helps teams become more flexible, as you can shift between tasks in various parts of the code base. It also helps you, when you have this bug right now affecting production.

A

I believe we can start implementing the principles I shared here with you on small steps. We can start in our teams and gradually improve them over time.

A

I hope that the principles and the concepts I shared here today would encourage you to rethink the way you share knowledge and code in your teams and the way we as developers in general, share knowledge on code.

A

I would love to hear your thoughts about it. I'm curious: what do you think about continuous documentation or documentation in general, so please feel free to reach out to me after this session. Thank you so much for joining and happy documenting.