Jenkins Google Summer of Code Office Hours, 11 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Jenkins Cloud Native SIG: External Fingerprint Storage demo

Description

Presentation of the External Fingerprint storage by Sumit Sarin, a GSoC 2020 student in the Jenkins organization. Sumit will itroduce the project and demo the current prototype of the external fingerprint storage, powered by Redis. Then we will discuss the architecture and potential use-cases for the Jenkins users.

References:

* Slides: https://docs.google.com/presentation/d/1hrdOYKXBWyQG-k53KUe6zFXJ3KWEK4t7smdtpNtp1Wo/edit?usp=sharing
* Project page: https://www.jenkins.io/projects/gsoc/2020/projects/external-fingerprint-storage/
* Plugin prototype: https://github.com/jenkinsci/redis-fingerprint-storage-plugin
* Jenkins Core patch: https://github.com/jenkinsci/jenkins/pull/4731
* Jenkins Enhancement Proposal draft: https://github.com/jenkinsci/jep/pull/289

A

Okay, we are recorded now hello, welcome to the quad- special interest group meeting today they have a presentation by a meet so material present external vendors in storage. This is a project is what known as part of google Summer of Code project will have a demo, and after that we will spend some time on design review and discuss it. I guess that's the plan.

A

So the floor is yours:.

A

Yeah before we begin, how do you prefer to answer the questions during the presentation? Send them all right away or after bit.

B

I'm playing the way I own, my slides I have a Q&A slide also at the end, so.

A

Since we don't have so many people, the call was just on September 5.

B

C

B

Hi everybody so welcome to cloud net. It's a meeting and today's demo for the external fingerprint storage project, which is one of the g-shock projects for this year, so at the beginning. So for the agenda for today, I'll start with the first person introduction and what exactly are fingerprints and what is the current architecture? What is our vision for the project and followed by a demo and Q&A? So so so Who am I. I am basically the zeesoc training student for this project.

B

I am currently pursuing a bachelor's in instrumentation and control engineering from MIT Delhi I started contributing to Jenkins in December. 2019 I know most of my contribution that we made some water on fingerprints which got me interested in pursuing this project or ji-suk and I'm really exciting excited with it.

B

I think we have both the power we had a community bonding for one month, maybe less than that, and we have spent about I think three four weeks, almost four weeks into coding, so so yeah so I representing that work for today, also so just for so for anybody who is new to fingerprints. I start with just a small introduction to fingerprinting, so basically, fingerprint friend, fingerprinting is in Jenkins is a way to track files, artifacts or anything across the CIC ad flow.

B

So what happens basically is that in different jobs, say you want to track say which artifact was used in, which particular build by say some other job. So you need to know that which version they use right. So fingerprints help us identify these files and they help basically versioning of these files and they can be a huge benefit to different plugins and they can also be used in tracking down where the source of issue came. So maybe a developer say fixed something in a particular build and another team is on another world.

B

So it helps that in jenkins layout. Basically, we have a this is how fingerprints UI is built. So say you have a job called demo and so say you go to a particular build. You can click on, see, fingerprints beside it and it will show you all the fingerprints that particular build recorded and you can click on any of those fingerprints and you'll get a view.

B

It shows that shows here it says number one to number three, so it's showing that it was used in prints, one two and three of the job demo, so that is how finger parenting is present, is built in jenkins.

B

So so what is the current fingerprint origin? So the main crux around this project is that currently fingerprints are being stored or you know, look in a local manner in an XML files inside Jenkins Ville, and that has issues all the issues with using a physical storage disk. So basically as fingerprints crew, they are considering that the storage of Jenkins master and their increasing reliance on the physical storage. So we cannot, you know, configure its out.

B

Storage is it which are also very which which offer benefits like pay pay, as you use to resolve in even most of times are cheaper. We can even configure live assets for say external databases, it helps in reliability, availability and, lastly, is that currently we cannot track fingerprints across think in the census. So that is another aspect of this project which may be then come in a later phase down. The line is that interacting the prints across the entire across different Jenkins instances. That is also something you want to solve in this project.

B

So so what is the vision for our project is basically in this diagram is that we can configure so basically, we weren't applicable architecture for fingerprint storage that any storage, any storage plug-in can come in implemented for that particular storage and then store those independents in that storage so say: maybe we have a MongoDB storage plug-in or a my sequel or the radius base. Any plugin can come in and we can use that to store our fingerprints and then maybe down the line.

B

We build another plugin, which the fingerprint next also is a PR plugin, so that we can, you know, track all these all this flow and may be represented in our in a word manner. So so that is, something will do as we go so so this is I. So this is the current model and how we are going to change it right. So now, I will go into demo or into the demo.

B

B

So if anybody wants to like so, we currently have a reference implementation for our plugin, which is backed by radius, so it lives in this repository, which is registering up in storage, plugins, I, think in CI we have a guide also for running it locally, for which I just walked through this of a city and yeah, and the API that is currently the pluggable, so VI that we are building on Jenkins cold side is actually it's an incremental build. So it's like it's still not merged, but it exists as a full reversed.

B

So yeah, it's I, think four, seven, three one and Jenkins cool right. So once you have this much, maybe so so it's still a beta ipm.

B

B

I'll flown this repository.

B

So we have yes so.

B

If a few seconds startup you.

A

It takes some time to start up streets.

C

C

C

Take a moment: okay,.

A

Now maybe you could just refreshed the page, because sometimes in development mode, we have a fish opening listening production.

B

So it's like, so we have the Kin sword, will go Jenkins and configure system. So we have a configuration page for our plugin over here. So it's not bid. So you can configure host port as a setter database. So radius has integer index databases and the V contains timeouts. We can a great angel plugins, so so say I don't like, and we can test at this. So I do it's failing before that: I, don't have insulin, setup, so I'll just part of instance locally.

B

It's on this 69! So now I can test it successful, so I apply and it's a also I start a command line, interface and.

B

Select the zero database and see what we have so as we can see, it's currently empty right. So what we can do now is start a new job or create a new job. Let's call it mo.

C

B

That a post-reduction to report these fingerprints, so so demo dot txt of the pilots. Will you get think of intern, not.

C

B

B

Now I have this job ready, so I can start a build for this job and I get a success rate so I look into this. Are you go to see? Fingerprints and I can see that demo, dot txt is being recorded and it was used in very fun. This particular job right so now in my interval, CLI.

B

Okay, so I know what happened. I think I forgot to in it will this particular plugin just a second.

B

So what happened is that in configuration half of all to enable this, we probably will be changing by enabling portion to sink in school. But so yeah I had to take this box and hit apply and I.

C

B

So now, like I, don't want to so yeah, so some clashes may happen so.

A

Yes, all right now you still have a memory representation on the finger prints right all.

B

Right so yeah I can.

B

So yeah so now I can go here so now, as we can see that I have this fingerprint demo dot txt right, which was very locally so so now. This should not happen in this particular way. So just.

B

So now, in this new job that I have configured.

B

Success yes and you can see finger prints for them or at the extreme, build one right. So now, let's see what happens when I have still alive. Yes, so now I am getting a fingerprint. You can check what storing. So it has the example blog for the thing of it. I am just out of curiosity. Let's see whether did it rip out the print or not so.

B

So so this is the same old thing of the day: I don't have any news, for this is for a demo flower necklace it. So no fingerprint was recorded for this new job that I created and basically we can also configure like doc, a fingerprint of say, build and plug which plugins were filled and publish plugin I know. So if it records, say fingerprint facets, so they are also handled. So we also get saved in the various instance, so I think that's about it. For the demo.

B

A

Just answer my question anyway: I want to toast about next steps. Okay,.

B

So next steps for us is so as of now we have not implemented any any savable. So basically, there is a component boil disable listener inside thinkin's and it can file some events on some particular. It can find some particular events at some particular times. So basically that does not get fired right now, because the API is such that we need to pass a XML file to it, so maybe be impossible to file awesome, or maybe we might change the API altogether.

B

So that is one of the next steps is extending the API and looking at so we can. Basically, what we want to do is look at plugins and see how we can help other plugins by 420k. You can introduce to help them down the line fingerprint cleanup. Basically it it's happening file storage right now, so what happens is old builds, get started. Definitely discarded routinely that tightly known facility right now is not implemented.

B

The new one will be lacking migrations, so so say when a person configures an external storage, so the fingerprints that are present locally need to be migrated to the external storage or say if he goes back home external storage to an internal storage. So these type of migrations or he may configure a new plugin. So these migrations are something that we want to tackle. Racing is something that I discussed earlier across instances for Jenkins and URM, also at the moment.

B

So basically, we need to fingerprint plugins to decide how they want to save these fingerprints, but reduce like right now is saving it examined blogs. So maybe you want to think tied to something which is more variable in the database, so these are some of the things we might be working on in the next few months.

B

Fine, so I have some links at the end of the slides but yeah before that, it's Q&A. So basically, if anybody has any questions, and/or, any suggestions would be really nice for a legend.

A

Thanks for the presentation, are there any questions or comments from others.

A

A

So here maybe you could play screen, share the current job, because it's one of the important topics, so we would like to really lent gentes enhancement to a proposal for this project. Similarly, to other pluggable storage, apps and Robert I guess it still needs some feedback from potential stakeholders so that we can.

A

Show that the dot represents all concerns so just to clarify for those who are not familiar with that the job process, they will be after draft stage. So basically it's a formally consistent, a job for discussion, and this is the plan for the next phase. But if you see any major issues or topics the bed rest, it's a good time to discuss that. So it's added to the draft and the reference for the refugee discussion.

A

A

Our current plan is to ship an alpha version within the first coding phase. So basically, it's say by the end of the month. In addition to job is a pull request to the Jenkins quarter. Basically, this pull request.

A

It becomes the major topic for adulteries because we still meter they pair it not listed in public, but we use all summer features like better annotations, etc from those who own the code would be great to get him back whether you see anything much of day, which needs to be discussed before we place it with the nature because a function of RS, it works pretty well right now, and you don't also pass a so test a got. Some additional test coverage as a part of the project.

A

So personally is that we've ever I'm tempted to say that it's ready to be merged.

A

Others I think especially JC and son, because they have been working on other pluggable storage implementations before.

A

A

Do we have any open comments on that or request at the moment, I.

B

Don't I need this one for that I configured I discussed this ready right. Oh that and enable button that, like I, showed you need to that cold side. So I, probably just either UI for the configuring die from a drop-down means, I think it's cold. That is.

B

A

Much brings up one problem about extensibility came because the injectors will have two extensibility models. One is let's say: factory is an extension point, so you have an object which can produce, let's say fingerprint storage, but the extension point is this Factory. Another approach is that you have a fingerprint storage extension point on and so on. So both approaches have their own advantages and disadvantages and, for example, if you want to support multiple fingerprint storages, let's say for migration path. Maybe it's better to think about the factory approach, but I'm not 100% sure about that.

A

But in principle it looks to be feasible for this particular case.

D

D

You can probably just keep it as simple as possible and just activate the new storage unconditionally when this plug-in is installed. You know once if there are multiple multiple implementations and a.

A

D

Users, then you can start to think about. Does it make sense to have multiple implementation simultaneously? Do you need to control which one is used in certain folders? Do you need to mark which engine was used for a given, build things like that? But it's probably premature to do things like that now so yeah the same question with chip 210, which was about pluggable logging and chap 202, which was about pluggable artifacts, so for 202 we wound up creating a configuration UI for this 4010. There's still nothing at the moment. Well,.

A

A

Yes, but both stories finally ended up is using factory as an extension point, because in that case, what was our solution, for example, for build histories ephedrine, because both stories actually depend on the build histories and they we were able to just keep storage reference right hand, side in built metadata, and hence we did not have to worry about the mutation flow, because our mutation flow was that you introduced new storage, all the bills and what result storage and eventually, when you delete all this history, you don't need won't be needed or something like that.

A

D

There are different design choices in terms of how much information about the external storage you actually want to keep in Jenkins build metadata for the artifact min address. 3 we chose to well the new artifact manager records the fact that it is using s3 for a particular build, but it does not record the details, such as the name of the s3 bucket. That's part of Jenkins global configuration.

D

We did that intentionally so that, for example, you could rename a bucket and not only change one spot in Jenkins global configuration, and you would still have access to historical build records that way so, depending on what the usage model is and how important access is for historical builds. You can make different choices about where to keep different pieces of information where they should be updated.

A

So just what would be your advice for fingerprint storage, because it's not really using Camille's. It's for a system, level, storage,.

D

It refers to builds by ranges, but yes, in this system level, storage essentially.

D

Yeah I haven't thought about it enough to really have an opinion.

A

It might be difficult, though yeah, oh and one that fingerprints have unique IDs. So potentially we could somehow much it on the level of fingerprint IDs. So if ID exists, existing storage just using word and when you choose the default storage, it becomes a lot more tricky. If you want to somehow really rate with. Multiple storage is based on folders, etc.

A

But I think that this use cases are rather out of the scope, so jisub project, because it's really foolish scale instances and to do that we need a lot of feedback so that we design it properly.

D

Yes, I think people people running large enough servers that they would benefit from external storage or probably willing to do some extra manual setup or migration, or something like that. So it's more important to know not worry so much about those kinds of things. At this stage.

A

Because we also do have issues with internal organization when we use radius. If we took out a storage like elasticsearch, there would be a lot of section which is on that side and principal. It would have been easy to use elastic, search to search the data, but storage is is when we start.

D

Is currently the at least the file fingerprints and Jenkins use md5 hashes. We would like to replace that with sha-256 or similar in the future.

D

Does I'm just wondering if that has any any relationship to this or if the interface that you're choosing will automatically support different different kinds of hash algorithms, if we switch to them in the future.

B

So I think the same thing in mind when this would start terminology two IDs. So at the moment we are in our API. We have left it flexible enough that when we do make the switch to say, maybe we can choose between different hashing algorithms, then this the same API can work even then, maybe.

A

It makes sense to show the code here because I guess so integrated in this pool request.

B

So here we've used the idea terminology just left, so we've not used md5 mark behind, but for this particular.

A

D

It's the implementation of that get hash string in the future. If we YouTube Hey.

A

So, basically, when we were discussing the API first of all, this API is before into the better notation, so it can be changed to eat if needed, but yeah. There was a discussion of what you want to use, they're, passing by array or whatever right hand, side, API or things I believe that our decision was to postpone the decision. Until we see all the use cases for that it was yeah.

A

The specifics of PBM zinc is that there is already a lot of conversions between by strings and strings, and if we could reduce number of these conversions, it would be also better for performance, etcetera, so I think it's subject to change.

B

Yeah, so user destroyed this end for at least four ladies, we were rather des using red string, so it made sense at least for addressed by or in future, if not.

D

My important thing for this API is to make sure that you've identified all of the access patterns and say you know what the correct keys are.

D

I think for fingerprint storage is relatively simple, but you know you can put a fingerprint, but the only query really is I'm giving you an ID and I want to know both set of build records associated with that and I can get a single response with all of that data, as one say, an HTTP response, or something like that, you for some other kinds of storage systems like j-unit, for example, it's a lot more complicated to try to decide what kinds of queries we might want to support here.

D

I think there's no, for example, there's no need for an efficient way of finding all of the fingerprints produced by one job or something because there is no Jenkins feature that does that right. The only thing we do is look up a particular fingerprint and get builds back from it.

A

Yes, and no, because that is all fingerprints you can open and see. Fingerprints action basically lists all fingerprints for build they're, also endorsed searches to dump on finger prints pages so yeah. Well, yes, I! Don't think that it's fully done deal how it would work right.

D

Thing given a build, enumerate all associated fingerprints, yes, that that certainly needs to be secured by the API.

A

Now we need to be added.

D

To this extension point yeah.

A

Also, maybe look up a case, so we just check for resistance itself. Is out really opening the date and be surprising there and yeah. They might be a lot of iterations, but a good print storage which could have an advantage because of the number of his pieces currently in Jenkins. Isn't that high? So it's artifacts its credentials to some extent it's adopter images and we very existence begins. So we can just take a look at these use. Cases maybe create some benchmarks, use encouraging case test, harness, etc, and, yes, that from there right.

D

Yeah I think the number of plugins are potentially affected by the API is very small credentials. Docker Commons or Ducker workflow.

A

Look accessibility if it still looks yep.

D

But yeah so as long as we have a pretty small set of places to look for all of the valid use cases as far as I know, but you do need to do that search. So you need to go through and do some sort of text search of open source, Jenkins plugins, looking for interesting uses of fingerprints and decide if those those uses are doing something that would be covered by this API or, if they're, not. If they're doing something important enough to add.

A

A

Anyway, it's a great start yeah. Basically, you have something working by the end of the first phase. So if you pick an alpha release, we'll probably get some feedback from users.

A

Finger prints is probably not the biggest offender in terms of disk space, etc. But still it's an isolated use case we could address and it will give us some more insights for what we do most for other stoppages.

D

Yeah for some users it can be pretty big. For example, if you use the pipeline maven plug-in and don't turn off the feature to keep fingerprints, you can quickly get multi-gigabyte databases, yeah.

A

Actually, that's a good subject to explore because your pipeline mightiness quite widely used plug-in. So why not I really didn't think about focusing testing on this plugin but yeah? It might make sense and right now a pipeline maven. Sorry, yes, pipeline maven plugin also has its own local database. So maybe they could do some opportunities to see whether it could be partially integrated with the fingerprint storage I'm, not sure.

D

Yeah I, don't remember with that database he's been.

A

D

I think it's snapshot tracking yeah, which.

A

D

Fingerprint certainly, but not.

A

D

Exactly the same could maybe re-implement that on top of fingerprints, we're not sure.

A

It might be interesting because otherwise you would have to maintain this databases, and it already has at least two of three, so we had H 2, it had Postgres and it had mysql or mariadb, so it has had. He becomes if I'm right.

A

So any additional feedback or comments.

D

Yeah I just track the that that question about the lookup of fingerprints per build and.

C

D

Doesn't seem to be an issue as far as I can tell, because that started as a run. Action yeah.

A

Because yeah I believe it's the storage limitations at the moment because they had a file system storage and we do not really cache fingerprints or whatever, so we load them on demand. So work is actually in order to get any kind of feasible performance without really great reputations or whatever. We don't and fingerprint release and actions. So we were persistent dissections and basically, this beta state still exists, but they exist as an action not in the database itself. Right so.

D

Yeah, so a nicer approach would be to allow this fingerprint storage extension point to turn off fingerprint action and have the equivalent functionality be re-implemented by a proper indexed search when a database I'm not sure. If that's something you could do in Redis exactly because it's that's something you would more do in a relational database.

D

Perhaps I, don't I, don't know Redis.

A

This is it a relational enough, I would say no yeah the might still post some issues with indexing, but I think its topic could be researched.

D

Yeah there would be nice because the implementation of fingerprint action is as some complicated optimizations and to try to minimize the amount of heap memory being used in each case, because that's often been a performance problem for us. So.

C

D

We could just keep that out if the build record entirely and just look it up from external storage. If and when that information is needed, that would be much nicer.

A

Any new use cases for fingerprints beyond 30 photos, potentials and images.

A

Anyway, where you would like to see fingerprints.

D

It might be interesting to see if there's any useful interaction with the artifact repository like Nexus, or something like that.

D

So, if you, if your build, is uploading artifacts to Nexus, for example, should we have the ability to query which, which already does its own its own hashing I'm, not sure if it does any kind of index of those hashes, but is there some sort of interesting interaction we could have with that, so that you would be able to see not only the Jenkins build that produced some given file, but also look it up in Nexus or vice versa.

D

I, don't know if there's anything to do here that that's something if you're thinking about the use cases for fingerprints and how that relates to external systems,.

A

It would be interesting, especially since obstructed on Nexus quite popular, maybe a discussion spotting, creating creative or artifact managers for that I believe it hasn't happened yet, but in principle it would be possible and may be useful for some.

A

Nice, his keys I was also thinking about using fingerprints for script approval.

A

So in Jenkins we have a script security engine where in some cases you can approve, screeds or particular pipeline commands or methods for being used, so I wonder whether it would make sense to add more traceability there. So if you participate it in your yards record on what ik for when you're presented in new UI, which adds some additional tracing or security, but my name is fingerprints, we couldn't go you and Farrah.

D

Yeah I, don't think you should use fingerprints for that I mean first of all, it should be a very small number of items then, ideally zero, if you're running Jenkins properly and second, it has more it's much more serious security implications than fingerprint matching does so I would keep that separate.

A

A

So any additional feedback I.

A

Think that we should really target integrating this pull request, at least as beta these all these cleaners, but I did some testing for file. Storage basically works, as is so pretty much like we presented during the demo and contributes to the code refactoring, which is also already a great thing, because fingerprint code isn't exactly the most structured code and the junk is called it's not that big, but yeah. We can go from four months back.

A

Okay, so if there is no more feedback thanks a lot to submit for the presentation, yeah looking forward to see it in production me do ya thanks, Alissa patient! If you have any questions, we have a very good shot we can join and where you can ask any questions, so yeah I think we will share this presentation after the call. So you can find all the references today, yeah thanks so also the recording.