Jakarta EE Tech Talks, 19 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Automated Refactoring for Java-Based Applications | Jakarta Tech Talk | October 19, 2021

Description

This talk will focus on how engineering teams can leverage automated refactoring tools that originated at Netflix to greatly reduce the time and effort required when migrating to new versions of frameworks like Micronaut, Spring, Jakarta, and between Java language versions.

Does this scenario sound familiar?

Your organization has many Java-based microservices and applications that are critical for the day-to-day operation of your business. Over time, you find that your applications have a large number of dependencies on third-party technologies and libraries that have different release cadences. Invariably, your organization encounters pressures to upgrade those dependencies for any number of the following reasons: critical bug fixes, security updates, expiring support, "modernization", or technical enablement. Refactoring a single application to update its third-party dependencies can be tricky and often requires manual intervention when there are breaking changes. At the organization level, performing the same refactoring operation across all of your applications can result in weeks of effort.

This talk introduces OpenRewrite, an open-source ecosystem that specializes in mass, automated refactoring with a focus on Java source code, Maven build files, XML, YAML, and property files. We will discuss how the framework enables the creation of composable, prepackaged recipes that can be used to automate common framework migration and how these recipes can be integrated into both Maven and Gradle builds.

This talk will include some live demonstrations of Rewrite's tools to perform automated migrations on real-world, Java applications and libraries.

A

Hello, everyone and welcome to another jakarta tech talk joining us today. We have tyler van gorder to discuss automated refactoring for java based applications. Tyler is a software engineer at modern, with a strong interest in learning new technologies, defining pragmatic approaches to software architecture and sharing with others. If you have any questions for tyler as we move through, today's presentation feel free to ask it in the chat or using the ask a questions tab at the bottom, so without any further delay tyler over to you all right.

B

Thank you thanks everybody for for coming. I know people are probably fatigued from online meetings uh this last year year and a half. um So I really appreciate uh you taking the time to come to this presentation um today. We're gonna be talking um about large-scale, uh automated refactoring and, more specifically, um we've got a project called open rain right. It's actually an ecosystem projects, um it's an open source, semantic type, aware a search and transformation.

B

It's a lot of words, but this framework. uh Basically it allows you know you to automate. You know things that are typically tedious. So you know sorry.

B

So open rewrite started at netflix in 2016.

B

and uh it was originally you know the intent was uh you know: netflix had a logging framework, it was very similar to slf for j and uh they wanted to move to slf 4j and they needed to figure out a way to kind of migrate. All of their. You know microservices to this common framework, so you can.

A

B

With netflix culture of you know, freedom responsibility. There was a wide range of you know, styles and you know different frameworks in place, so it actually is kind of tricky, even though the framework, you know you know, logging, it's you know not that exciting, but you know there were slight differences between you know the api.

B

So, for instance, you know the arguments for you know, like you know, passing parameters were in different orders and that sort of thing, so you know it was a little bit tricky and not easy to do with something like regular expressions. um Additionally, because netflix had um you know different styles, you know it was really difficult to build something that would automate.

B

This and kind of you know maintain the same styles, so open rewrite was a framework to kind of you know built to kind of tackle these problems, so um a couple of things that it can do. Well, obviously it can take. You know code that typically you wouldn't be able to do with like something like regular expression and transform it.

B

um It can also be used to patch uh security vulnerabilities, so you know think you know if the cv comes in, there are a lot of tools out there that will detect uh that you have a security issue. um Owen rewrite can find and actually auto remediate those things um with uh using what we call recipes um and then, finally, uh you know a couple other things it can do um finding and fixing. You know issues that are like reported by stack analysis tools. So, like you know, I think everybody has sonar.

B

uh You know or something similar in their ci pipeline and basically you get these reports and you get these really nasty. You know, hey you've got you know 60 days of tech debt because of these issues so again, open rewrite is a framework that can not only you know, report these issues but can also remediate a lot of those and um and finally um I think that if this you know gets a little, this takes off a little bit more. We, we really definitely see this.

B

uh You know being used for framework migration, so think of library. Authors like uh micronaut uh spring quarkus, that sort of thing when they make a breaking change. um You know, wouldn't it be nice if you had a tool that could help your users migrate to the new versions and again open rear.

B

It has the capability of doing that and we're going to show a lot of uh demos today, where we, basically, you know, we'll, show this in in practice um and then specifically since uh we're you know we're talking with jakarta folks, uh we will definitely focus on some of the jakarta stuff and also um you know migrating to a new version like java, 11 and java 17, and you know some of the panes that come with that.

B

Like you know, moving from java x to jakarta name spaces, um you know adding uh dependencies for like jax, b and jax ws to your runtime when you move to java 11.. These are the type things that open rewrite is really good at um so bear with me. The this is a very slide light. uh You know demo or a presentation.

B

I think I have a total of seven slides, including the title and like the end, so there you won't hear me drone on too much longer. um So, let's get let's keep going so for uh for code, maintainability is destiny. I mean 20 to 30 of a developer's time is spent on these tedious. You know things like uh you know. You have to keep your frameworks up to date, so you know you're getting support so you're getting the latest security.

B

You know uh updates um and, of course, each migration you know can potentially take you know months of effort depending on your organization. You know you figure changing one project sure you can do that in a bit, but if you're changing it across your entire org, you know like, for instance, moving from java 8 to java 11 or java 11 to 17.

B

These are notoriously long running tasks and can be really hard to do and and really difficult to cross your whole org so and then, of course, cv patching is constant right. You, you have to address security concerns. So um let's talk a little about this. So open rewrite uh is built on abstract, syntax trees. um Again, that's not that exciting we're, not the first person or the first organization to to use. You know asts.

B

um However, there are a couple of attributes about uh rewrites asts that are kind of special. um You know. The first is that we have type information. I think they're, probably other asts, that have type information. You know semantically aware, like you know, you have a structure of the code in a tree, but you can tell that this variable is an int or you know this method returns a string that sort of thing. So that's one attribute the second one and it doesn't sound.

B

That exciting is that it preserves formatting and that is really actually quite tricky um again getting back to you know the original problem in netflix. um You know some teams might use spaces, some might use tabs, they have different edition styles. They have different import orders. You know that sort of thing there's a lot to consider with formatting.

B

So if you're going to do any code transformations, uh you need to be able to detect uh the formatting uh of the of the code uh when you make the change so that when you know the automated changes are applied, they're idiomatic with inconsistent with the formatting of the code around them, so that's actually really uh quite tricky.

B

So we're you know we're doing a job presentation. So clearly, there's there's a java ast, but that's not the only thing that open rewrite uh does we have uh parsers for xml yaml property files? So you figure. Not only do you have semantic awareness of java code, but you also have symmetric awareness about you know these fun. You know configuration formats, so it allows you to kind of extend the platform into making changes to properties, and you know formatting yaml and then, of course, if you support xml, we also support maven.

B

So we've uh built a semantically aware parser on top of our xml parser, so we then have better awareness when we're looking at an xml file that happens to be a palm file. We know that these are the dependencies and these you know these plug-ins dependency management. That sort of thing um and then similarly, if we have a gamble parser that you know, opens up a lot of doors for us to do things like building uh semantic awareness for kubernetes uh github actions. uh You know that sort of thing, so it's it's definitely expanding.

B

um We also have uh a parser for hcl. So hashcorp's, you know configuration language so that kind of lets us kind of extend into that as well and then soon coming very soon. We just introduced, uh you know groovy, so it's got.

B

It has some initial support for groovy uh and obviously built on top of that will be uh gradle support, so it'll allow us to do some transformations uh with with gradle as well, and then you know finally on our on our roadmap is uh typescript as well, um so we are planning on kind of making this. uh You know you know polyglot and you know multi-language so, okay, so all this talk, let's get into kind of what what this is really good at. um This is an example.

B

Actually I had to do an organization I I worked at. uh We were moving to junit5, I'm sure a lot of people have had this task or are maybe tasked with it. You know now- um and this is what jada4 looks like when you're trying to like uh you know, assert that a certain exception is being thrown. um You know, you have a rule here, that's you know a an instance variable of your test class and then, basically you need to make sure you set that to expect and the framework will.

B

Then you know, do its thing in java and j into five. uh They changed this quite a bit. Obviously the you know the instance variable went away, uh and now you can see that you know you were into search throws, and you say I'm looking at this class and there's a lambda expression. So you know so. Instead, uh you know a little nicer, really difficult to do uh this transformation in an automated way with you know, tools that are out there today.

B

So this is kind of what got me hooked on on open rewrite. Is it actually can do this very transformation? So I was, I was excited as a developer, to see something that did that and it can do it at scale so like, as I promised it's kind of a slide like presentation, I would like to go through a lot of demos. Actually, so all the things you're going to see in our sas. This is a sass that modern has set up, but all of these recipes are open source.

B

So again, a recipe is just a a unit of transformation, either a search or a transformation recipe um they're composable, but everything you're going to see here is in open source. So I will also demonstrate how you can apply these same recipes using either a gradle or maven plug-in uh you. You have support for for doing these. These same exact transformations uh one project at a time. This is just the kind of a nicer way to kind of demonstrate how uh open rewrite or the capabilities that open marian has so further ado.

B

um We're going to start with uh github actions. uh I know we're in a java talk, but I just wanted to talk a little bit about this, to kind of demonstrate that not only just java, but we do some other things as well.

B

So uh recently you know, obviously, uh oh the adopt open, jdk moved to eclipse, uh timron and uh and basically, oh, the adopt open, jdk image will no longer be updated, so uh this is going to be kind of a hassle for for folks that are using github actions that have tons of projects on it. So this just kind of gives you an idea what we can do here so first kind of like what we're looking at what the heck are. We seeing here um this sas?

B

What we've done is: we've basically ingested a whole ton of open source uh repositories. You can see we have 37 repositories and apache just to give you an idea. These are not small projects. These are pretty good-sized projects, um so we're going to run all these recipes across this it's kind of like a sandbox.

B

You know approach so going back to github actions.

B

And I'm just going to select all repositories and do a drive run here. So what this is going to do is it's going to go basically traverse over all of the projects we have adjusted here, find all github action files. You know across all those repos and then make these changes not terribly exciting, but you can see here just an example.

B

uh Look it. It went into the workflow and and just changed it. I mean it's a pretty simple change, but you can imagine if you have thousands of these uh like there are not thousands but there's quite a few um and then uh some things that we've done here is. We can basically come here and say just commit all so this is one quick way to kind of remediate: hey, let's fix all of our github actions in one shot, um so just kind of a you know.

B

I wanted to first step into kind of things that are not java. Just to give you an idea uh what is out there, and so you know, we'll notice. We have github actions and concourse and circle ci we're always looking for for new recipes. uh You know so please chime in uh if you are, if there's something that we're missing.

B

um Okay, we're gonna go to java now and we're going to use some of the search capabilities of open rewrite. So with this we're going to go to uh search- and in this case um I'm just going to- I want to find like the surface area of like, like an apis, so we're going to go to find methods.

B

And this recipe uses a syntax. That's it's very similar to the point cut expressions so that you may have seen or if you've ever used aspect j. um So in here I'm going to do com google common collect, so let's basically want to sign any project. That's using anything in the collect package, I'm going to use any method with any arguments and I'll just run this across all repositories. Why not.

B

Okay and we open rewrite trade, just searches, basically as kind of a specialized transformation, um so we kind of take advantage of the fact that we can do just on this thing and we make we make what we call. uh We add a marker to the class so like normally, if you printed out the source code, um you wouldn't see anything, but if you have a special printer that knows how to print markers, then we basically reify these markers as just comments and then they show up in the diffs. So you can see here.

B

This has just given us an idea of wherever you know. It's not immediately obvious that this is. uh You know, google, commons right. If I go to this, though see.

B

Somewhere in here there it is google common. So uh just give you a quick idea, I mean this is not you know terribly uh different from what you might get with with sim grip, uh but we we like.

B

We kind of take a different approach here. Is we don't have a query language per se? We basically have these recipes that are kind of building blocks. So you know going back to you know to java uh with search you'll, see, there's a ton of other different type of searches here, so we can search for uh let's, let's try this one, we're gonna look for all uses of deprecated classes. So obviously there's going to be a ton of these. um I could run this across the whole repo. Why not we'll just run it across everything.

B

Oops, okay, so you can see here it's saying: hey this webmbc configure adapter again, I just picked the first one off here. uh Is it's saying it's it's deprecated. So let's go make sure that's true!

B

um I won't be able to go to that because I am not the right project. um Let me try to find a better one here. How about we go to this.

B

Okay, let's go to active queue, so you can see here. It's saying this reliable transport is uh is deprecated. If you click on these little bars over here, uh it'll take us to so you can see we're just on github at that project. So if we click on this reliable transfer- and we actually go here- it is indeed deprecated.

B

So um that's uh you know pretty powerful and and honestly you once you start kind of using this stuff, you, you really get used to the capabilities uh and it's hard for you to picture a world uh without having this type of search capabilities, uh and it's really fast. I mean you know like I said uh we have 65 million lines of code in here and uh again I've got this filtered right now, so I'm not showing I'm not showing projects where we don't have any results.

B

So uh you can see, there's obviously going to be a ton of deprecated. You know apis and use. So, okay, that's kind of the search capabilities. um Let's go a different direction. Now, let's go to maven it's kind of a similar theme. This is uh we're going to do some searching in maven, but in this case I want to do dependency insight. um What if I wanted to do something like this?

B

I want to find all maven projects where junit is in the compiled scope and go.

B

Obviously, you wouldn't want to have junit.

A

B

Your compile scope, that's generally a bad thing and again we'll just look at one of these.

B

Again, I'm picking on the projects that happen to start with a because you're closer, but again I'll try to pick some other ones, so it doesn't feel like I'm picking on the same people all the time. um So in this case uh you could see that it's telling us that the j unit uh you know dependency is actually leaking in uh as a transitive dependency of this other thing. So if we.

B

That this is indeed a maven file and it's in just dependencies there's no scope to find so yeah. It's going to add this. So again, this is just kind of giving you a taste of what semantic awareness and maven java- and you know, yaml kind of gives you uh you get a little better awareness in this platform, so all right cool. So obviously the best thing to do here would be to exclude this. So yes, why can't we use?

B

Why not use open rewrite to actually do to automate that so um I'm going to go back to uh maven.

B

So, in addition to search, we have building blocks like this. One exclude maven dependency, so we're basically going to say all right, junit j unit. I want you to remove it from the compile scope and this time since we were looking at that one specific project. uh It was alibaba me clear everything.

B

uh Jars link yeah, okay,.

B

Another dry run here and you can see, there's two results again and cool, and this time it actually added the exclusions for us. So um again, this is uh just kind of showing off some of the some. This is our first semantic transformation where we've- actually, you know, made this change here, um a couple of things that are really kind of non-obvious here at first, uh because as a human, you just go. Of course I wouldn't indent it this level is uh we're preserving formatting.

B

So what this is actually done is it's checked it's finding out whether it's using tabs or spaces, it's figuring out what the indention level is and making sure it gets. uh You know the the change is, is idiomatic with you know, consistent with the you know, the code around it. So it's not just java, but you know we can do that in maven as well. um So that's pretty cool um and now, uh let's switch gears again uh we're going to go over to security this time. So uh we'll do some java security.

B

uh This will be our first uh again, I'm gonna I'm gonna kind of build on this. You know to give you an idea that you know, like I said: recipes are composable, so we're going to start with a couple of really small recipes and we'll kind of you know get more and more complicated as we go so security in java we're going to go to core, and you can see here, we have uh this one here. It says: uh use secure, temp, uh file creation, so it's basically telling us hey, there's exploitable file permissions.

B

You want to use just the normal java, io file create temp file and it said hey. You should probably be using this in. I o version, so we're gonna go ahead and click that and let's change the repositories. We're gonna apply this against.

B

um Let's just do them all.

B

uh The recipes run relatively quickly so.

B

Let's try this.

B

And as I say that you watch, it won't go as fast as I would. I would have hoped.

B

Okay, good stuff, um so again it takes a while to you know. You know basically provision the work on and we have. We have a you know a lot of workers behind the scenes doing this work, but um specifically, let's go look at what it what it did here. So I can go to apache druid.

B

And just give you an idea, um so this is our first uh look at what the java transformations look like. um You can see here that it's not exactly trivial here. We had to make sure that we're adding the import, and we also had to make sure, obviously we're using files instead and we're going to file. So we basically are taking the arguments that were originally in here and place them in here so again, because we have asts, we can kind of do this kind of you know very intelligent.

B

You know change and again just to kind of hammer it home again and again uh it is going to detect styling and including you'll notice that it actually put this in the right spot. It does detect uh import order and we'll make sure that it puts it in the right spot so like. If you have things organized by groups and you have static imports at the bottom, that sort of thing it will detect that and make sure it puts it in the right spot.

B

So uh you can see here. It's just you know a bunch of these. It's uh pretty pretty interesting. So that's that, um let's uh move over again to code cleanup, we're gonna, get a little a little more complicated here, going back to java I'll just do a search this time um common, static analysis.

B

Okay, again, I just want to hammer home. All of these recipes are open source. The fact that we're using the sass to kind of do this- it's just nicer to see the it's quicker to see the results all right. So what this is is actually a composite recipe which you will recognize, probably if you've ever used a sonar. You'll recognize many of these things on this list.

B

What this is going to do is not only find them but auto remediate them, which is super cool, uh so you can see, like you know, adding serializability when you know serializable class doesn't have it. You know big moving big decimal rounding constant to the rounding mode enumerations.

B

uh You know things like using the diamond operator right since java 7., uh you know you, don't you don't need to. uh You know, specify the type on the right side when you know the left, when you know the type on the left side. So this is going to run all of this stuff um across you know. Basically, um you know whatever repositories, we decide that we're going to run it on.

B

So, uh for the sake of I guess time, I am going to uh clear this and I'm gonna pick on one of our own projects. um I'll use open rewrite uh we'll just pick our spring. One.

B

All right we'll pick kubernetes, it got snuck in there too, uh and do a dry run on it.

B

So now this is going to run all of those remediation things on the projects. So let's go see what type of things it did here, um so this is uh basically reversing uh the equals so that we don't get a no pointer exception. uh That's pretty much! What all of these are and again uh in the sas. We always have this here. It'll tell us which recipe made the change, so that information is always attached with these changes. So you can get the same information when we're using maven or the gradle plugin as well.

B

So you can get an idea like, for instance, this one's saying, hey. This is the final class and you're like. Why did it make this change uh as the final modifier? Okay, sure, there's no public or packaged private constructors makes sense. So these are just things that we've pulled from from sonar. um There are obviously many static analysis tools out there, and this is an ever-growing library of these type of things.

B

So if there's something here that isn't covered, uh we would love to hear from users in terms of you know what we're missing so, okay, so that's the I guess the first peek into uh you know a little more complex. uh You know code transformation, um this one's actually kind of fun too. uh I'm gonna do another one, just because it's uh it's kind of cool um fix, serializable all right. This is it's a sonar issue, um but this one's kind of interesting.

B

uh So, basically, uh if you think about this, uh this is a little bit tricky to automate.

B

If you have a class that implements serializable um all of its builds either the marked transient or, alternatively, you need to make sure the class that is included is serializable.

B

That's kind of what this thing is going to do. um The way it works is it basically says hey if the source is in the same project, as you know, where it's discovered, where you know, I feel it's not transient. It'll actually go and make that other thing serializable.

B

If it's a a type that is not as part of the project, it will instead make it transient. And, of course, you have options here, you can say I don't ever want to mark and make anything serializable I want to make them always make them transient or, alternatively, I can kind of, like you know, mix this I can say like if I run.

A

B

A

B

I had like a factory that got included and then the factory has a bunch of things to serialize, so you can imagine how this would explode and everything in the world becomes serializable.

B

So there's ways of short-circuiting uh this. This work, so um I'm going to uh run this against. uh Let's see um just pick one day, final sure, um okay, so let's see what it does here, so seven results. Okay, all right- and you can see here- it's basically telling us uh based on this transformation. It's saying hey: it marked this http session uh as a transient, interesting, um so yeah. That tells me that http session is not serializable, but something about this thing is so this thing extends hashmap and again.

B

A lot of these things are not always obvious, but if we look at hashmap and we sure enough, it implements serializable. So these type of issues can kind of leak in when you don't when you least expect them. uh So it's doing the right thing here. um It always could take some human intervention to kind of make a decision whether or not this is indeed what they want to do or whether or not you know maybe there's something else we need to do here. um So just gives you an idea.

B

You know, like it'll kind of detect these sort of things again, here's a map. It's saying this thing has to be transient because of course map is it implements serializable as well? uh So that's you know kind of interesting. You can see here this window interface. It's not always obvious that these things influence it like in this case, you know, render info implement serialize. Well, that's pretty obvious, um but again going back to this one.

B

It wasn't immediately obvious because it doesn't actually implement serializable. It's only because we're extending hash map that the contract kind of gets sucked in there. So again, semantic awareness is key here, knowing you know the inheritance and knowing what you know, interfaces that the parent may implement uh really helps in these sort of situations, so uh cool all right. Now, uh let's go back to something that will hit a little closer, hopefully to this audience.

B

um Since this is a jakarta, talk, um we're going to try uh moving from java x to jakarta, so uh we have a whole category of things that in java we call modernize so you're going to find things like.

B

Basically, obviously, this one we're going to run here, but there's also things recipes, and these are very much in their infancy. So I'm a little nervous to run this with this audience, because uh you know again the jakarta experts here, um but we'll see we'll see how this goes. um So, I'm gonna again for the sake of this I'm trying to pick different. uh You know different repositories. um We're gonna go to quark.

B

Carcass cxf yeah all right, we're just gonna run it on this. One just see how this works all right, cool, it's done.

B

And you can see uh again because we have semantic awareness inside the maven. You know the maven structure. We can really do intelligent things like we can follow the dependency to the property where it's defined. So uh it's pretty cool that we can do this sort of thing. So you can see here change the version cool um to you know three 301.. So of course we're gonna go from two to three uh we're going from jade uh we're going from jakarta, e8 to jakarta ee9, so we're gonna have to also change the name spaces here.

B

So that's also doing that work. So you can see here um we have some building blocks for changing whole packages. So again, if it's not very exciting, you can see, we've got this recipe, that's basically cool changing that. um So it's going to be changing all this xml bind and inject. You know that sort of thing. So um and then let's see what did we change here? Oh yeah, jakarta, namespace. So again it's detecting the fully qualified name in line so against magic. Wares.

B

Is this key here, so it'll do all those transformations and again another important thing. Is it's not too crazy different, but you can tell that it's inserted in the right place, you'll notice that it didn't add the jakarta stuff under in this. You know second group. So again it's detecting import order and trying to do its best to put it in the right spot.

B

So, okay, so neat, that's again, just in its infancy. um I I again, I kind of want to show what that might uh look like.

B

um This is not something I actually planned on doing, but let's, let's just go, uh I just kind of want to show real briefly what this stuff looks like uh in code, so I I mentioned this already that you know we've got a whole framework for building this and we're actually going to try to code one of these real quick in a in a bit, but I wanted to touch on like again. This is just an open source project.

B

You can get this and you get to this uh via github uh and let's see java x to jakarta. So uh this is the yaml file. So you're gonna see here that the gigantic ammo file is basically defining uh all. Basically, all the things we're going to do with this recipe, so just give you an idea that you know some of these things are kind of imperative, and some of these things are declarative and so just know, there's you know behind the scenes this stuff can be composed via yaml files.

B

uh You know pretty neat okay, so.

B

That's that one uh let's go kind of a different direction uh and go to java and again you'll notice that we have things from micronaut for quarkus and for spring. um I'm gonna run a couple of these just to give you an idea, so we're gonna uh we're really excited about this. um We we hope that more framework authors do this. um The micron team uh had the presence of mind to kind of work with us to build recipes to help bring their users from micronaut 2 to 3.

B

So, basically, if they made breaking changes, they provided recipes to help their users migrate to 3-0 and kind of, like you know, ease the burden of moving to the new version you could you could imagine uh the uh this kind of drives support. uh You know you, you you're able to deprecate things quicker. uh Your users aren't as pissed off because they can. You know, safely, move to new versions. So let's go look at what this thing. uh What this thing does so, in this case I've kind of cheated.

B

I have a specific repository I want to show. I want to show this transformation.

A

B

So again, uh mike are not. This is actually included with you know as part of their their migration strategy. So if you go to their documentation.

B

B

Oh, I picked the wrong thing.

B

I picked quarkus when I should have picked the micron project. So let me go back to that. Let me try that again.

A

B

Three would help if I picked the right project um so in this case.

B

Hipster sample micronaut, let's actually pick a micronet project, we might get better results. I don't know.

B

Okay, yeah, we have some results.

B

Let's see what this thing did uh so again. This is a transformation. That's now going across both java and maven. uh You can see it is smart enough to traverse dependencies up into the property and go oh. I need to make that change in the properties, not in the actual dependencies, but if the version were in the dependencies, it would make the change there um and then again. This is where it really becomes handy. Like you see here, it says, type requires introspection. What the heck does that mean?

B

uh We try to make a description of why this change was made uh with each one of these again. We would love to work with more framework authors and we honestly love to have the framework authors more or less own. These recipes uh with our support. So we can kind of provide this type of interaction with all the frameworks for anybody that wants to do it. So um so these are all the type. You know introspection introspection, um here's one, that's a little more complicated! You can see.

B

It's made a change here to once per filter. What the heck does this mean? uh Okay executed once per request, directly implemented instead of extending so okay. What does that mean? So you can see here it. uh It changed the import uh it's using this http server filter. Now you can also see that it went from an extends to an implement that filter and uh it changed this to a protected. You know method instead of public, okay, cool neat um and again this is all being done.

B

I I would challenge you to try to do that with the regular expression. You'd have a pretty tough time doing it. um Okay, so that's it! So that's just an idea of you know kind of the type of experience you can give your users as you make breaking changes. First of all, you're not as scared, hopefully you're a little bit scared of making breaking changes, but not as scared as a framework offer. It just makes this a little less painful.

B

B

Quarkus, I'm looking at time, I'm running out of time here: uh okay,.

B

We're gonna pick marcus and this time we're going to do a migration to 2-0 so again same same drill, but now we're going to do this on a carcass project. I have two projects selected. I don't really need this micro knot project for a quarkus migration uh I'll do a dry run here.

B

Okay, cool same story, uh we're doing you know again. This is gonna, start being kind of a rinse and a repeat, so uh maybe I'll stop with the framework migrations, but uh you get the idea you can see here. It's just changing the version, two two three uh and then, of course, it's going to do a bunch of you know things that are required to make this work in quarkx 3.. So, um let's see here change type change type. Those are very exciting.

B

Here's one where we're, basically, uh here's a more complicated one. So this will give you a description, saying, hey place with the grpc, client and sure enough there. It is okay, cool uh here's, a fun one factory service. You can see here this. This had a whole bunch of recipes changing this. So if we go in here, it'll tell us wow, there's a lot of stuff in here everything I made a change is here again all this information is available. uh When you use, you know maven and as well.

B

So if you look at this, this is actually it's a pretty complicated. You know things. It's changing! That's just a method! Name change! Okay! That's not that exciting, but you can see here it's taking this this chain of calls in the builder or a factory and changing it to something different. So this gives you an idea of just kind of some of the power of the framework and the things that it is capable of doing. um You know this can seem kind of magical at first when you first see it so neat, okay.

B

Finally, uh let's see um I'm going to try something a little bit different, I'm going to show you what this experience is like uh when you are not in the sass. So to this point, you've seen all this stuff in this assets now very cool.

B

The first thing I am going to do, though, is I'm going to I'm going to use semantic search, I'm going to get back to the you know, the jakarta e theme, um I'm going to use a semantic search.

B

And I'll use maven and I will do search and I will say I want to find any project and again we're going to look at all of them.

B

We're going to look for your java x, xml web, so basically we're going to look for the old jax ws artifact uh we're going to look for the api. I want to find projects that are actually using the api.

B

Again, this is super cool, um a little better than trying to search github, for, I guess doing a text-based search. uh This will actually tell me all the places where projects are actually using this in one of their dependencies, so yeah cool meta model. Look at this apache metal model, I'm back to picking uh on projects that have to start with a sorry. um You can see here it's finding that in its sub-project salesforce palm xml uh sure enough there. It is so what? If we wanted to migrate this uh to to?

B

Let's assume this is a java 8 project. um Let's say we wanted to move this to java 11.. um What would be? What would this entail? So again, let's go! Look um all I've done here. Is I've just checked out this, uh this meta model project. So again, this is just obviously just out here on github, not very exciting. There's the palm for salesforce- and you know basically there it is it's definitely using jack's ws cool, okay,.

B

All right, I've made a couple of changes here. uh Let's just go look what these changes are. I wanted to kind of just highlight this um a little bit so.

B

In the palm, the only thing I've done here is I've, just added the uh rewrite maven plug-in. Let's make this up make this a little bigger, so people can see it.

B

So all I've done here is, you know, obviously there's documentation for how to how to add this, um I'm just adding an active recipe, I'm. Basically, this is a new one. That's in development! It actually isn't out. You know publicly, yet so still working on it. So this is a hint for all. You jakarta folks out there. I would love to have some help with this. um In terms of you know what we are doing with this recipe, and the other thing I did is for the sake of just making this a breaking change.

B

That was from like say, java, 8 java 11., I'm just actually referencing a jaxp context here, obviously that's coming in through the the jax ws api. We also know that if you try to run this in java, 11 that's going to break because that class won't no longer be there. So let's just prove that out so java uh version, you can see I'm using uh java 11..

B

If I do a clean package, let's do that, I'm in the uh salesforce sub project, so just for the sake of time, I don't want to run the whole build for a meta model.

B

Sure enough air, and if we roll up to this thing, you can see sure enough. It's saying I cannot find the jax b what the heck is going on so just like in the sas. uh You can do this from command line, so we'll do maven rewrite run and again, all I'm doing is I'm activating this new recipe? That's not there yet, but you know again all the recipes that you're seeing can be run through through maven.

B

A

B

To do now is download a whole bunch of stuff.

B

All right, you can see it's activating the recipe um it's going to parse all the maven yaml xml, anything it finds in here. It's going to you know basically parse it, and then you can see here. It actually says: hey. Some changes have been made. uh Two things made a change. Let's go look at what the changes look like and sure enough. The salesforce palm has been changed.

B

um Okay, what did it do? First of all, it removed the jax w uh the java x. uh You know jax ws api in favor of the jakarta one cool. That makes sense. I wanted to do that. It's using the same scope as the original and then additionally, it's finding anywhere, where there's a transit dependency on this api and making sure that there's also a runtime component. Here and again, you can see that it's just picking com and using the runtime there.

B

So if it's made those changes, then in theory I should be able to come back to this and run this again. Hopefully this will work now.

A

B

Yay it works, so this is a recipe like I said in current development. uh We want to make this generally available. This is gonna. We have. We have recipes similar for both jax b and jax ws, where they will do the same sort of thing: they'll detect the use of the java x artifact and move it to again. You'll notice that it's using the latest version of uh jakarta ee-8, um obviously we're going to have other things that will then take uh you from jh, jakarta, e8, 2, 9 and then do all those name.

B

Space changes, so we're trying to you know make these building blocks so uh organizations can uh you know kind of pick, you know what pace they wanna. You know make these changes so um that just gives you an idea what the experience is like from the command line. uh Again we have build plugins for maven and gradle, so neat. um Last but not least, um I'm gonna actually try to code up one of these recipes.

B

To give you just a taste of how these you know, the framework is, is, is used so uh I'm gonna go over to intellij and do this hopefully very quickly, um I, for the sake of time, I'm gonna be doing some cheating. I have this code. You know pasted um we're going to build a recipe that basically is going to add the missing, override annotation to any method that is overriding its parent. So again, this is something where semantic awareness really comes into play.

B

So I've mentioned already everything that we do is called a recipe: um okay, cool.

A

Let's see here.

B

All right we'll do that and then it's playing because there's things here we haven't implemented so implement uh display name so again, all that metadata you're, seeing on the site uh is you're basically defined in each recipe, so we're making a we're, making an imperative recipe here. So, um okay, so again, for the sake of time, I'm gonna just do some copy copy and paste here.

B

Okay, so we're doing that um and then the other thing the recipe provides. Is it's a wrapper around a visitor model, so we use uh the visitor pattern uh for uh for basically navigating our afc's, uh so we're gonna override methods here- and there is a get visitor here- and this is the visitor- that's actually going to do the concrete work. um So what we're going to do instead is we're going to return it again notice it's tree visitor, so this could be maven. This could be java.

B

What not so we're going to tell it we're since we know we're doing the java transformation we're going to use this java. Iso visitor.

B

And there'll be a lot there's a lot of documentation out there, so you know don't feel like if you feel like you're lost here at some point. uh Don't worry lots of docs and uh we're actually on slack a lot too. So we will. We can help people out uh if, when you're first using this api so same thing uh in the visitor, uh you know for the visitor pattern.

B

If you look at the methods in the visitor you're going to find there's a visit from what for every uh you know basically aspect of the java language, so we know that we're going to be looking for a method declaration, so we'll just do visit method declaration, we're gonna override the method. So basically uh the frameworks kind of call into this- and you know as it traverses the ast and it finds methods. uh This is the method it's going to call for the sake of time. um Do some cutting more cutting and pasting here.

B

um I know that in here we're going to be looking for a specific sort of, and we want to see if this annotation already exists on the method declaration. We're going to use this annotation matcher to do that.

B

We also know that if it's not there, we're going to need to be able to add the snippet of code and you'll notice. It's just a string here. This template is deceivingly complex, but what it's really doing is it allows you to take a a chunk of code. That's just written as a string and we'll then convert it to an it will basically insert it into an existing ast, parse it and then create ast elements.

B

So, instead of having to manually, you know you could do it you, you could definitely try to manually, create the ast it it's complicated um or you can use this template to do it. So that's this template kind of aids in doing that, and then finally, uh let's just implement the details of this method and again, I'm just going to paste for the sake of time here.

B

Okay, so what this is doing here is again. The method declaration has a lot of information that we can kind of interrogate. So we're basically saying hey are if you're static. uh We don't want to make this change.

B

um If you can see here we're grabbing all the annotations that are on this particular method and if none of them match uh this matcher, meaning it's not there, okay, cool and we're going to want to do this and then finally, uh we need to make sure that we check the type information to see if this is indeed overwriting, something in one of the super classes or one of the interfaces. So this is where that type information gets really uh really useful.

B

uh You can do really complex things easily with with this framework and then, finally, if all of those things are true, then we're gonna go ahead and you can see here we're mutating so, instead of returning basically the same method, we're going to mutate it with our template- and you can see inside here we have this coordinate system I'll get into this a little bit so like if I'm inside a method declaration. These are the type of places I can insert code using this template, like I can replace, throws the parameters.

B

I can add an annotation. um So again, there's a lot of help here in terms of how this and then something's kind of interesting. You notice that uh you kind of may wonder like. Why do we have a comparator here? Well, this is telling us that if we're going to insert this annotation on a method, where do we want to put it when there's more than one annotation, so we're basically saying all right. We want to alphabetize by the simple name here, so that's it! That's all that the code that's required.

B

um I've taken the time already to build a kind of a unit testing framework here uh to kind of you know, use this again. This is this. Is we switched over to kotlin, which is why I'm using intellij? You know I'm sorry. I apologize to all the clips folks out there. I do have eclipse installed by the way there it's I have both, um but this makes it easier when using kotlin. So um all I'm doing here is we're creating some stubs. So you can see here.

B

I've just created a couple of interfaces and then this interface extends this interface. That sort of thing- and then I have a couple of parents- and then all I have here is a couple of uh you know test methods in here. So basically saying you know, um let me just pull this over, so it's a little easier to kind of look at so in this case uh we're basically saying if, given this source code here, we expect this source code afterwards. So we are expecting uh what do we have an error? Okay,.

B

So you can see here trying to test to make sure that the sorting's working correctly we're basically saying that, because test parent and again, this method, test parent test, parent parent is uh in the test parent base class. So we kind of expect it to to work that way. So if I run this real quick, hopefully if I've done my my job correctly, this will actually work.

B

Yeah, it does and again that's not terribly exciting. If you don't trust me, uh let's just do this. Let's add an extra space here and again we're uh we're pretty sensitive to even the smallest of white spaces. So if there's an extra white space, this thing's going to fail um and it does so.

B

If we look at the difference sure enough, you know basically it's because there's an extra white space, so you can see the actual doesn't have one, but here we do so um again uh trying to get this formatting correct when you're making these automated transformations is really quite tricky. um We we, we spent a lot of time, uh making sure uh that this stuff works um and then I have a bunch of other tests that I think I'm running out of time.

B

So at this point I'm just going to uh you know, I guess ended here and one more thing go back to my slide. Deck.

B

Current slide, okay, last piece: um first of all, thanks for coming, uh we really do need your help. um Please please uh come look at us. uh We're at github open rewrite. um We would love if you would open an issue on github asking for a fix or migration or better.

B

Yet, if you've got some jakarta experience, which I suspect maybe some of the people in this, uh you know watching this presentation do we would love to get some help uh with some of these migration ideas um in in terms of develop further developing these recipes, um you can follow uh modern inc on twitter. That's uh the company! I work for uh our team is kind of split between working on uh the open source project.

B

So you know we do a lot of work in there and then we also have that the sas and then finally, all that cool- I guess ui you were seeing me use- is actually available um as a sandbox. So if you go to app.modernio, you can actually play around with these things, and I would encourage folks to just kind of play around and see the capabilities. The framework, uh um it's uh it's pretty darn compelling. um Okay, that's it.

A

All right, so it looks like we have one question here: um is there anything we can do to improve the resolution of the screen capture.

A

B

Is that just geared at the the video capture here.

A

um Jonathan, I'm not sure if you can add to your question if you're still here, uh oh.

B

That's jonathan yeah.

A

A

So I'm not sure that.

B

A

I'm not sure if he's still in here or not but looks like he might not be so yeah.

B

Okay, jonathan works, probably he's probably jonathan. I know the jump so what I guess my question is: was it just fuzzy or too small.

A

All right, so, in the meantime, um I would like to thank you again for that excellent presentation today, tyler and just a reminder, if anyone has any final questions in the audience just to pop them in the ask a question tab now at the bottom um and then this is just a reminder.

A

We have our next jakarta tech talk on november, 9th with alex sovik as he joins us to present jakarta grpc services, and you can register for that using the link I'll pop in the chat, um we're also looking to book more jakarta tech talks throughout the rest of 2021 and into 2022.

A

So if you're interested in presenting a jakarta tech talk feel free to fill out the form which I'll also put in the chat, and finally, if you haven't already done so, I would encourage you to register for the jakarta. One live stream in december.

A

It's the largest jakarta event of the year and you will not want to miss out, as we have a load of great talks coming so I'll put all these in the chat now and I don't see any other questions.

A

So I think that's pretty much it so we can kind of wrap up. So thanks again, tyler.

B

Yeah no problem. Thank you.

A

Thank you, everyone who came out. I hope you all have a great day.

B

A

Thanks thanks bye.