DFFML Rolling Alice: Engineering Logs: Volume 0: Architecting Alice, 7 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Architecting Alice: Volume 0: Documentation: Part 80: Cleaning Up the Docs Build

Description

https://github.com/intel/dffml/pull/1401#issuecomment-1170489046

A

All right, let's go so we are going to refactor the meta issue, creation to accept dynamic inputs, um and that means we're gonna make it into a let's flip over. Where are we let's go back? Okay? uh So what is this? We don't need this. We fix that bug all right, okay, so good status, so detached state. Why are we in a detached state.

A

Let's make sure we have all the exchanges: okay, is there anything or else no okay, gh pages? Why is the gh pages deploy failing? Why is the gh pages deploy failing.

A

Okay, that's it that's a question. Okay, so um let's check that out actually because this is really something that should work. So what's up um looks like because why do I say this? Well, the side is down now, but the action says it's fine. So uh clearly something is awry. um So it's 48 megabytes, which is a little bit shocking.

A

Let's see, what's in there, that's a lot, but I was in here.

A

Artifact.Tar great, thank you.

A

Thank you. Yes, okay main no jekyll images, doctrine and okay, so it looks like the main branch is building.

A

Main branches building.

A

So if we go to.

A

Well, this works.

A

Well, let's just actually point the box url about right, um interesting, so the oh, that's! Well, that's fantastic um is alice under here.

A

Now I thought we'd put her in the docks.

A

A

Let's find out, why is the ci not working? I want to know why? um Because I want the docs to work okay, so it's running from maine.

A

We're on line two to seven in dot, ci slash, run dot, sh; um okay, we grab the version.

A

All right go straight: we run the build off main, remove pages to main docs html.

A

Okay and then we built the release docs, okay, so that's the main docs. This.

B

A

A

We built the release docs and we checked out the last release and we do the install and we build the docks.

A

And this is what.

A

This, what happened here? We change.

B

A

I know um 15 days ago. What did I do? I.

B

Know I know it's me.

A

A

Three years ago, well, four months ago, this is probably the culprit. What is this.

A

hmm Let's just look at the history of this file.

A

Let's just see it might just be sweet.

A

um Get log dash b.

A

Let's run okay, so let's see what happened here so we removed the web ui we built with no strict, uh we renamed now prior to the rename it was working. So something went awry in the renamed.

A

Looks fine to me.

A

Why, what's the job say.

A

Almost last time was right.

A

I can't see what you triggered you that's annoying, okay, so we're looking for something that was 10 hours ago. Okay, so 10 hours ago, is probably a scheduled job around 3 a.m. Utc.

A

That's not second party, because it's not the pull request. It's either schedule schedule. Okay, there. It is.

A

So this is a doc's build that says it was successful, uh but we're not getting the main okay. So.

A

Let's see we have.

A

Okay, here we go.

B

A

A

B

Gives you these.

A

Urls that are good for a short amount of time that you can load the logs from um okay. So let's view this.

A

Let's view this side by side with um asci file.

A

All right, okay! So what are we looking for.

A

All right, oh, we should change this.

A

Oh, this is probably screwing up all my commits. I was just wondering what the hell is happening: okay, um yeah, let's just make this house.

A

Okay, that's so much better! Okay, um all right! Okay,.

A

All right so commit here, there's the commit- and there is the commit over here all right. So.

A

Html pages, which is image file, not readable,.

A

And this is within the latest release docs, so is it warning treated as error? Okay, so this is what's happening, so this is actually erroring this command here, okay and we're not catching the return code.

B

A

A

A

A

uh What is it speed info type bill set.

A

Okay upset okay, so option name.

A

Pipe fail so pipe build. The return value pipeline is the status of the last command to exit with non-zero status or zero. If no command exited with a non-zero status. Okay, I fail, I think we can just say, set o pipe fail. I think that's.

A

I always spell it wrong: okay and then we'll do set plus. Oh, I fail so yeah. These should be around this. Oh wait! That's not pipeville! um Why is this not erroring? Oh because or script stocks sh?

A

What does the last release say.

A

Get show um 0.4.0 uh colon dot, slash script stocks sh is that even in the last release.

A

Let's try 3.0.

A

Okay, 4.0, it is.

A

A

Wow we haven't done a release way too long. Okay,.

A

A

Make last release dogs.

A

Okay, it's just two.

A

A

Not f pages index.html.

A

Exit one all right great, so we'll just check if there's no accuracy, if docs filled.

B

A

On two right, so we want to output this to standard error and we'll say dots.

A

Do that here, as well.

A

A

B

I made a mistake.

A

A

A

A

Okay, I know what I did wrong. Okay, uh I should have cherry picked from from maine to alice. Instead, I should have picked from alice to maine, so I think alice is going to be unhappy about the rebase here, um so we're gonna have to um real quick okay um time with it.

A

A

All right, okay, all right: okay, okay,.

A

A

So the main docks are being broken.

A

A

B

A

All right, I'm sure else,.

A

A

um Right so now, let's um get.

B

A

Pick um alice, I know I'm doing it again.

A

A

I think I selected the wrong one: okay,.

A

Okay, now we got them okay, so.

A

Okay, so yeah: let's go ahead.

A

Okay, so also, what's the error there is that it can't find this image, so it's that's going to be a problem with all of them. um So how are we going to? How are we going to get around that? um Well for now,.

A

We'll just pull it from the old repo, um a copy of the old repo okay um because it exists there so we'll pull it from there. um Okay, get grip, docs, all right, we're gonna, look for images, slash it docks; okay! So here's all the images that we stored in here. Okay, let's.

B

Now, look for svgs.

A

A

Those ones we didn't remove um and then let us.

A

A

All right all right, all right, okay, um so let's do get grab dash e. I think I'm gonna try to make a regex for this.

A

Okay, all right, let's see.

A

How would I do this.

A

How do I do not.

A

I already do a direct match.

B

Okay, didn't think so.

A

Okay, I give up all right so.

A

Okay, let's see so this website is great.

A

A

So we want images.

A

Slash right there: okay, so until dot, okay, yeah, we want images until dot.

A

All right, that's perfect! Okay, if we can do this, then give pre image removal.

A

Okay, so this is the backup we can point to these image. Urls.

A

Where is a image where's an image?

A

A

Okay! What's the url? Okay, here's the url okay! So um now we can do sed dash scd dash. I we can do uh yeah, okay, so we'll grab we'll grab that output and we'll do we'll pass that through scd we're going to grab all the files um and.

B

A

All right, we grab all the filenames sort, unique anytime. You need to duplicate it.

A

Right, okay, so now what do we want to do? We want to say sed, I so edit them in place with batch. I and then use this little regex where we're going to say s that means so match this match, match the string.

B

A

Right, I think I have no idea um and then replace whenever you see this images, slash doo, doo, doo, doo doot, whatever right, I want you to replace it with this url um and then we need to go backslash escape right. So basically, do this go over here. Now, right, don't go it do don't do the local copy just go over here, grab this url right and then okay.

A

So actually, let's just do dash e and let's uh pipe that through correct because dash I will edit in place e is just gonna say: do it and show us the output and so then we'll grab for um what was it images yeah? So we'll just do this and we'll see what happens?

A

Okay so that didn't give us any output at all.

A

A

Let's try it this way or file in that.

A

A

The scd thing on the file.

A

And then look for images in the output.

A

Didn't like that.

A

A

Okay, here's that okay, so here we see it at the bottom here: okay, so oh yeah, and it's true because it has no.

A

It has no images, it has no slash at the end right, so all of them have no slash at the end now. So let's try.

A

Slash and ampersand and see what happens? Okay? No! It's not that we'll! Just do the ampersand all right! There we go. Okay, these are the right urls, so we want to go ahead and just say dash. I, and actually, let's check our get diff.

A

We have a clean tree, so let's go ahead and div again right and here's.

B

A

Our edits right, okay, so it looks like some of these had a leading slash. So let's do it check out dash dot.

A

We run our little regex again and actually wait a minute. Did that happen.

A

So, let's do the ones with deleting slash first, so we'll just do that and then we'll do that and then we should get them all.

A

Oh, no, okay, so yeah.

A

Okay, so it's with the slash and.

B

A

A

And that way we won't overwrite the existing ones. So now let's go check the urls okay. So do they look good.

A

Okay, that one's got a dot slash, so we need to take care of those dot slashes.

A

So let's check it out again so reset and let's take care of the ones that are dot slash first, so we match the larp the the longest one first, so backslash dot. We have that and we do it with actually should.

A

A

A

One coin: okay: no, it's the assets.

A

Images and then it's space.

B

A

And then it's space, slash images.

A

A

See what happened? Okay? No! We that one up! um Okay! Now we a bunch of them up.

A

Okay, let's just fix up these ones.

A

Okay and we'll just do where you see underscore space, which is do.

A

A

A

Okay, god, damn okay, so space space, slash.

A

Okay, let's check this out now, all right so.

A

Yeah, it all looks good.

A

All right, they all look like they resolve: okay, wait! Okay, so we've got some problems here with this image colon calling space, that's an easy face.

B

A

Okay, so this one's problem, but that's that's easy problem.

A

Okay, that was this one, so we fixed it now.

A

All right, I think, we're good here.

A

So I think we can push these changes to this branch and, oh, the last release, which we should tag a post. Three, I think, are post two: um okay,.

A

A

So we'll cherry pick this commit.

A

A

A

Related, what's that housekeeping issue.

A

A

A

A

A

Okay, so um let's see.

A

So we're going to push that up.

B

A

Right now, let's go to alice.

A

All right, let's rebase main what happens.

A

This looks like I did the wrong thing.

A

Okay, this is the only discrepancy.

A

A

A

So that's in there.

A

And that's in there.

A

A

A

All right, oh yeah, or we don't know yeah okay,.

A

Okay, um re-raised main into alice branch, revo urls, I don't know yeah, I don't know when I stopped talking um so yeah. I just went in here and I.

A

A

Yeah I didn't like that.

A

Yeah I, like non-linear trees,.

A

But we'll get there: okay, um there's no reason it has to be linear.

A

Over time, all right, okay, so here is man into alice branch.

A

Okay, so all right now, I have to do it. Okay, let's just wait for this yeah, okay, so which is fine because I'm just going for it. So let's get. Let's just stop all this nonsense, so um we just want main just right. Man use the runners for me and see if we get the uh doc set back, that's what we care about wow. We have way too many tests. um Well, we don't have too many tests too many tests we don't have to.

A

We have a great amount of tests and we want more, which is why we have to split out everything in the second part.

A

And it's just too much testing right now.

A

And it's supposed to be structured as a polyrepo setup, anyways, so now's the time.

A

B

So we did all that.

A

Great did some regex great uh okay, so hopefully we end up with a doc site that works um and also, and we'll just note that the um the main was working so main and.

A

Okay and then what we'll do is, I think, we'll just try to build alice okay, so we should go over the main right now. I have to just get checked out.

A

A

All right we'll say this.

A

Is all right, so this is the this. Is the trust boundary the domain? This is the user within the trust, boundary.

A

A

All right, let's just do it: okay,.

A

The same thing so we'll just do this.

B

A

Latest release: docs; okay, so.

A

Dallas docks.

A

All right so we'll say alice.

A

We only need to clone it at depth, one actually yeah. We only need client default.

A

Okay and we'll call this house repo.

A

A

All right clinton that directory cross branch, repo.

A

And then build the docs to.

A

A

Can we make a github job, worn.

A

A

Okay, we want to know about the things we can print. um What are the things that set output.

A

What is this okay, so set output.

B

A

A

A

A

Outputs good job, outputs.

A

Shouldn't there be an example of.

A

Where could we see? Where can we find set up right in here.

A

This is not working okay. How do I search the github docs.

A

A

What about other things that are like set output.

A

Okay, here this is fun. This is fun. I like this.

A

So we want to also build the box for hours.

A

Okay, um ipad notice in group extra variable is debug, summary say straight second: set failed, set up a start group warning warning what is warning core dot warning? Okay,.

A

A

Okay, so set output corresponds to this core dot warning okay. So where are these core things? Core actions toolkit.

A

Set output: let's see if we can find that in here set output. Docs commands okay. So let's do okay. So this is what we're looking at here.

A

Actually, I'm sure take us to this.

B

A

A

A

Warning my warning message.

B

A

Okay, that's what we needed to know.

A

A

Oh and we need to, um we need to.

A

We need to make sure that alice's docs envy pages release docs.html and the main dot html mv stocks, html.

A

A

A

See I run dark now. I know um ci talks. Okay, should we see I run box? Okay, um see I run docs okay, um build alex.

A

A

Man and then okay, now no more out.

A

Nope, previously we're not releasing in maine until we.

A

A

Hopefully this goes well we'll see them right. So now.

A

I ended up just cleaning up the docks. Okay, all right, okay. So now, let's actually let me let me let me actually just rename this.

A

A