DFFML Rolling Alice: Engineering Logs: Volume 1: Coach Alice, 28 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Coach Alice: Volume 1: Part 43:

Description

https://github.com/intel/dffml/discussions/1406#discussioncomment-3268573

A

Into the matrix, all right, so where are we at? We are with the lovely alaska, repo, okay, so, and we are in gather inputs. Remember.

A

When we come across, contributed readmemd, so get rid of it, because we want to figure out. Why is contribute readmemd not being called so you know what we should do.

A

Can we do async within this, probably not? Unfortunately, that's really annoying.

A

A

A

I so if we import an estate sync file and do nested async io apply, then we can do this.

A

Okay, um so now we can do async dev.

A

uh Waffles just do this and get ruffles ruffles. Here's you you're good.

A

I don't know how to spell rockets all right, um yeah, okay, so this is not going to fly. It doesn't give us a function. Okay um doesn't want to. Let us define one on demand, so in that case I guess it's just fine. Okay, so.

A

A

Okay, so ct expired.

A

Does it have any conditions.

A

No, it doesn't have any conditions, so we should never bail out here, but we should hit it here.

A

There it is, we should hit it three times. Remember.

A

Okay, there must be three cycles in this whole thing, so, um okay,.

A

So now we're here.

A

Let's return value.

A

A

So let's just go here.

A

A

Let's find out where we are.

A

Return so we're returning. Okay, okay, return, return; okay, yeah, because we didn't find the input so we're returning. Okay, so um so gather input, name.

A

Okay, so let's go to the third one right, so this is the third time through uh gather inputs right because the orchestra does this permutation thing every time, there's new inputs, so anytime there's a set of them that come in it grabs the set, and then it you know does this set? It does the batch it's like a batch job dispatch um and based on whatever the batch of new inputs is okay, so.

A

Okay gather, let's gather okay, so repo base operation, let's see, what's operation, dot inputs, okay, so operation, dot inputs is repo and base and commit message. So there's no base okay, so um you know we should be making a.

A

What is this I'm still very pleased with this okay? um This is today. 28 today is 28.

A

What am I cooking something?

A

Oh, I am cooking. It's the rice, okay,.

A

A

A

So this was some good debug here. I think on this.

A

Like really unnecessary, I always end up doing this. It's really unnecessary, but um so, where is base coming from so let's go back and check this, so base base should be coming from determined base branch right. So what what's? What's up with that return on base branch? Okay, so.

A

A

When, in doubt,.

A

A

Okay, so config input notification set so yeah. That's that that's that batched q, there so and then ctx hd. So what's down here, so this looks like some inputs in the network. Perhaps so.

A

Ctxhd, okay, so cpxhd.

A

What is this organized by.

A

The string input set context. Oh this must be the string input set context or this subflow that the the two. So if we have, if we consider the top level flow to be the cli.

A

Then this sub flow first subflow that kicks off is the ascii repo stuff, and this is and n equals two like that. The level equals two all right, so there's cli zero or wait. No, no yeah, so cli is zero top level and then what's one down from that one down from that is alice, k, repo and one down from that is.

A

This yeah this thing where we're doing this custom context: okay, okay,.

B

A

I didn't get enough sleep, obviously so.

A

A

Ctx hd dot ease.

A

Okay, so this basically tells us that we are the only context running.

A

Okay, so this is the only thing in there.

A

ah Give me our proper up arrow thing: okay, that's really neat! Okay, so.

A

Okay, so this is.

A

Okay, so we want to create a list, because you can't just select.

A

Okay, so now here's our inputs inputs, dot.

A

A

A

So you print it because this is not very nice display, okay, so that is much nicer. Okay,.

A

Okay, so what do we got here so read me: give repo! Let me share with you here: buddy, okay, so inputs.definitions. Where did I get this from okay? I just grabbed another ctx hd, okay, so and we're wondering why is there no base branch right? What was it so?

A

A

A

Gather all right so.

A

Let's put all these together, so this is what's going on here, and this is what we'd like to understand a little bit more about is: why is there no base branch right.

A

Attempting to figure out why an operation is not being involved.

A

A

A

Okay, so what is this this is.

A

Okay, so contribute.

A

Reading be should.

A

Be getting base, but it's not.

A

A

Okay, so the looks like okay, so here's this, so I see this. This looks very much like. We really need this value to be record that I really don't like the fact that that's not rapper. Okay, um that's really unclear.

A

A

So there is a repo base branch, so why is it not being considered? So what is the origin required so.

A

Okay, so it was discarded for some reason right so.

A

Suspect discarded because of miss match definition or mismatch. Origin.

A

If not that well, let's check definitely.

A

A

Let's get to it so.

A

So here we do yeah the input flow check so.

A

A

This is it's always something super dumb when something like this, it takes forever.

A

I um hopefully right so unless it's a big problem.

A

Okay, so origin and origins. So let's take a look, so it looks like the exact same strings to me here. So let's figure.

A

Okay origins check out.

A

So now we're going to go and suspect something else.

A

Okay, so alternate definitions, perhaps.

A

Let's see if we break there and let's see if we break there,.

A

Okay, we do break here, okay, so it didn't continue on any of these. So it says you are something that I'm going to compare myself to. Okay, so, okay,.

A

I know this is a returning.

A

Okay, we're gonna get rid of this.

A

So it is perhaps this alternate origin thing. It must be right now, where's this.

A

Origin: okay, so now.

B

A

In the right spot, okay, so.

A

Input source input, source list, okay, so input source is not a list, it's a top where it's a dict. So let's jump into that case.

A

Okay. Now there is no case for that. So it's going to get added to.

A

A

Okay, so what is the input that we're looking at right? Now? Item get repost back, okay,.

A

Item your reboost back, okay, so then yeah, basically, okay, so.

A

And item dot definition, dot, spec dot, consonant class call name, not equal.

A

All right so we're just this is just our quick and dirty debug here.

A

A

Item read me to repo okay, so this is.

A

All right, okay, so now we're not hitting it. So we want to look at base.

A

Let's see if we hit this for bass, okay, so that was not or something that would happen here.

A

A

Okay, where are we.

A

A

All right, so we didn't get here. So let's try to move this up and let's resuspect the origin issue, all right. Okay, so here we are so yeah. So maybe it's an origin issue, so origin seed, origin, seed so by origin, there's by origin by origin, is here so none. So it's not coming from seed, so we're looking for base and we want base to come from seed. So why is it that you want base to come from seed?

A

What would have given you that idea.

A

Why do you think bass? Oh I'm sure you read me md, okay, because we've isolated this flow.

A

Because we sandbox wow, of course, because we did this to ourselves. Of course. Oh all right, okay, you can tell how surprising so.

A

Already overlay get boom.

B

A

Is there's our culprit yeah so.

A

We didn't include that definition because it was not.

A

Why didn't? We include that definition so run custom, there's our run custom code at so first off. We know that we've done this readme overlay right, so we didn't include the other flow, so it better have already been this input better already exists if we're going to interact with this thing at this point, um so actually we should do the forward. We should do to do okay, so to do to do you do optional, a support forward, etcetera.

A

Okay, I'm gonna write this in the other notes, because there's no way.

A

A

Okay, so this is really exciting. Okay, so.

A

Okay, so I thank god- this is the problem. Hopefully so, okay,.

A

A

Okay, a really good face branch- all right, so where is this thing being created here? Determine base branch?

A

Okay? So basically it's saying I think you're coming from seed, because I don't think you're in my flow.

A

Right because it would have auto floated, but it doesn't think it's in its flow, but it is in its flow okay. So.

A

We're not adding them. Are we adding them directly yeah? We are we're, adding them with their origins, so so where's the addition happening.

A

Okay, so here so this check here.

A

So this here from 239 to 251 in our workman and community standards right now, um we are looking at the the really really grotesque stuff we're doing with run custom, um but we're prototyping the generic form. So what we have here on line 244 through 249 is.

A

A

The check to see if we should add the item from the parent flow to the seed values of the data flow in the child. um Now I don't know if we copied that honestly, quite honestly, I'm a little bit worried about this so or no okay. So we we did this, so the other one, I'm a little bit worried about whatever okay. So um actually, no, that's bad. Okay, so go click. Do want.

A

A

To do copy that deep copy.

A

Okay, I think I'm gonna.

A

A

And move the question mark: okay,.

A

Okay, so now here this is gonna go and we're gonna. You know where this basically says you know: overlay, readme, okay, so then overly reading.

A

Okay, so what we would do is we would say to this overlay.

B

A

Yeah, we need to tell it to load this.

A

Let's just do that.

A

Should we do this.

A

Is that should we do that.

A

So I just added it here.

A

So I added it to entry point cxt. I did.

A

Okay, so here we'll see, there's a overlay.load right, so we're doing this overlay readme. So this overlay readme dataflow this one that we just talked about having the other one. We had to do the deep copy, and here we're looking through the values and right now, there's nothing else, but I think I mentioned this last night, but you know this is why so, basically we then say: hey.

A

We expose us this entry point which we load off of this is what the default you know installed: flows the overlays that default overlay stuff that we were working on a long time ago, um and then we were working on kind of recently a little bit more. um You know this. It's going to do this type of thing by default, um but right now we don't we're just doing the static apply right. So I think that.

A

I think that if we do this, then it will load the other flow. So it basically says hey if you are running overlay, readme also try to load overlay kit and we can optionally do that based on dependency selection. I believe so, let's see if that makes it happen.

A

A

Gather base input, name, base, origin matches.

A

Okay, so a lot.

B

A

Stuff is happening, so I think we need to remove these brakes.

A

This is exciting.

A

See what happens.

A

I think we might have caused my loop.

A

Man, this thing has been just the best just saving all the output wow. That is nice.

A

Did I post that somewhere.

A

A

Okay, so let's find out what went wrong.

A

Okay check it out check if valid get repository. Url is happening multiple times, so I think that there is.

A

A

A

Okay, so where were we oh yeah, we had an infinite loop, of course, um who doesn't love anything all right, all right so and again and again again,.

A

Hitting the buttons right, so let's do.

A

All right, where are we at and what are we adding here so.

A

A

Let's see okay, so this is what is going to get passed to this flow right, so these are going to be the inputs that it has so repostering guest get url, url, okay, so.

A

Excuse me, you know, maybe we should reuse the redundancy checker.

A

So we have separate inputs but reused, redundancy checker.

A

That should work.

A

A

Write this down.

B

A

Okay, so uh we were missing.

A

This is the wrong thing.

A

Okay, so found.

B

A

That it was the soft origin seed versus.

A

Comes from overlay, get uh registered overlay, get as an overlay of overlay readme right so that its definitions get loaded.

A

A

This way, autoflow will.

A

um This way, autoflow will.

A

Make the input the expected origin, the output from overlay, git operations.

A

Rather than seed okay, uh which is the default when no matching outputs I've seen on inspection on beautiful load,.

A

Button: okay, um he's doing that.

A

Are we on the internet, we're on the internet? Okay, okay, all right so, um but we then found we found it created an infinite loop.

A

A

New uh we'll try reusing, we don't see.

A

Okay, different operations.

A

A

A

Okay, so we need the ability to pass so parent.

A

Can we just add that or something um subflow octx we had that already fantastic.

A

Oh that's how it's not looking forward to that all right. Great.

B

B

A

A

Come on we're close redundancy checker, so why did we say redundancy trigger so it seems to be causing an infinite loop. Probably, why would the redundancy checker help that.

A

Well, because it's not doing it in the parent flow, so clearly it must be doing. It must have the redundancy correct check, correct there. So octx run.

A

I always check the memory actually, because bass is, I think, out of date, with memory is really the gold standard for what we consider to be like the api on the orchestrator and so forth.

A

A

Run a data flow, that's what we're looking for! Okay, so.

A

A

Is it on subflow.

A

A

A

So here we need to take reuse.

A

All right so and then we'll pass reuse down here.

A

Okay, so now we can call subflow with reuse.

A

Okay, one custom.

A

Okay, so so club reuse.

A

A

A

All right so now we can pass reuse to run custom.

A

But we have to pass it big, so here's the config so reuse equals rccx.

A

A

A

A

And we have doubled our inputs.

A

So alice can truly read me so.

A

Why are we back here.

A

Why are we back? We can't forget to set the lock.

A

A

A

Let's not reuse and see what happens.

A

A

Is it maxing out there or what's happening no far from it, so this is only getting more and more and more.

A

A

So we got somewhere eventually, but we also kept calling this up the same thing over and over and over again repeatedly. So we were so basically we did get into the flow, but we also kept doing the same over and over and over again. So oh, you know why it is because we're passing in operations that go to overlay get so we want to make sure. So this is the upstream.

A

A

Okay, so repeating flow overlay.

A

Remote data flow upstream.

A

All right so read me: dataflow equals upstream overlays.

A

Okay, that looks correct.

A

All right, so that should do the same thing.

A

Read me data, so then we only want to add if.

A

If only if it's in the upstream definition, values.

A

A

A

Okay, so read me therefore, upstream.

A

Okay, so now, basically yeah, so what it says is only add the things that this flow cares about right. So if it's in this flow's definition set, then add it right so add add the other flow to do the do the auto flow with, but do only add inputs from this. This flow so.

A

All right, okay, so auto blow auto flow.

A

uh With overlays only.

A

Add slash forward inputs to this flow.

A

A

I'm okay going to take this off and see what happens.

A

A

So it made a commit now.

A

Contribute reading and date, okay, great.

A

Why did you change that.

A

A

A

Did we not run black on that earlier.

A

A

A

A

Okay, so my guess is: we have more of the same types of problems, it's not a problem.

A

I just didn't mean to do. Sam, okay,.

A

A

A

B

So uh what did I just do.

A

There I just wiped that last comment away, because I didn't like that all right, so all right, okay, so all right all right.

B

A

The cli okay, let's do this: let's do this.

A

All right there we go now we got what we wanted right so that just that you copy okay. So now what else we got so this if multi-face.

A

All right, so this is what enable git commit.

A

Sm vf base uh open, ctx.

A

Allow subflow allow or passing views.

A

Context, uh orchestrator.

A

That's subcontext for reuse.

A

Okay, now, what have we got? Okay,.

A

A

All right, so we dip the file, we add the file, we commit the file operation, data flow run. Okay, so.

A

A

Straighter sub contacts to config to operation, big.

A

All right so now all this all right so.

A

So we don't want this stuff. We just want the stuff that we did.

A

All right does the rest. Look like formatting. Yes, it does all right.

A

Okay, so and we'll say this is big- commit.

A

All right, so I.

A

A

Okay uh over there read me.

A

Ensure overlay or reuse, reuse, rctx, redundancy, checker context and overlay gate.

A

A

I don't understand how this changed.

A

Keep those foreign.

A

A

A

So, what's the problem now.

A

Okay, so that was just like a random graphql there. It looks like.

A

Issue, this is looking nice, okay,.

B

B

A

A

A

Okay, so now we want to figure out why.

A

Oh, I think I gotta take a break for lunch and then we'll try to figure out. Why the what is it, I'm trying to figure out why.

A

Let's see what so just we can actually put it on one page now, so no get branch given it's still not so there's something wrong with the diagram command, because I think that is coming through so readme pr still is missing a few things so.

A

Let's go check it out.

A

Okay, so overlink it. So what do we know? We know we recently had this issue with over that kit, so it looks like.

A

The rest of what we have we need it's a writable giveaway. I remember seeing both those. So, let's see grooming branch is that being produced.

A

So leave me a message with my video title. Contribute read me.

A

The truth read me: outputs, none.

A

Now I do log in why.

A

A

Definitions were seen earlier.

A

On so start to be, present must be something else.

A

The logs tell us that almost contributing.

A

A

A

Which means that the downstream operation is not called since.

B

A

A

Okay, so let's go figure out why it's returning none! So where might it return? None.

A

Nowhere, why is it returning none? How could it possibly have returned now.

A

How could it have possibly returned one.

A

B

A

So failed to create branch for contribution, so it's trying to raise in run time error. Why are we not seeing the run timer failed to create branch for contribution wtf?

A

A

A

So does this not complete if event is.

A

Is it not complete.

B

B

A

That was my other theory was.

A

It's being cut off somehow, because it's returning and there's a subcontext there should be only one context per.

A

Because they're nesting, okay, so.

A

Okay, I think I'm gonna make lunch.