DFFML Rolling Alice: Engineering Logs: Volume 1: Coach Alice, 20 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Coach Alice: Volume 1: Chapter 1: Part 13: Alice, Please Contribute Recommended Community Standards

Description

https://github.com/intel/dffml/discussions/1369#discussioncomment-2965009

A

A

Where are we at where we are we're on the internet live all right, fantastic.

B

Boom start an ssh ssh.

A

A

And we've got okay, so.

A

All right we're going to jump right in.

A

All right, so this is what we're doing here right, so we're shooting to implement this command uh alice. Please contribute recommended community standards and to the project right eventually uh we'll make that into some. Let's see, let's see, maybe.

A

Oh wait: this is not.

A

So here we are: what branch are we on? So we do get status all right, so we're on origin manifest all right, so dude do it get diff? What were we doing here? Okay, the locking right. So this is what we were messing with last time, all right, yeah. I gave up on this okay. um You know I liked this code as debug code. Let's connect this and we can throw it out later um so df memory added.

A

A

A

Print operation on lock acquisition.

A

All right so, and that was obviously not helpful, because the thing is still there, so we need to go bisect it and figure out when that issue was reduced, introduce and and uh if, if it was um introduced by um if it was introduced by something we did recently within the manifest branch, uh we need to then just undo that right just fix whatever we did.

A

If it's a lingering issue which has been around for a while, then we're going to go back and grab the metrics implementation and and the locking that was working there in the initial implementation of you know. So all right. Okay, let's push that, but we'll do that later.

B

All right, let's see where we go, we had some comments here.

A

All right, great, okay, so now that the work is saved, let's push together, okay, so good status. What else do we have in here? That's not saved all right, nothing, interesting, okay! So.

B

Let's see so yeah, so let's put this in the terminal, so we don't have to like that.

A

And then we can use.

B

A

Docs and where's this going to go, write, docs, tutorials.

A

Docs tutorials alice.

B

And this would be joss and.

B

Directory, it would be.

A

A

Open source guide, okay, yeah.

B

A

All right, so we make the directory for it, and then we create the file within the directory right. So we cat we pipe can. Can you see that okay, yeah okay, um so we cat, so we we use cat and- and that basically just says whatever I send to your standard.

C

A

We pipe it to a file with bash and we say: okay pipe it over to this file right. So then we paste, uh let's right, click with putty, and now we can do this. uh Was that rich markdown? Yes of this, this python library rich? We can use this uh okay yeah! It's an rs2 file, whatever it's going to interpret it uh it'll be fun so now, he'll just dump the mark.

B

Down for us in a nice little console format: okay, great! Thank you for this.

A

Okay, great so now we've got this in our terminal for us all right great um now we don't have to switch around okay. So what are we gonna start with here? So let's go grab.

A

Let's see entities alice, alice, cli right, so, let's jump into here. Let's do let's run an alice command, so let's do alice help just to see where she's at. I can't remember if there's any tracebacks in here all right, so we have should I and threats all right. So we want please. We want alice, please, um because we're gonna ask I'll skip things nicely.

A

We want alice to do something. Please do something for ourselves, um so.

A

See line all right, so we just followed the pattern. What did we do? Wait? A minute produce? Where is.

B

This coming from now, let's produce cli, that's fun.

A

Oh cli, okay, well, that codes dead code all right. Well, we're not gonna mess with that right now, um all right, so we're subclassing from we're creating a new cli command class. When we do that it, uh you know it basically ends up being another. One of these uh commands over here right so alice should I is this is alice. This is, should I should I go so should I command? Should I command? Is, I think, imported or no? This is, should I cli is right here, so that one has two commands as well.

A

So should I and then we have dash h and then you'll see, contribute, reuse and use right and so those map to those definitions within the class, and then you point it at you know whatever thing you wanted to do right, and so, if one of these implements, if one of these classes implements a a method named run, then it will run run it's actually up front.

A

um Then it will run the cli command, um and you know we just that's where we put the code for the cli command. So that's just what we're going to do right now. So um only we're not going to do it right in there we're going to create a few more of these, because we need um please contribute right, and that is actually all we need. So we just need alice bits contribute.

A

um So we have alice, please cli and then we have alice is contribute all right, so we say, contribute equals this thing, and now here we can write, contribute so over here. We'll do alice, please contribute and we'll say help. I would save this file yet.

A

Okay, now I save the file so now we'll say alice, please contribute, uh and it just gives us the help message. Okay, great, and what if we say, if we just run it, then it says feed paste, okay, great, and so that's what we over here. We did return feedbacks.

A

So now now we see that we're executing the code that we're writing. So, let's see so we need. What do we want for this? So alice, please contribute so alice. Please contribute! Is this fundamentally about repos?

A

Well, let's, please contribute we're running you from the cli. So when we're running alice from the cli, we assume, based on the context that this is about a git repo right. So let's note that assumption so note when running alice from the cli.

A

We will- uh or maybe this is a to do to do right so when we get that top level system context in there that we were struggling to get in there with the cli command unification that we attempted with the system context as a class once that's ironed out, then, when we're doing this, we should uh facilitate this.

A

You know reach into the top level system context and and and for what we're doing here, and so when this is being called when this, what what is now a class becomes a system context in a data flow uh at that point, um we'll want to implement the mechanisms to infer right. So basically, this in future work down here in fur context, cwd right. So if I, if, if if alice says, uh where am I being executed from- and she says, what's my top level system context when she runs so she runs.

A

This data flow, this data flow, which just returns feedback right. Viewing that return statement as if it's a single operation data flow right, so she's going to um she's going to say well now this is about to do something more interesting with github and right, um and so how does she know that? That's what it should do over? How does please contribute not mean? Maybe you know, give somebody a donation or something right. um You know it depends on the context.

A

So if we are in our terminal within a git repo and we run the data flow which is alice, please contribute we're going to say: okay, we're going to detect right, and so we're going to have many overlays here and we're going to have an overlay onto the default flow, which is just this alice. Please contribute which is this stub right, and then we have overlays for different environments right.

A

So when we say alice, please contribute in within the context of speaking to her, if like if it will means depending where you are and where you are, and what you're doing it's going to be different than uh when you say from the terminal alice. Please contribute right because she says okay, now you're in I'm in the shell right, so okay, so we will inspect the top level system context.

A

When running house from the cli, we will inspect the top level system context in the future applied overlay, which is the uh ls. Please contribute overlay, which provides cli functionality.

A

Cli applications right so anytime, okay, so.

A

It should auto populate the revo's uh config.

A

Let's see input required to the data flow to the base data flow, this repo.

B

A

I think that's sufficient, um along with the recording, so all right. So what are we gonna do so we want that blob. um So, let's go ahead and play with how we would get that blob right. So read: docs portals.

B

A

So here's our content right, so this is what we want to generate. So, let's think about this right. Let's take the data flow approach to thinking about this.

A

This is the list of files we're going to return to our threats md in a moment.

A

So this is what we want. This is the end result that we're looking for right. So let's do this like test driven development style um and, let's over here start our nodemon right, which watches for file changes um and.

A

Let's run the command which we desire to be the correct command right, so we just write this over here, that's one and so now so so. Why do we do that? We do the clear so it clears the screen. um We do whatever command we wanted to run and then we want. uh You know we just do test one at the end, because or else bash will complain if the thing blows up um so or note, one will complain right. So now we're running feed phase. Okay.

A

So let's print this just watch, it change save the file right. How am I saving the file there? It is, it watches change all right. I do escape I'm in bim, I do escape.

A

I do colon w and then I hit enter um okay, so there we go so every time we save the pilot changes. So we want this thing so should be. um Content should be.

A

A

All right so and I'm going to use text, wrap dot d dent so that we just get a little cleaner view of that.

B

And I'm going to call strip to.

A

Call I'm going to call left strip um to remove this uh first new line here. That's going to show up.

A

All right so now we've got our nice little clean thing that says content should be right and then we say content was right. So let's now do the same, for content was, and so we say okay. So so it's going to be the same right now and we'll say, unit test dot, test case.

B

A

Equal and we'll say, content should be, content was right and then we'll begin the process of actually generating the content in between.

B

Okay, so we should do that: okay,.

A

So we import the modules that we need. We need tech, strap and unit test and okay, so return feed face great well that okay, blank fantastic. So now, let's remove one of the lines and we should see it blow up all right. So great. It says assertion error. This string does not equal this string and it gives us a little minus showing that uh you know the license was not found.

A

That line was not found right so now we know that our check is working and that uh you know, uh as we start to go about developing our our our our data flow, which implements that the population of these uh fields uh will know that uh it's you know the check is, is valid so and then down here we'll say to do implement uh post to our creation of issues. Now that we, once we have body text generation working.

A

Okay, so now here we go so, let's take a we're going to. You know, take a data flow approach to this and we're just going to do it a very simple implementation and we're going to get that supplement implementation or, let's see yeah so read. Where is that, um I think it's in operations, inner source from all operations inner source um is in cli.

A

Yes, it is in cli, okay, so right now it was in it was in the should.

B

I stuff uh yeah.

A

Okay, all right here, you know we'll just write functions. First, okay, so basically we say uh has readme um and it takes a repo object. And then we say you know, return path, live so path. Lib is a library right, so we have to say import path, lib, so we say return path, lib.path and then repo dot directory.

A

We pass these in each as arguments, so we say the first thing so so we're creating this path, lift up, half object and.

B

Hi dog: um let's see pathway.

A

So this is the documentation for the path lib module. um Let's see where's the top of it okay yeah. So this is the documentation for the path module. So it has these classes here, and uh so one of them is this path right. So this is the class that we're we're about to instantiate, and we we do it. We say it has a star args. um So basically, you know what that means. Is that we're just going to pass in? You know as many elements to the path as we need right.

A

So, what's the first element of the path it's the directory, so this is going to end up looking like if you had a bash variable, this would create the same thing right so repo divectory, slash, readme, dot, md.

A

Right, so that's basically the equivalent there, um and so, but this doing it with this path. Lib module allows us to have cross-platform portability between windows, linux and mac, who handle or who handle paths differently, really just windows and and linux and mac across the windows divide there. So uh if this way we can say has readme and then we just say exists right and so, uh according to the path lib module.

A

It has this exists function and then it tells you whether the path exists right and it returns a boolean value. So if we just call that function right here, then it'll it'll tell us if it exists or not right. So we just do this for all of our files right.

A

So now we have five files right and then we can go through and we can check each one right um and and we'll get more advanced. But we'll start out with this right, so conduct.

A

A

A

B

A

um Okay, so and then these files.

A

Right so it's just just you know, rinse and repeat on these right. So yes, we can use more sophisticated methods and loops and things, but this is just what we're going to do right now, because this is going to help us.

A

You know, take the take the most straightforward approach right. Do these files exist right? um Just just do the straightforward approach, first, right! um Okay, so we have these right, so content should be. We want to run these functions and we want to get uh you know. Content content was um okay, and so how are we going to do that? We're going to do async for.

A

Ctx comma results in run and, let's see we're already india, but we need to input okay, so in dfml.run and then we're going to give it a data flow right. So we're going to say op amps in because this is going to give us all our operation, implementations.

A

Okay, I'm going to try locals, um but I haven't tried this before so we'll see what happens so. Basically, this is going to grab all the local variables where this function is at right, so set new, so we're on line 105 right now and so uh locals is going to print out and we'll we'll do it here. Actually so.

B

From pretty prince import video print and then we'll do this, so we'll just print this first that way, we can all see it. Okay, so.

A

A

All right, so this is this: the local variables at this point right so here's has contributing, has license, has read me right, and so it's a it's a dictionary object right. So um I wonder if it'll support this, let's see: okay, um okay, so where's this offense, and so uh I don't remember uh where how to call this file, so I'm just going to do uh get rep um down here.

A

I'm doing get grab dash c op-amps in and we'll just see, oops so dash c needs oh yeah, so we want four lines of context on either side. Okay, so that's not the name of the function, then: okay, op-amp in okay. So we pass this function. So this function here which comes from dfmdfbase if we pass it.

A

Sysmodule's name, so if you pass it something which is an object, it will give you back all these all these operation implementations that exist within there. um So, basically anything that has been wrapped with this at bfml, dot, op, and, and so why is that? Why is that something that we uh care about? um Well, it's because uh when you wrap it with the fml off uh it says: okay do uh basically take this. What is a function that is python right and give me some information about how I might call it?

A

So it's not necessarily we're we're moving from this model where we're saying okay, I call this function this way. You know this is how I write my code um to more of a way where the call is is like a suggestion right. So um so that way, then alice can based on the intent knit together the functions in whatever ways is most suitable to achieve her in goals right for whatever job she's fulfilling, um so we're going to annotate them with these these decorators.

A

So we say what is the inputs and what is the outputs so that she knows you know how should she call this thing, because this is what allows us to just say: hey use the whole lot of these, rather than just tell her specifically how how we want her to do it so, okay, so let's do inputs repo, and so this is basically this concept of this open architecture, which is this uh proxy format for domain specific representations of architecture, and so uh the the functions are.

A

Python are a domain specific representation of architecture and we're going to use them as they are, but we need to know how they, how to use them, and so we're adding in this this, what is effectively metadata right um and then the execution gets based off the metadata by linking it together, based off intent, is the goal down the road?

A

Okay, so repo is a um givemo feature.

A

It um dot features, dot, um dot, definitions, dot, get repository.

A

That's part: that's probably wrong. All right, um read featured.

B

A

So then, we'll import that.

A

Okay uh and then the outputs, so this will just create a new so we're in the process of of switching things over between a more user-friendly way in a less user-friendly way. So we have not a very. This is not the most user-friendly way to do this. The most user-friendly way to do this is purely with type hints, which are basically to say here.

A

This is the repo it's a type uh git. Where was that file? uh It's a type get repository spec. uh This is a new thing that they've added in python recently, and it returns a new type, which is you know. Repo has read me.

A

Which is right, so this is a much more basically. This is what we're shooting for here um and this because this would be very, this would describe the problem using um using. You know, just just the python type system, which is what we want right, and so then we would be off to be able to operate off purely the python type system, and so that way you know you wouldn't have to do anything to write a data flow based code versus not data flow based code.

A

All you're doing is writing standard python code and you you end up with you know code that you can run in your data flows so and it has the type checking and all those guarantees that that you want um so all right, but for now we don't have that. So uh I have to finish that stuff first, but.

A

A

Okay, are we on youtube still? Okay, we are good.

A

All right, so I think we do have the um new type support in here.

B

Is that still clicking on my voice.

A

Okay, fantastic, fantastic, okay, so then the one trick is once again, you know we have to encapsulate these uh in a dictionary, because that's how the implementation works right now,.

A

But I think, with moving to returning.

A

Actual input objects from the operation implementation context. This is going to be easier to to make this wrapping theme.

A

If anybody wants to look at the op code.

A

Okay sounds better: okay,.

A

This is also just going to make the result, because that's what it's going to be.

A

Okay, that's not the bug, but.

B

And we'll just.

B

Okay, so we have a missing.

A

Definitions, I probably spoke on.

B

What it imported explicitly definition.

A

If my future get future.

A

A

Okay, so once again, we're going to have to just verbose form these as code of conduct.

A

It's so much nicer without the headset.

B

B

A

I'm sure there's something obvious there that I'm not seeing on how to make this an easier transition.

B

A

All right so we're just going easy. Goes it copy pasta, I'm just gonna figure it out.

A

All right, what do we got list object has no attribute export, no retry run and dispatch; okay, let's throw on logging. Somehow we got into a data flow.

A

So we just add dash log debug, all right. There we go. There's our locks.

A

All right, where the hell where, where the hell are we all right? So what do we got? Okay, so we're applying overlays? What who came in and asked for a data flow.

B

A

Line 152., we have to say.

B

B

So you did okay.

A

Oh because this is not available.

B

A

Okay, initializing data flow object. Blank.

B

A

All right, so what are we doing here.

A

We're gonna put it all in one right, so.

A

So repo so repos, so we have a list of repos that we're about to give on the command line right. So we need something to say what is this right? So um basically uh wait or what do I know what kind of repo object this is um or okay, maybe let's just make something that makes a recall, object all right.

B

A

Now we're just going to make the object.

A

It doesn't have to be.

A

Inputs, okay, so what is this input so.

C

A

I want to make the type engine work.

A

Doesn't it work.

B

I have that on here. I didn't push that.

A

Let's see um this is just so ugly.

B

All right, let's.

A

Just make it work first, all right just make it work. First,.

C

Make it work first.

B

Just do this just forget about that.

A

About this, just make the get repost back.

A

A

All right, let's see what happens all right so.

A

All right so repo for refunds out there have.

A

Field, okay, so repose to attribute.

A

All right inputs.

A

Come up that input.

A

Get repost back not find okay,.

C

Let me just grab it and put it in front of that all right.

A

Oh because it's not a list of strings.

B

B

A

No wait: we're not.

A

Okay, all right so now what happened.

A

Seating data flow with input said I recommended community standards also got stuck in there.

A

A

Let's see if there's any operational implementations being detected there.

A

Okay looks like none okay. So how do we? Why are you not detecting the operational implementation so.

A

Swear simple name space.

A

B

There's a little trick.

A

All right there we go so there's a little trick to my helper, so uh I have simple namespace. So when did they introduce this.

A

All right, great fantastic, all right, we can use it okay, so I think what we really need on this, but honestly, is that.

A

That should work.

C

B

Okay: okay, here are all iterable inspective numbers, okay, so.

A

So here on line five, seven, nine we're doing it in speculators to check so if to check, if not, is if.

A

So basically, if it's not addict, then throw it to to check if it is addict, then run it through this simple namespace conversion right to make it into an object and then.

A

Yeah and then you'll be fine, so that's what we just did on the other one. Okay, um are we importing types now.

A

Okay, so now we're good we'll get there all right, okay! So now, let's run these right now they ran okay. So, let's see so has read me false as security, false inputs, repo directory recommended community standards directory all right. So let's do repos okay. So let's actually split these out. That's not a repose!

A

This is a repose right. Okay,.

A

So, let's see what is our data flow? Okay, so it looks like we're still executing two contacts.

A

Why are you executing another contact system.

A

Or repo and self-taught.

A

Requests we gave you the dash dash. Why are you getting to.

B

B

This, how is she kidding.

A

All right locals there: how is she getting that.

A

Why are you seeing that repo's less stirred.

A

Why why are you putting that in there what happened to dash dash? Don't you respect dash.

B

Dash, why is she doing that.

A

All right, let's get to the bottom of this soft repos. Why are you in there setting repos.

A

All right, the problem must commit what we have there.

A

A

All right, so it looks like there was a formatting thing that we missed.

B

So we have to do that first.

A

All right so good, commit or save save, where we're at that we know df base overlay or what is this.

A

A

A

A

And I guess that's the function that we were in um okay, so build simple, namespace.

A

When passed, when, given.

A

Okay, so now we have to go into the electron code.

A

A

A

Sitting repose.

A

B

Run it without screen.

A

Setting repository.

A

Field naming kwx and not so I'm on 181 the field naming kit of lyrics and not have that yourself field field name setting blank okay. So it's setting repos to be this value so, but why is it doing that so, okay, we work field name. So why is kwrx field name.

B

Repos, in this case,.

A

A

See: let's look for reposse.

B

B

Okay, here it is okay yeah, it's.

A

A

Okay, here it is repost to contribute to nx plus, oh because.

A

Because I am wrong: okay,.

A

All right, um I was wrong. So what's the problem, the problem is that this is asking for a list of repository. We currently ignore unknown arguments due to the need for the nested, all domestic config stuff.

A

We don't enumerate the plugins before load. So um so, if we were to enumerate all the plugins prior to instantiating all the cli commands, we can go through and actually add our press levels for each one of them, but that would incur huge load time. So we already have large load time so repos we're supposed to contribute to.

B

Okay, so this should be.

B

A

Default equals nine, and then that way it should show up. Okay, yeah there we go so now. It's an optional parameter.

A

We'll yeah a list for us.

A

Okay, so because this man- okay, perfect, that's what I like all right: great okay! So now it's nothing.

B

A

A

B

Okay, how do we do this.

B

B

C

B

Oh, I think this is this is what we found out. Okay,.

B

A

hmm I was fine.

A

A

A

All right, I'm just playing with something here.

A

A

We've got this: we've got that all right, let's just add, cli command right if the event equals.

A

That's yet so maybe if the ctf.

B

A

So we do this thing with the context.

A

A

They have this handle, which you have to call a weight to get.

C

C

That's your question.

A

B

A

A

A

A

A

Cli has reposted an unexpected curator.

A

All right looks like we got a few outputs here. What do we get so code of contact? Okay, so no code of conduct, yes contributing no license? Yes, readme, yes, security, all right and we've analyzed this repo okay.

A

So um let's see so what do we do? Cli is meant on this repo seal and has repos okay. So we just did the infer right here where we said just recommended community standards.

A

So now we'll do this repost dot, and we should see that we have the same same output but um cli has repos is now outputting. You know the repo object from the ci and then the the one where we're inferring is not right. So, okay, so okay, so we have success.

A

That's pretty exciting! Okay! um So let's just go ahead and.

A

A

Okay and let's edit our thing here, just so that we get inferred.

A

A

Alice cli, uh please contribute.

B

Repose and basic.

B

B