Hyper Cloud hyper, 6 Dec 2021

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Hyper Query and Bulk Workshop NodeJS

Description

Hey hyper community,

This workshop introduces the hyper.data.query and hyper.data.bulk commands for the hyper data service. The workshop takes about 20 minutes and will give you insights into these advanced commands and how you can use them to return data from the hyper data service.

Github:
https://github.com/hyper63/workshops-nodejs

Support:
Having problems working through the workshop, join our community:
https://hyper.io/slack

A

Hello, my name is tom wilson and today we're going to go through the hyper query workshop. These workshops are basically.

A

Introductions to the hyper service and in this one we're going to talk about two advanced features: uh query and bulk, both on the hyper data service. This is part three of a three-part series, part one. We introduced the data service part two.

A

We talked about hyper connect and part three we're going to talk about the query function and the bulk function of the hyper data service in order to get the most out of this workshop, you'll need to know http json, apis, javascript and git, and you will also need a github account so to set up our project.

A

What we want to do is launch in get pod and you can click this link right here or and and and I'm in the know, the hyper 63 workshops, no js repo and I'm looking at the data or the hyper query or data query, folder, the hyper query workshop and um so to get set up. You want to launch this in git pod that'll set up a virtual machine for your development environment or you can you know clone this repo and run it locally.

A

You'll need to have you know, version 16 of node.js installed at least or greater, um and we will want to launch in git pod now.

A

The way that you launch in getpod is that you can either click this link and it should launch you or you can install the extension and you get a button like this in github and then the last way is to just manually type the url. So you can open up a new tab and just type getpod.io and then pound and.

A

Github hyper 63 workshop, slash, node.js and when you type that it will take you through kind of an onboarding, and you will be in a visual studio code in your browser, which is your virtual machine like this. So once you get that running, it just takes a few minutes to spin up.

A

Then we want to head over to dashboard.hyper.io log in there with github and click on uh the application we've been using in these workshops and then go into that dashboard of the application in the keys section and click on the copy link here for the connection string.

A

So we want to take this connection string and we want to go back to our git pod and open up the folder data query and create a new file there, dot emv and that dot emv file, we're going to put our environment variable hyper and set it equal and paste that connection string in so that that single connection string gives us everything we need to securely connect to the hyper service.

A

So now that we have our environment variable set up we're ready to get started with our workshop.

A

So the first thing we need to do is uh go ahead and run npm, install or yarn, but we'll cd into the data, dot, query, folder and I'll use yarn to install all the dependencies, and then I want to just install the latest version of hyper connect. So I'll do yarn, add hyper connect.

A

And now that we have a hyper connect installed at 0.1.8, which is great, then we're going to run a setup script, and this is just yarn setup or npm run setup, and this script will actually just update our database and make sure that we've got some data, some characters, some games and some appearances.

A

So that's really great. All of those ran well and now we're going to just run yarn start to start our api server and it's uh running on port 3000 and the servers got from the previous exercises kind of the create update, get uh delete or remove and list.

A

So we can kind of test that out and see some data by just doing curl localhost 3000 api characters and that should list our characters which it does cool and we could do curl games and I'm just going to use npm x to pipe in json to that to print that out a little nicer. So you can kind of see our our games that we got here. Donkey kong, super mario yoshi's island, smash brothers, super mario maker, pretty cool okay.

A

So we got some good data now. What we want to do is set up a query where we return all the um all the characters that appeared in a given game.

A

So if we do curl post localhost, 3000 api characters, characters query and then we give it an attribute game id equals game. One then that we want that to list all the characters that appeared in game one we said game two, then we would want it to list all the characters that appeared in game two to do that we go to our api folder go to the characters folder and the query file underscore query.js.

A

This file has a handler for a post that matches this post request and right now, it'll return not implemented. So if we run this result, we're going to get not implemented. So what we want to do is implement the hyper data. Query command in this web api handler so to to do that before we were kind of importing hyperconnect, and we could do that here, but what we did since that's a repetitive process.

A

We went ahead and kind of took care of that at the server so that we can attach it via middleware to our request object. So we can just kind of pull hyper out here and we also need the query params because we're passing a query parameter of game id. So we can just pull those out of the request, object and get rid of this status. 500 and we will do const result equals await hyper dot, data, dot, query and we're going to send in a selector.

A

So we want to say type equals type appearance. So appearance is the document that kind of contains the game and the character relationship.

A

So we want that and we want to test it on the game id so we'll say game dot id and then we want to say: query dot, game id right and if everything works well, that selector should return a result and that result should have some documents in it.

A

So we can save that and if we run the curl command now we get data. We get a an appearance object with a game id of game, one and name donkey kong. Pretty cool and then the character is character, one and name mario.

A

So what we want is just to list the characters, so we can add a an option called fields and fields will allow us to select specific nodes on our document. In this case, we just want to select the character node, and so with that option we can come down here and re-run our our game, and now we just get the character um and and the character node and its value.

A

We could take it further and kind of pluck that character out using a map and um just get a list of characters like that, and that's uh pretty easy to do so. We can just say map and then um the document we get and we just want to return doc, dot, character right and now, we'll get our character.

A

We can also test with game two and we get two characters. We get mario and luigi super cool. So with this query we were able to say, give us document type appearance and we were able to use the dot identifier to drill deep into the document. So there's a game property and inside the game. Property is an id property and that id property. We want to match to query game id.

A

So with the query selector, it works a lot like um mongodb or mango um in in couch um selector format, and you can see all the selectors that you can do um in our hyper data documentation and there's a link in the readme here to to read our docs.

A

But when we specify type and then the character value, we're basically saying that it's equal, so we want to say, type is equal to character and that'll, give us all the character documents. um So so we've got our kind of query and we also have this index command.

A

So I'm going to refactor that, where we're doing this list all the documents and filter by type, I'm just going to refactor that to use query and say give us the documents that are type character, so query type character like that, and then uh I don't need this filter here. I can just set characters, equals to docs and then send characters, and that should work just as good and we'll you know.

A

Each time we make a call to the database we're not having to pull back all the documents so now we're just pulling back just the character documents, and we can test that by saying local host 3000 api characters and that should list all the characters which is cool.

A

You can type that into mpx json to kind of list them out, but you can see. Each document is a character type.

A

Cool now that is query and query is super powerful. You can use fields like we did our limit to limit the amount. You can also sort the objects based on properties and you can specify an index so there's a create index command in hyper, where you can create an index and say index these properties on an object, and then you can specify to use that index to optimize the query. So you get pretty fast read times on that.

A

So now that we did, that with character, something you can do on your own is do the same thing for games, so go into the games folder and create the same query function except traverse the games and the characters. So you would pass a character one on the query, param and it would return all the games. So that's an exercise for you guys to do just to help. You reinforce understanding the query function a little bit better.

A

It does take some practice and but once you get the hang of it, it's pretty much like a weir clause on um on your documents or a filter, and you know you don't need to to go too crazy on it. um You just want to basically provide a a grouping of getting the documents back and then then you can use functional functions like map and filter, to make your shape. Look the way that you want to return as well as to you know, kind of include or remove certain documents from the the initial query.

A

So so it's kind of a collaborative effort and the so that that's um hyper data, query hyper data bulk allows you to create, update or remove um characters. So in this case we're going to do that. We've got some data here, we're going to just copy that data and we're going to create a new file in our data.

A

Query folder we're just going to call that bulk dot js and in this new file, we're going to just paste that data variable and what we're going to do is just create this bulk load for our our little api. So we can just always load this data and to do that, we need to import.

A

A

From hyper connect and then we're going to create an instance of hyper by calling connect and getting the environment variable okay and then we're going to call just log this out, console.log and we'll await hyper.data.bulk.

A

And we're going to pass in our data like that and that's really it for bulk- is there's not a lot to it. So we can run that and we can run that down here on the terminal, a little cd into data.query and we'll just type node and we're going to just require our dot emv config file and then type walk.js hit enter. If everything works, we get the response back, okay, true and the results, so we updated character, 10 11, 12, game, 10, 11 and appearance 11..

A

So if we wanted to remove any of these documents, we can just add on the document deleted troop and then this bulk will actually remove that document from the database. So we do that and now the appearance 10 document is is gone and.

A

That's pretty much it that's that's bulk and and query. um You know definitely give all of these a try, um they're, very easy to use um and one of the the nice things you can do with query and bulk is do composition.

A

So here's an example of let's say you delete a character from your wiki and you want to remove all the appearances where that character is all the relationships.

A

Then you can use bulk to query to query all those appearances and then bulk to remove them same thing with the game when a game is deleted. Here's an example where we find all the appearances of that game and then we mark all those dots deleted to true, and then we throw those in the bulk. So it removes all the appearances.

A

Then we remove game one and that's um a very nice way to do a cascading delete in um a you know, single table database design structure. So hopefully you enjoyed this workshop. If you have any questions reach out and we will be working on the cash workshop and the new search workshop coming soon, thanks.