GitLab Data Team, 6 Dec 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Taylor does a quick walkthrough of Snowflake for Telemetry PM Sid R.

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).

A

So just great cool, okay, so yeah. So this is the main stem. Flaking interface feel free to jump in. If you have any questions, but I'll start kind of where you started here so in the upper right hand corner there are. This is your switch roll reference and so by default, you'll see your user name and then the role that you're currently using you can alter. So you can alter the behavior of like what your default role is, which I definitely recommend.

A

So if you search alter user snowflake, you can change your default role, and you probably said you probably want this to be product manager, I, think because the wait so the way we do permissioning is that by default everybody gets their own user role, which is basically identical to their user name. So you should you'll, be a member of like public, but also s ready, but you're. Also, a member of the product manager, role and I think my default. It puts you in public whenever you join, which is probably annoying, so you can do so.

A

You can type alter user s, ready, set default role, equal to s, ready, I, don't know what the exact quotation is.

A

You should be able to run that as yourself, and that would change your default role so that every time you log in that's what it said is: okay, so what's confusing it's a little bit confusing is that you can also have these worksheets, where you can actually scope your roles differently, depending on what you have so like I'm, super user, I've, sysadmin and I can basically jump to any role that I want to, but when you're in a one of these worksheet tabs here, you can actually come into this context area and select a different role to it.

A

Basically, it's query the database s so oftentimes. What I'll do when I'm testing, like KMI permission to people correctly I, can come over here, select the product manager role and actually see what you see so on the left side. Here, all basse objects, if databases at the top schemas and then within a schema, there's tables and views. You can click on these and expand them and see all the tables and views.

A

You can also specify your warehouse, so it looks like you have access to pursue me. The product manager role has access to analyst extra-small reporting and analyst extra-large generally use that the extra small, because it's like it, they're easy to instances. Essentially. So if you start up the Excel one, it just costs like eight times as much as the next small one, but if you're clearing like snowplow data like you'll need the big one probably you're like, and then this this is like so from here.

A

You can actually go ahead and start querying data, because you had your role, you have your data warehouse. You can come here. Let's say you one of the query the date details table. You can actually come over here and double-click, and it's going to populate the database. The schema and the table.

A

When you don't have a database or a schema selected, you have to specify it in your query. So let's say I just did select star from eight details. It's gonna say it does not exist. There's like oh okay, well, I'm in the a Linux schema, so you can do that and it's gonna say well I, don't know what database that you want it from. So you have to be overly verbose here to get what you want, but you could come in here and select analytics and then select whatever scheme.

A

You want my default and then my original query here would work because it it like just knows the context right and you'll notice to the the difference like snowflake has by default when you double-click it does like the full quoting, but in general you can just reference a table without quotes and in lower case, and it works just fine the. So let me actually switch to your role at the top level.

A

You'll you'll see different things, I think top.

A

Yeah, so this this is kind of this is the view you would see. I think so.

A

You could come over to the database, for you and you'll see the the three main databases that we have so Raw's, where all the raw data is coming analytics is where DBT rights to, after all, the transformations and it snowflake is the built in one that snowflake manages and the like world of query history and by the data is written to, and you can click in here and see tables and views and schemas and all that fun stuff that you this you have access to.

A

I don't often come to this view, just because it's slow and less useful than the other view I think. But if you need to look up like how big is this table, you can do that. But honestly, you can also do that from from this view, if you hover over it says it's probably pretty small, but the little pop-up says number of rows and the size as well. Okay, and then it looks like you- do- have access to the history.

A

This is probably automatically filtered by roles that you actually have permission on, so you'll probably only be able to see product manager queries if I understand how fishing works.

B

Maybe not you see, see mine up there right now that we altered the default rules. Yeah nice frame it okay, yeah.

A

That looks like it worked, so you it, this may be showing more just cuz I'm logged in I. Don't know what you actually see on the history tab. You see I.

B

Just see what I ran yeah that makes sense.

A

Okay yeah, so, if you're doing any automated, if you're like connecting with DBT or you've, got automate like other scripts running this I guess that's a good way to check it. But this is how I, like I, can check on things the nice thing about this. So let's look at a more complicated query.

A

You can come in here and look at the query profile. Sorry, so let me let me go back to just fill check so on any of these queries. It has a unique grid for the query identified entity. You can click on that it'll.

A

Take you to this page, which shows the details how long it took to run and all that fun stuff, but then there's a query profile where you can see like hey, what's slow in my query and so 98% of this query is a table scan and then it is a filter and a sort and there's the result. So in more complicated queries, it's pretty nice isms or.

B

Generate the query plan basically yeah.

A

Exactly yeah yep shares and data exchange. I don't need to worry about. We don't do anything with that warehouses. This is just the compute instances that are used to actually query the data. That is like that's one of those things about snowflake. It is a usage based model. So whatever these warehouses are on, you know it's being charged at a fairly low rate, especially for the extra small ones. I, don't think it. Oh yeah, you don't have privileges to monitors, warehouse, that's fine! We do have a periscope dashboard, which is it's snowflakes and I.

A

Think what you can see like approximately how much money we're spending on different warehouses.

A

But yeah so you can, they will automatically spend. So you don't really have to worry about that. You should just feel comfortable using the excess at any time and not really worried about turning it on or off it should can automatically happen and I. Think that's the main thing or there yeah use your results down here. You can download that.

A

Let them hear if it's small enough, you can just copy it straight to the clipboard.

A

Try to think very soon, I know they will be simply bought numeracy, which is like a really nice UI for querying data, which should make hey this video deprecated, but it should make the UI like a lot nicer with autocomplete and things like that. That's one of the frustrating things about this setup currently is there's no autocomplete, but so they'll have like nice features where you can kind of scroll through select things they'll give you like small charts and stuff so I've been talking with our sales, rep and hopefully that'll be out this month.

B

Yeah, what else can I uh I had a question about that? Maybe you spoke about it, but uh the analyst extra small-extra-large and it was one for reporting. What was that for.

A

That one I, like everybody, has access to that. That's the one that periscope it primarily uses. um Ideally you wouldn't be using that one I think, but it's not a big deal. I.

A

We try to scope the warehouses, kind of to function, and so most of the periscope queries run through reporting. It is an extra small one, but I would just stick with the analyst ones. It's a possibility.

B

If the periscope report is hitting snowplow data rate, wouldn't that require the extra large so.

A

We've we've had these discussions. The problem is that if we switch to the extra large for periscope every single query for everything is going to be run on that okay, most of the incident, that's we floated an idea like okay. These queries are running long like we can bump up the warehouse size generally, though, I think we want to try and optimize the data underneath and then like work with one of the analysts to better because a lot of the times, it's just like a either a bad query, or it's like a very exploratory.

A

Kind of query, and in those cases I'd almost recommend to the person like hey like you- can definitely make these queries. Let's just get you get like straight database access and then you can, you know I think we want to try and keep that periscope one as small as possible because it's gonna be running. Basically, a hundred percent of the time make sense cool.

B

A

Think that's really the main thing and you should be able to connect if you want to use like Datagram or another tool, you should just be able to pass in your credentials and, like I, used a to grip quite a bit, and it's a bit nicer for certain things.

A

Try to think yeah, you can have a bunch of different worksheets as well, so you can see like I have one for sis admin and then, when I'm like provisioning, new people, I just switch over to security admin.

B

But yeah I guess we couldn't done. Thank you, you know from was useful. Thank you all right.