DataHub Product Demos, 19 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: New feature alert! Bulk Edits in the DataHub UI

Description

John Joyce (Acryl Data) walks through a demo of bulk edits from the DataHub UI during the July 2022 Town Hall.

Learn more about DataHub: https://datahubproject.io
Join us on Slack: http://slack.datahubproject.io
Follow us on Twitter: https://twitter.com/datahubproject

A

So I'm going to be talking about something that we've at acro have been working on the past few weeks and that will be available not in this release, but in the next release, which is called bulk editing via the ui and I'm going to start with just a scenario to introduce the problem and then our solution to it.

A

So, let's imagine you know I'm a data analyst at my company and I've been given a task which is to assign a needs review tag to all of my hive tables that I've ingested from hud. Let's imagine I have about a hundred of course. This could be a lot more in reality, but we'll stick with 100 for this example.

A

So let's look at how we could solve this problem using the tools that we have today provided by datahub. So one option is that we can add the tag to each table through datahub's web ui. So what would this look like? Well, first, I would probably search for all of my hive tables and then I would kind of iterate through them one by one and add that tag using the add tags. Modal.

A

There now the problem here is that this is one at a time like very high friction and if I have to do this for 100 data sets, it's going to take me all day, so not not ideal.

A

Now, if I'm a little bit more savvy or more technical, actually I may think of another solution, which is to add the tags to all these entities via our graphql api, using the add tag mutation and what would that look like well? First, I would, you know, write a python script that would find me all of the hive assets. Maybe I would issue a search query and then second, I would issue add tag mutations for all of those those entities, and hopefully it would just work now.

A

The downside to this is that it's really just not practical or desirable for most users, including technical users. I personally am a software engineer and I don't want to do this.

A

So what we're introducing is a new solution to changing entities in bulk via the data hub ui, and that's what we're calling edit bulk editing from the ui.

A

So at this point, I'm going to jump into a demo of what this will will look like over here in my data hub there's a few different surface areas where we're going to be allowing you to make changes in bulk.

A

The first one is on the search page, so let's go and actually search down for all of my hive assets. So now I actually have eight. You know in reality. Imagine this is a hundred once I've searched down to the assets I care about. I can go ahead and click this drop down and then click edit, and what this will allow me to do is have a selection of the assets on the results page, maybe even all of them and then take some action on them.

A

So you can see I can do things like adding owners or removing owners, adding glossary terms, removing glossary terms, tags domains and you'll notice. Deprecation is grayed out, because actually this action is not supported for everything that I have selected. We don't support deprecation for databases and schemas. So let me unselect these.

A

If I wanted to use deprecation- and now we can obviously use that action, but what I'm going to demo is just adding a tag, so I'm going to go ahead and click, add tag, I'm going to search for needs review and then I'm just going to add it to all of them at once.

A

So you can see this is pretty useful for this kind of batch updating case. Obviously I still have all these selected, so I can continue to make changes or I can just exit my selection and be done.

A

So this is the first surface area, but what you're going to see when we roll this out is that this is going to appear on every kind of search, surface area that we expose in the product. So, for example, if we go to this domain, we can see all of the entities inside of it. This is again a search surface where we can take batch actions, so we can select everything you know, maybe on the first page of this domain and then do something here as well.

A

um The final thing, I'd like to demo is being able to kind of bulk, apply domains and glossary terms from the profiles themselves. So maybe I know that you know there's a bunch of assets in my ecosystem that should be under data engineering, and I don't want to go and assign that domain one by one. I can actually come right into this domain. Page click, add assets and then go ahead and search for the assets that I may care about and add the domain. So maybe everything owned by datahub right and there you have it.

A

So it's the same thing for glossary terms, we're going to be rolling this out for them as well quickly I'll show that so same exact modal, where you can search, filter, select and then take some action in batch. We've also added this handy drop-down that allows you to change the number of results per page so that your batches can be pretty large, maybe 100 at a time all right and with that I'll jump back into the presentation.

A

So a quick recap: um the features we're rolling out in the next release. Not this release are search, select, enrich with all the core enrichments that you would care to do, owners, glossary terms, tags, domains, deprecation and more and in terms of availability.

A

Again, this will be in the next release, which is scheduled for 8 12, 20 22, and it will support tags terms, domains, owners, deprecation and hopefully, soft deletion, which will make cleaning up assets much easier from the ui.