DataHub Product Demos, 24 Sep 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Faceted Search: Sept 24 2021 Community Town Hall

Description

Gabe Lyons from Acryl Data walks us through upcoming changes to the DataHub search experience.

A

So one thing that we want to demo for you today is uh improvement, that we've made two data hubs, search abilities, and so I'm going to talk through the improvements that we've made uh go a little bit into the architecture of how this was built and then also at the end, I'm going to ask for some feedback from the community to hear about how you feel about the directions, feature and we'd love to get input from you to help guide us in how we improve search in the future.

A

So the intention here is to try to condense, search into a more unified experience and help users of data hub get to the information that they need in as few clicks and as little work as possible.

A

So what I've done here is entered a search query for the phrase customer and previously in the old iteration of data hub search. You would have seen multiple different modules each with its own entity type. So some data sets that relate to this. Some charts that relate to this. The glossary terms that relate to this and so forth in their own modules. And if you wanted to go and explore more, you would have to click one of those entities and dive into it.

A

What we've done here is take all the results across the different entity types and blend them together, so that the top ranked results from each entity type appear.

A

We also are taking the filters from each entity type and blending them together, so that if, for example, certain charts and data sets came from the same platform as if you were to have some looker charts and look reviews, selecting this looker platform would filter just for entities across that platform for all different entity types.

A

If I wanted to search for a single entity type, I can go to select this entity filter, and then I see just the data set entities and by clicking on a platform I can filter down more and just see data set entities that relate to liquor. If there are tags or terms or other filters that are shared across these entities, they will also appear, on the left hand bar and, as new filters are added, they will dynamically insert themselves into the bar.

A

So, no matter what you search for relevant filters and the relevant entities are going to appear in those search results.

A

So now I'm just going to talk through briefly the changes in architecture that were able to support this feature.

A

So previously we had siloed search results, so each entity type would be searched independently and the search results and filters would be brought back to the user independently.

A

So a user would issue a search and we would search data sets, charts, flows etc, and those entity types would then give their own distinct responses. The filters for data sets the filters for charts and the filters for pipelines.

A

What we've done here is put a layer in between that will aggregate the results together, rank them amongst each other and then bring a combined search results and combine search builders back to the user.

A

So the user issues a query data data, then aggregates the search, results and search filters together handles pagination and ranks these filters based on a custom filter, ranking that you can provide data hub, provides a default filter ranking, but you can also choose which filters are important to you and your users and rank those top then it provide. Then it returns. This consolidated condensed result back to users so that they can go and filter as they wish and choose the folders that are important to that.

A

So next we want to take things back to you, the the core users of diva hub and help us understand better how this combined search result can suit your needs. So we're going to be posting a poll in the ui channel. I think maggie just submitted it, and what we want to understand from you is how you use datahub search and what you would expect from data search. So please go to data hub, ui channel and tell us when you enter a search term.

A

Are you usually trying to search across entity types, or do you usually hope to search within a single entity and we'll use this information to help steer the direction of this aggregated search and the development of search going forward, feel free to vote and definitely give us your opinion, but also if you want to throw unstructured comments in we support that as well. You have just general thoughts about search, feel free to comment on that poll and give us more feedback.

A

We really look forward to help building search with the rest of the community.