GitLab 15.5 GitLab Kickoff, 15 Sep 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitLab 15.5 Kickoff - Enablement:Global Search

Description

Planning issue: https://gitlab.com/gitlab-org/search-team/team-tasks/-/issues/110

Presented by: Global Search team, Tomas Bulva, John Mason, Madelein van Niekerk, Dmitry Gruzd, Terri Chu, and Changzheng Liu

A

Hi I'm CG I'm the engineer manager of global search team at Advocate lab today, I have tamash Zhang and Maddie Dimitri and Terry with me. We are going to talk about the features that we are going to do in the keystart 15 5 milestone.

A

So, first of all, we have our newest team member Maddie and sidas join the team. We'll get more help in the future. We hope we can deliver more exciting features to our users.

A

uh Okay, so I'll pass to other team members to uh walk you through the features that we'll be working on in the milestone.

B

um Thank you, CCR I will take this one. So uh first uh thing on the search pages that we are moving navigation from the horizontal tabs to the vertical navigation- and this is uh first big change. We will be uh adding more uh controls on the on the left side in in that sidebar, but this one is a coming along nicely. It's it's! uh It's stretching across multiple uh milestones and it's going to be ready in the next one.

C

Oh hi, uh my name is Perry. The next issue that we're going to be working on is completing the work to um filtering by program language to the code search the back end. Work has been completed and for this Milestone we'll be implementing the front end work as well as performing the rollout of the feature flag in gitlab.com.

A

D

So continuing the topic of facets, we're going to add labels facets for issues uh in in previous milestones. We with indexed label labels for issues and in this Milestone we're going to introduce new aggregation, API and the client side. So that's that's very exciting and go to Mission.

E

Hi, so my name is John. um What we worked on last Milestone was a bit of the design work behind adding users to elasticsearch. This is a priority for us, because we know that user search is a feature accitlab that is frequently used and we see an opportunity to have a really big performance boost by indexing users into elasticsearch instead of using the database. So for this Milestone, one of our stretch, goals is to work on the backend indexing portion of this, so that we start when users are created or updated.

E

We start indexing them into elasticsearch.

A

I believe these are the features that we're going to work in the next Milestone. As next part um is the maintenance work. The maintenance work is about to improve, for example, the performance and probably also uh some of the usability issues.

E

Yeah I can take this one.

E

um So uh speaking of performance, one of the things that we need to do as part of maintaining gitlab.com elasticsearch index with a corpus of data- that's constantly growing, as we scale is to resize our indices and to have a appropriate amount of shards so that we can distribute the load, and we have quite a few indices that we need to re-index into having more shards for this Milestone I.

E

Think what we're going to focus on is notes because we're going to see the biggest performance enhancement there right now, whenever uh notes are being searched, it's only being distributed across two shards we're going to change that to 24. So we're going to see a big Improvement there and I could take this.

E

One too CZ um regarding performance in elasticsearch um search has a feature where, if there is an operation that is running slow, it'll, let you know by putting it in the slow log um it the API for that changed and major version 8, and so uh our previous settings were deprecated. So it's just a small minor change of adding that back in so we can maintain and manage any slow operations of plastic search.

A

D

I can take this one so in in this Milestone we are going to work on in the index lifecycle management policy. So, in order to avoid constantly reducing our indices, we are going to introduce a new code that can create new indices automatically, so they will be combined under one alias, that's quite exciting change and we are going to work on phase one okay,.

A

Thank you, Commission.

C

uh I can take this one, so uh we are going to start the work of uh using rally for elastic search performance testing. The first item is going to be a spike ticket of research, where we figure out what steps will get us to have our first rally custom workload track.

C

um We have some ideas from training that we would like to implement here, but yeah. The I think the outcome of this is going to be additional issues created that we will be working into a future milestones.

A

Okay, thanks for coming uh I believe uh these are the maintenance work that we are going to do other than the feature and maintenance work. We are also going to fix a few uh bugs that we hope to render the better search results and the prop we improved usability I will have Dimitri and Thomas talking about two major issues that we are going to work out.

D

Right so this issue is about code search, so we know that sometimes, when we have special characters, the results are not there. So we're going to change our analyzers and hope that it will improve our code search quality drastically.

A

Vector dimension.

A

B

Password uh so a few Milestones ago we fixed um a regression issue where a long, long um search results in the code search were not wrapping on multiple lines. uh But this is a follow-up issue where we wanna um polish this even further, and then we have really long uh search results. We want to make them a little bit sharper and uh have just maximum about a thousand characters. So we will, uh we will cut them on both sides, trim them around the Highlight or around what you're searching for and offer better search results.

A

Thank you so much other than these two major issues that we just talked about. We will also address some of the configuration issues and also will be addressed some issues in the advanced search syntax as well as search API instrumentation.

A

The last part is the design, uh so in 155 um we will looking into a maybe summarize other foundational uh research results in a blog post, we'll also looking into the design of the user search with elasticsearch, as well as the code search implements.

A

um Yeah I believe, uh that's all for the 15-5 planning. um Thank you for watching.