GitLab Quality Department, 28 Apr 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Multi-modal end-to-end test demo

Description

Demo of multi-modal test code. This demo shows how we can change out end-to-end tests to test both the API and UI in the same test file. This has the advantage of running faster and still gives us the flexibility to choose if we want to prioritize speed or to test the UI.

Link to issue: https://gitlab.com/gitlab-org/gitlab/-/issues/212390
Link to demo code: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/27964/diffs

CORRECTION: it should be 'export ELASTIC_URL="http://localhost:9200"' at time index 2:40

A

Oh hello, my name is Eric banks, I am a senior software engineer and test with the quality department and I am working closely with the advanced global search group and today, I'm here to show you the multimodal end-to-end tests of the API and the UI. We share my screen.

A

There is a issue for this here: I'll link this in the description for the video below, as well as some demo test code that I have up. So the first thing we're gonna do is prepare our environment to run the test.

A

One of the things we'll need to do in in the get lab UI is to go to integrations, which is this page and expand elasticsearch and ensure that elasticsearch is on and that our URL is set to localhost 9200 or wherever it is that you run your elasticsearch service.

A

You can confirm that it's running at the appropriate location by just hitting localhost 9200. The next thing we'll want to do is well before you do that you'll actually want to make sure that you have a license and the license. So here I already have one installed. Otherwise you won't get elasticsearch okay, so we have. We have an e license. We have elasticsearch on the next thing.

A

We'll do here is set up our environment variables because we're going to be doing an API test, but we want to make sure that the UI gets triggered the least amount possible, and one way to do that is to unset the variable EE license.

A

Otherwise, otherwise the UI will trigger and it will try to generate one for you or it'll. Try verify that there's one there rather and so will unset that and then we'll make sure that we have a gitlab QA access token, which I've generated beforehand. So I will export that.

A

Next, we'll set our elasticsearch URL to the appropriate location, in this case local hosts 9200 and we're setting this new environment variable called execute test by a API we'll set that to false first, because we want to see the test run initially through the UI.

A

Next you'll want to make sure your GDK has been started. So we'll do that and if you haven't already start the elastic search.

A

Okay, we're good there, let's pop over to the code for a second. This is the test that I've changed. I've moved it to a place called multimodal under the e test, and this is advance global, advanced, syntax search, spec, really what we're doing I'll just jump to the meat of the test here. What we're doing is we're using search to find a project with this string and we're expecting to find it and then we're, then we're going to search in the project description and we expect to find this string and what we're doing is we're.

A

Checking that environment variable called execute test by API and if we're saying execute test by API is false, then we'll use the UI and we'll execute this to search by the UI. If it's true, then we'll execute the search by the API, so, as you can see in the API search, we're just generating an API call and verifying that the JSON body matches what we expect and when we're using the UI, we're actually logging in and clicking buttons and searching for the right strings.

A

But the test logic is essentially the same. So, let's see how these run.

A

Here is the test call and in this case we're going to execute, headless false and see what happens. I'll move this out of the way, so you can see the test kind of executes better.

A

So there's the first pass and there should be one more successful test.

A

Yeah, okay, so we finished the test and it took twenty eight point seven seconds now. What we're going to do is we're going to change that environment variable execute test by an API to true and we'll run the test again.

A

A

Yeah you'll see the UI kind of popped up in the background, but it wasn't really doing the same things so this time the test finished in eight point three seconds, which is the main advantage of this approach, is that the API runs a lot faster than the UI, so you can get results faster.

A

We can also set it up the sched we can set up the schedules so that for every mr, for example, you can run just the API versions of these tests to confirm that the magic behind your changes is working correctly and then on a nightly schedule. We can change it to run the UI and therefore check off the UI and the API functionality to make sure they're the same and verify that they're working both equally well with the changes that you've made so yeah, that's my demo.

A

If you have any comments or questions feel free to write them out onto the issue or ping me directly. Ok, thanks.