OpenJS Foundation OpenJS World 2021 - Automation, 2 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: A Robust, Modular UI Automation Framework with WebdriverIO - Olga Smolyar, InterSystems

Description

A Robust, Modular UI Automation Framework with WebdriverIO - Olga Smolyar, InterSystems

Including automated UI testing as part of a DevOps pipeline has become an industry-standard practice. A variety of tools are available, both open-source and commercial, that provide APIs for simulating user interaction with a web interface. WebdriverIO has emerged as the leading javascript-based platform for functional and UI testing, providing an array of protocols, services, and BDD/TDD frameworks.
Historically, UI automation has been a brittle undertaking, requiring considerable effort to safeguard test suites against flakiness and unreliable results. WebdriverIO addresses some of the inherent instability intrinsically.
Harnessing the full power of any UI automation tool requires the use of strategies to build a robust framework on top of the interaction layer provided by its APIs. We discuss here some of these strategies in the context of a modular, flexible and robust functional testing framework built with WebdriverIO and Cucumber.

Join the speaker for live Q&A on Slack, channel - #openjs_world-automation_ci_cd
Thursday, June 3 from 12:40 - 13:00 PDT / 21:40 - 22:00 CEST
Thursday, June 3 from 13:00 - 13:20 PDT / 22:00 - 22:20 CEST

A

Hello and welcome to openjs foundation's world 2021 talk on ui automation, with webdriver io and cucumber a robust modular btt framework. I'm olga smolyar representing entry systems. Ui automation has historically been regarded as a brittle undertaking requiring considerable effort to safeguard against unreliable results and flakiness.

A

At the same time, there's been a desire to shift various automation, suites leftward in a devops or qa pipeline. This, in turn increases pressure on the automation suites to be more reliable and more robust.

A

Webdriver io has emerged as the leading javascript-based platform for functional and ui testing. Webdriver io is highly extensible and supports an array of protocols and services.

A

It also addresses some of the instability inherent to ui automation intrinsically via built-in strategies, such as its automatic weight, a locator weight strategy and other techniques.

A

However, harnessing the full power of any ui automation tool requires the use of additional strategies to build a framework on top of the interaction layer provided by its apis.

A

We're going to talk about this in the context of the ui automation framework we use at intersystems for a bit of background. Intersystems is a leading provider of data technology across a range of sectors.

A

Some of our main products are intersystems, iris and iris, for help which are data, platforms and health share and track care which are product suites specific to the healthcare industry.

A

The quality development organization consists of about 70 people and the data platforms. Division runs a little over 5 000, selenium tests and many tens of thousands of other backend tests and the health share. Division which uses an agile methodology consists of nine scrum teams running about 700, selenium tests currently using selenium grid and jenkins.

A

We also have a selenium community of practice which meets bi-weekly across these various scrum teams and divisions and hosts presentations, roundtables workshops and code reviews.

A

It has also been instrumental in extending the framework which we're going to look at some of the framework features and strategies we'd like to share include separation of concerns across a range of levels, the use of bdd principles which allow, among other things, streamlined test data for input and validation, and a number of guardrails that we've put in place to address the reliability aspects mentioned earlier, including implicit, constructor-based, page validation, stateful navigation and static typing of various entities.

A

Finally, we'll look at some utilities that we've developed for efficient data input and validation, including get and set universal for type agnostic element; interaction, get and set properties for bulk population of inputs and a custom table. Validator separation of concerns is implemented via modularization and object-oriented principles which confer the obvious advantages of maintainability reusability, flexibility and readability.

A

The approach we take is to maintain a separation between page level document object, model and business logic, between navigation and validation, between input, data and functionality and between code expertise and business domain. Knowledge.

A

This is a schematic of the structure of our ui automation's architecture, because this slide is a bit busy, we'll go through it layer by layer at the bottom is the open source layer, including webdriver. I o, of course, and a number of services.

A

Including typescript cucumber and the allure reporter, which another breakout session in this conference will be addressing. Also next is the ui map layer, which contains locator maps that resolve elements based on selectors and give them names.

A

Additionally, we define custom commands to extend webdriverio to add logging and optional parameters, as well as new commands such as get and set universal, get and set properties set, checkbox and radio and many others. We also define common utilities, including page stack and process table and other commonly used functions.

A

These rectangles shaded in blue are part of a common repository that teams can map their um their code to to take advantage of the latest developments in common code, whereas the rest of the framework is project or team specific.

A

The page object layer facilitates logic-less interaction with elements, page object, classes, extend locator maps and define custom atomic methods on elements if needed. They also include page validations in their constructor for implicit validation.

A

They can also refer to common widgets and panels.

A

The business layer contains the application's navigation and validation logic and is implemented via so-called business ui service classes. These extend page object, classes and implement workflows within and across pages. They also model the relationships. Among pages.

A

And the flow within the application for one thing by having their navigation methods return an instance of the landing page, meaning anytime, an action results in navigation to a new page. The method that performs the action instantiates and returns an instance of the child page class in the shaded rectangle is the class hierarchy that comprises the page object as a given page object and represents a page or panel in the application, depending on the complexity of the application. This can be simplified and can condense to just one or two classes or files per page.

A

If there are not many elements per page or the business logic is not very complex, but we find this separation quite useful, especially in order to provide flexibility for multiple teams, testing, different types of applications, business ui services, reference business option, type, definitions which specify the available parameters for their methods, and likewise there are business option, classes that specify defaults and construct. The custom cucumber world object.

A

The cucumber world object is designed to maintain maintain state among steps in a scenario it contains business options and other data.

A

Finally, the test layer which is powered by cucumber, consists of feature files and step definitions, feature files, use the so-called gurken syntax, to define test scenarios and scenario outlines and to specify test data feature files, call steps defined in step definition, files which map steps to ui service methods.

A

They can be used to chain navigation and validation commands for readability.

A

The test layer is isolated from dependencies on downstream pages by the mechanism described here, where business ui services return instances of pages that they navigate to so that, rather than having to import and depend on every page, a test interacts with it usually only has to import the login page and all the rest are organically generated during navigation of the workflow.

A

Next, we'll take a quick look at our use of bdd principles, which is basically about cucumber and what is cucumber. It's a behavior-driven development test framework which defines a test layer on top of a test framework written in other languages for which cucumber provides findings.

A

It provides a business facing language which gets mapped to code via step definitions.

A

If you look at cucumber's website, they build themselves as an open source tool for executable specifications that merges specification and test documentation into one cohesive whole. So this describes an ideal of bdd development where end-to-end traceability from requirements to code is implemented using gherkin, but it can also be used to simply write tests and more in the spirit of test driven development and then write the code, both application and test code to make the test pass, and this is more what enter systems strives for.

A

This is a simple example of a feature file with standard sections. The feature provides the name of the feature and a free form description.

A

The background section is optional and gets executed once before each scenario, whereas scenario is basically a test, it's one set of steps with no examples table, whereas a scenario outline is a set of steps that gets repeated for each row of an examples table like a for loop, the keywords given when then and and are used to provide logical flow for the scenarios.

A

Additionally, most ides have plugins that support cucumber with stub text, auto completion and very useful keyboard shortcuts for formatting tables.

A

Individual steps can also be configured to accept a table as an input parameter. This is a capability of cucumber that we find particularly useful for streamlining and organizing our data visually.

A

When steps accept a data table, they are passed an object with methods that can be used to access the data, depending on the type of um the format in which the step data table is presented. Along with the method that's used to parse it different data structures are generated.

A

The most common one table.rosehesh simply takes two columns interpreted as keys and values to create an object. Similarly, table.hashes creates an array of objects corresponding to the rows of a table and we'll come back to this one. When looking at our process table validation, utility scalar parameters can also be embedded in line in the step definition.

A

Next, we'll take a look at some of our guardrails. Firstly, implicit page validation within page constructors allows page validation on navigation. This implicitly enforces that correct navigation has occurred before continuing with the test.

A

This is an example of a constructor that calls a validation method which can validate any aspects of a page, and in this case it's just looking for the title.

A

Context.Page is a global that constructors also take care of that keeps track of the current page for stateful navigation. By this we mean that step definitions and methods can check whether the current page class matches where they expect to start and if not, they can navigate to it.

A

Additionally, all ui service constructors contain a param at least one parameter called validate, and this allows you by the classes to be imported into other files without the compiler trying to execute page validation while loading the files, which would cause an error.

A

Instead, the validation only gets executed when the constructor is invoked by the page, getting instantiated by another page's navigation method that passes the validate parameter as true. Additionally, divergent paths for expected landing pages can be parametrized using multiple values or constructor parameters, as shown here where a login page can expect to land on the home page if given valid credentials or land on an error, and the appropriate validation pathways can be controlled using these parameters.

A

In addition to the current page, we also maintain a page stack which we refer to as stack p, an array that keeps track of the page navigation history, and this was devised to support backwards navigation while avoiding circular dependencies.

A

These would arise if each forward navigation action imports and constructs an instance of its child landing page, then backward navigation would require the child page to import and instantiate its parent, and this would, of course, cause an error. Instead, we devise these utility methods.

A

Push p pushes the current page onto the page stack when it's called in the page constructor.

A

It sets context.page to the current page class and prints out the page stack as seen here on the right poppy pops, the last page off the page stack when navigating backwards from a page, for instance by closing a modal clicking, a browser back button or using breadcrumbs.

A

This can also optionally validate that we've landed back on the parent page and sets context.page to the correct page.

A

It also prints the stack so on the right here we have a snippet of our console output, showing the custom commands, logging that logs the action and selector for each interaction or weight command, and the page stack on each navigation and taken together. These provide a great snapshot of what the test was doing.

A

If an error occurs during development or troubleshooting, we mentioned that we use typescript for ensuring type safety across the framework, but in particular we use it to define business options which correspond to the properties applicable to a specific functional area or workflow or page.

A

In this example, here role, business options represents all the possible input values that might be provided when creating or editing a role in an admin application, whereas role validation, options list. The validation touch points that we might want to harvest while navigating through the workflow and then check later to see if they match expected values.

A

And this is one way in which we separate the validation and navigation logic type. The type definitions are then used to build the context interface, which is gets used as the type of cucumber world's context, property and multiple type definition. Files can extend the context interface next business option files are used to construct the cucumber world object whereby their constructor initializes the business options with defaults, where applicable, and these can get populated or overridden within step. Definitions by data passed in by the feature files, the classes, get instantiated and merged into cucumber world objects.

A

Context property during world construction for each scenario.

A

For a closer look at the cucumber world object, this is provided to maintain context between and among steps within a scenario.

A

This is needed because, for instance, if one step appends a time stamp to a unique entity, that's being created, other steps will be able to have access to that same type. Stamped value using the world object.

A

This gets regenerated automatically by cucumber using the world constructor, which can be overridden or extended as follows.

A

In this example, we have our business option, classes getting instantiated merged and populated into the context, property and the custom world constructor can be tied to step definitions, as shown here.

A

The world object is exposed within cucumber step definitions as this keyword and also within certain hooks as a parameter, but it's not accessible anywhere else in the code, such as the page object hierarchy.

A

So individual business options sets get passed by step, definitions to ui service methods, which provides an additional layer of access control and avoids the use of globals.

A

Another result of this is that step definitions must be defined using function, syntax, not arrow syntax. In order to use cucumber world because arrow functions don't have this keyword.

A

So, finally, we'll look at some of the utilities that we use for efficient data input and validation, get and set universal, extend webdriverio element and they first determine the element type using tag, name and type, and then they call the appropriate command to get or set the value based on the element's tag, name and type, and this is useful because it precludes the need for page objects to define accessors for every element based on the element. Type they're also harnessed in these two important custom commands, get and set properties.

A

Sub properties call set universal iterating over an item array of properties from which it constructs the page's locators and the corresponding business option value for each. So this notation takes care of takes advantage of javascript's accessor properties and access or notation, which.

A

Of course is what makes this construct possible, because a language that doesn't allow accessing properties using variables would not be able to to perform this type of interpolation. So, in order for it to make this work, names of locator definitions must match the business option. Names.

A

So an example of how we would use set properties to fill in this input form with one command. We would first define the locators whose names have to match the business options and the ui service class would define an edit array listing all the business options.

A

And then, in order to use set properties, we would pass an instance of the current page. This which gives access to the along with desired business, option, values, user options or validation options here, and the array of locator and option names to iterate over so then get and set properties call get or set universal for each item in the property array, referencing the corresponding locator and business option value. If provided- and this is why we mentioned in the beginning- that page objects only define accessors if they're needed.

A

So if an accessor needs some kind of custom handling such as first expanding a panel or switching a tab on a page or.

A

Any kind of custom weight or other custom parameter, then we define an accessor in the page object and otherwise we use the ui services to simply set groups of standard input elements all at once and similarly for the getters.

A

Finally, the process table utility.

A

Is used to validate data provided in cucumber step data tables and it can parse an html table given only the table selector, it processes a given subset of rows and columns and returns two arrays of objects, text and elements from the table both corresponding exactly to cucumbers, hash's input, table format and the resulting data structure.

A

What this allows us to do is to define a validation step like the one shown here um and the expected data for row. Numbers xyz is retrieved and then, as the input step, we simply indicate the portion of the table. The columns and rows that we want to validate and then use process table to compare the actual table. Contents to the expected subset process table also accepts faux selectors to accommodate non-standard table structure.

A

Finally, we mentioned that we use allure reports, so just a quick plug for them as being intuitive and interactable, and allowing very transparent troubleshooting by allowing you to drill down to scenarios and steps in order to view stack, traces and screenshots for failures.

A

They also provide a variety of reports and metrics and integrate with jenkins and other build managers.

A

Here's an example of a top-level suite view of an allure report with each row here, corresponding to a feature file and failures, classified into product defects or assertion, failures and test defects or broken tests.

A

You can drill into these colored bars to get to the scenario view which lists the names of scenarios in a feature file on the left hand, column, and if you select one of them, it shows the steps of the scenario in the right hand panel, along with screenshots and stack traces for any failed steps.

A

Some takeaways that we hope you'll gather from this talk are one the importance of separation of concerns, including the use of bdd principles, use of guard rails to model and enforce application logic, common utilities for efficiency and flexibility, and the use of tools for debugging and analysis that allow rich logging and reporting.

A

Thank you very much for your time.