a.k.a. Flint-Script
This part is a limited first edition to describe some of the WEM Test Script DSL features as part of the WEM Regression Test Framework.
In future, we plan to provide a Portal in which you can manage your own testsuites and scripts, request test-executions and review test-results. This WEM Test Script will be an important part.
Statements that can be used in the WEM Test Scripts to perform specific actions.
Use := to assign an expression/value or a web element (an element on your page in the Runtime to work with) to a local variable.
Make a screenshot of the current page. Screenshots will be compared to check differences between the test environment and the reference environment.
If no name is given, a default name will be used like: "#-screenshot".
The resulting screenshots can be accessed in the Artifacts collection of the Job Execution Result.
Navigate to a specified page (perform an HTTP-GET request).
The <page>
value will be added to the end of the given base URL (test/reference host + relative path). If <page>
starts with https:// or http://, the base URL will be ignored and <page>
will be considered an absolute path (which may be an external URL).
Every script should start with a Navigate to statement.
Perform a Click-action on specified (web) element.
Hover over a web element, to test tooltips.
Select value or text from specified web element, like a dropdown.
Optional keyword text or value, determines which attribute is used for <value>, default is text.
Put a value into specified web element, most useful to perform adding values to Input Fields.
Capture text or html of specified web element.
Optional keyword text or html, determines whether outer-html or text is captured, default is text.
Wait for a duration specified in milliseconds. This may be required if certain actions (loading a page, retrieving data) take a longer time. With Test Automation it is sometimes necessary to put in manual wait statements to make sure that the test can perform realistically and this means to have to wait until some longer running process is finished before the next step should be executed. Often this is a matter of trying a few times until the optimal waiting time is found...
Tries to download specified link.
Use Wait at the end of the script if it's a big download.
Download does not work in Edge browser.
The <link>
parameter should be the Text attribute of the link (the visible text that is clickable).
The resulting file can be accessed in the Artifacts collection of the Job Execution Result.
Resize browser viewport (to test responsive elements) to indicated width and height. When no arguments are present it will revert back to full screen. If height is not present, the width-value will be used for height as well.
Continues executing statement block while <condition>
is true.
Keyword end
specifies end of statement block.
Optional keyword break
, if this is reached, the loop is ended and flow continues after the end
.
If <condition>
is true, executes statement block.
Keyword end
specifies end of statement block.
Optional keyword elseif
, if previous conditions are false, execute the statements in the elseif
block, if its specified condition is true.
Optional keyword else
, if previous conditions are false, execute the statements in the else
block.
Every new statement should be on a new line.
Get the web element in another web element.
Check if one or the other is true.
Check if both are true.
Negates expression.
=
, !=
Compare values with each other. Equal (=
) or Not-Equal (!=
)
<
, >
, >=
, <=
Check if values are greater or less than (or equals).
+
, -
, *
, /
Calculation symbols.
If + is used with a string, it concatenates all values into a string.
WebElements are html elements that can be interacted with using the above named statements.
Button
Panel
Input
FormItem
Tab
Link
DataGrid (class=table)
Pagination
Element
To click on a datagrid or repeater "page", use the Pagination element: click on pagination[text=42]
Using selectors, the correct WebElement can be specified. If no selectors are present the first displayed element will be used.
Selectors available are: text, title, class, label, index and all html attributes. Index is a special selector, starting from 1 it chooses the element on the given index All other selectors after index will be ignored.
A special WebElement is Element. It has it's own special selectors, all other selectors won't work. These selectors are: id, xpath and css.
To avoid a "stale" WebElement exception (the specified element is no longer part of the current HTML Document in the Browser Session) use * in front of the WebElement to create a lazy expression, this will execute the expression only when needed. Only works on WebElements.
Boolean (true or false)
Number
String
$Variable
Function(<expr>, ...)
Exists(<webEl>)
Checks if <webEl> hasn't gotten stale and still exists in the current HTML Document.
Returns true or false.
Functions always have a return value
A lazy assignment of a button element to a variable using index as a selector. Using the function Exists() to check if the current button is still valid, and increasing the index selector by 1 to loop through all button elements on the page. Making a screenshot of all overlays and closing them using click on button[text="close"].
Using click on datagrid[title="ID"] to sort specified columns of a Datagrid.
Entering "hello world" into a form-item. Afterwards collapsing the panel.
Change language to Italian using the dropdown and capturing the html of a web element.
The WEM Regression Test Framework is currently being developed for Internal Use first, to use specific WEM Projects with specific testable flows and pages that can be compared on different environments specifically to check for Regression. It uses Selenium to enable testing with "headless browser sessions" (no user interaction - optimal for browser-test-automation [wiki]).
Because WEM has its own specific elements and ways, that need to be taken into account when WEM Projects need to be tested automatically, the WEM Test DSL (Domain Specific Language, [wiki]) has been developed, providing specific script-commands to execute tests and perform specific actions on a WEM Project's Page in Runtime. Internally this is called "Flint-Script
" after the person who created this DSL . The next page provides details on this script.
The WEM Regression Test Framework consists of the following parts:
WEM Projects, available in staging/live runtimes, with specific test-able features that can be performed and checked using test-automation;
Collections of test-cases, that may be organized and managed in suites, groups, written in WEM-Test-Script;
Test Execution Server, that can accept collections of test-cases, execute them, perform comparison checks (based on screenshots) and return the results.
The Test Execution Server (TES
) is at the time of writing only available for WEM Internal Use as we are still developing and improving this WEM Test Framework, but we intend to make this available in future. The TES
hosts the Selenium Hubs, running on Kubernetes nodes. It can receive Test Jobs (containing all information about the Test Scripts to be executed against which runtimes) and executes them on Chrome, Edge and Firefox browsers (using the Selenium Drivers), and provides results. It provides a REST/JSON API to exchange the information.
When a Test (a collection of test scripts) is being executed as a Test Job on the TES
, the Test Job will have 2 specific URLs: Test and Reference, to compare the test cases on the Test Environment (the environment that is being tested/checked for changes) to the situation on the Reference Environment (the unchanged / current environment that holds the situation as it is expected to be). These hosts are provided on Test-Job level as parameters and can be to compare:
Test-Runtime against Live-Runtime (what we usually do when Runtime Changes are about to be deployed);
Staging against Live (to check certain changes before publishing a project);
Public Runtime (staging/live) against Private Runtime (staging/live).
This approach requires that the tested project is available on both (test and reference) runtime environment, published and containing the same datasets (which may require additional manual work, unless within the project a specific data-initialization flow is available).
During Q3-Q4 2023 we will be working on a specific projects in WEM in which the Test Suites/Groups/Cases can be managed and organized, the test-scripts can be written in a user-friendly way, and ultimately can be sent to the TES
for execution and test-results can be reviewed. The plan is to make this available at some point (when ready) via MyWEM or WEM DevOps Portals so WEM Users with the proper access can use this solution online (in runtime) to create and execute their own automated tests against their own WEM projects.