Keyword-driven testing with Squish and Robot Framework

Last edited on

Introduction

Keyword-driven testing is a software testing methodology which uses keywords (or action words) to symbolize a functionality to be tested. This method separates the description of the tests from its implementation and therefore reduces the future maintenance time caused by changes in the AUT. In the case of GUI changes in the AUT, only very small changes or even no changes have to be completed in test case documentation. Some changes need to be applied to keyword documentation only.

Robot Framework is a test automation framework which utilizes the keyword-driven testing approach. Its advantage is that users can create new higher-level keywords from existing ones. To integrate the Squish GUI Testing tool with Robot Framework, the keywords library needs to be implemented with Python using the squishtest module.

As an example, the Qt AddressBook application was chosen, which ships with Squish for Qt. Using a similar approach, we can automate Java, Windows, Mac, iOS, Android and Web applications.

Example

Robot Test Case

A test case can be defined in a plain text file with the txt extension. Test cases are constructed in test case tables and built from the available keywords. Keywords can be imported from test libraries or resource files (e.g. resource.txt). The first column in the test case table contains test case names (e.g. Add Entry, Empty AddressBook). The second column normally contains keyword names (e.g. New Addressbook, Add Entry), columns after the keyword name contain possible arguments for the specified keyword (e.g. Tom Paw tom@m.com 12345). Additionally we can define test setup and teardown actions. A test setup is something that is executed before a test case, and a test teardown is executed after a test case. In the example below the Start AddressBook keywords is executed before every test case.

*** Settings ***
Documentation     A test suite with two tests for addressbook.
...
...               This test has a workflow that is created using keywords in
...               the imported resource file.
Resource          keywords.resource
Test Setup        Start AddressBook

*** Test Cases ***
Empty AddressBook
        New AddressBook
        Entries Should Be       0

Add Entry
        New AddressBook
        Add Entry       Tom     Paw     tom@m.com       12345
        Entries Should Be       1
add_entry.robot

Keywords Resource File

A resource file contains keywords definitions that are used in test cases.

To define keywords we utilize low level keywords provided by the SquishLibrary (see "Low Level Keywords Library" below).

*** Settings ***
Documentation     A resource file with reusable keywords and variables.
...
...               The system specific keywords created here form our own
...               domain specific language. They utilize keywords provided
...               by the imported SquishLibrary.
Library           SquishLibrary

*** Keywords ***
Start AddressBook
        Start Application   %{AUT}  %{AUT_WRAPPER}

New AddressBook
        Click Button  names.address_Book_New_QToolButton

Add Entry
        [Arguments]     ${forename}     ${surname}      ${mail}         ${phone}
        Click Button    names.address_Book_Add_QToolButton
        Enter           names.forename_LineEdit                 ${forename}
        Enter           names.surname_LineEdit                  ${surname}
        Enter           names.email_LineEdit                    ${mail}
        Enter           names.phone_LineEdit                    ${phone}
        Click Button    names.address_Book_Add_OK_QPushButton

Entries Should Be
        [Arguments]     ${num}
        Property Should Be      names.address_Book_Unnamed_File_QTableWidget       rowCount        ${num}
keywords.resource

Low Level Keywords Library

The following example library "SquishLibrary" contains "low level" keywords, that is, keywords that are actually defined in Python code.

It utilizes the Squish Python module squishtest , which allows us to use the Squish functions in Python code.

These low level keywords are meant to be used in our *.resource file mentioned above.

import os
import sys

import names
import squishtest


class SquishLibrary:
    """Example Squish Library for use with the Robot Framework"""

    ROBOT_LIBRARY_SCOPE = 'TEST CASE'

    def __init__(self):
        pass

    def start_application(self, aut, wrapper):
        """Starts AUT, args=(aut, wrapper)"""
        print("*INFO* Starting AUT: %s with %s wrapper" % (aut, wrapper))

        # Remove double quotes for setWrappersForApplication
        aut = aut.replace('"', '')

        squishtest.testSettings.setWrappersForApplication(aut, [wrapper])

        # Add double quotes for startApplication:
        squishtest.startApplication('"' + aut + '"')

    def __LookupRealName__(self, name):
        split = name.split('.')
        module = sys.modules.get(split[0])
        return module.__getattribute__(split[1])

    def click_button(self, objName):
        """Clicks on button, args=(objName)"""
        squishtest.clickButton(squishtest.waitForObject(self.__LookupRealName__(objName)))

    def enter(self, objName, txt):
         """Enters given text into TextField, args=(objName, txt)"""
         print('*INFO* Entering "%s" into object "%s"' %(txt, objName))
         squishtest.type(squishtest.waitForObject(self.__LookupRealName__(objName)), txt)

    def property_should_be(self, objName, prop, expected):
        """Verifies that given object has property with expected value, args= (objName, prop, expected)"""
        print("*INFO* Got arguments: %s, %s, %s" % (objName, prop, expected))
        obj = squishtest.waitForObject(self.__LookupRealName__(objName))
        val = getattr(obj, prop)
        if not (str(val) == expected):
                raise AssertionError("Expected value is %s, but got: %s" % (expected, val))
        print("*INFO* Assertion passed, value is %s" % val)


# Execute this on module level, because it must
# be executed only once:
squishtest.setTestResult("xml3.4", "%s/squishtest_results_xml" % os.environ["RESULTS_DIR"])
SquishLibrary.py

Executing the Example

Requirements

Note: The separate Python installation must have the same Python 2 or Python 3 major and minor version as the Python installation in the Squish for Qt installation, as well as the same architecture.

General Example Execution

Start cmd.exe / Command Prompt and execute the following commands in it:

cd "path\to\squish_with_robot_framework"

execute_tests_qt_addressbook.bat "path\to\python_with_robot_framework" "path\to\squish_for_qt"

(Replace "path\to\python_with_robot_framework" and "path\to\squish_for_qt" as needed.)

Specific Example Execution & Output

cd "C:\Users\myuser\squish_with_robot_framework"

execute_tests_qt_addressbook.bat "C:\Python38-64\python.exe" "C:\Users\myuser\squish-7.0.0-qt62x-win64-msvc142"
==============================================================================
Addressbook Tests
==============================================================================
Addressbook Tests.Add Entry :: A test suite with two tests for addressbook.
==============================================================================
Empty AddressBook                                                     | PASS |
------------------------------------------------------------------------------
Add Entry                                                             | PASS |
------------------------------------------------------------------------------
Addressbook Tests.Add Entry :: A test suite with two tests for add... | PASS |
2 tests, 2 passed, 0 failed
==============================================================================
Addressbook Tests                                                     | PASS |
2 tests, 2 passed, 0 failed
==============================================================================
Output:  C:\Users\myuser\squish_with_robot_framework\results\output.xml
Log:     C:\Users\myuser\squish_with_robot_framework\results\log.html
Report:  C:\Users\myuser\squish_with_robot_framework\results\report.html