Functional Testing

This topic discusses the types of testing, test coverage, and testing tools used by Stibo Systems to validate STEP functionality.

You can use this information as a guide in deciding:

  • The types of testing you want to do for your own development within STEP, for example, new Web UI screens.

  • Considerations for validating a new STEP version on a test system before upgrading your production system.

Types of Testing

While the following testing types can be automated, a cost-benefit analysis of automating a particular test type is recommended before committing the required budget and resources to automated testing (compared to manual testing). For more information on cost-benefit analysis, search the web.

Stibo Systems performs the following types of tests:

  • User Acceptance Testing (UAT) - as part of an implementation to ensure that end users are able to do their job completely and efficiently. This type of testing may also be part of bigger initiatives including major upgrades.

  • Configuration Testing / System Testing - on customer-configured systems at the time of implementation and upgrades.

  • Regression Testing - when rolling out new initiatives, implementing configuration updates, and/or performing migration of configuration from development / test / production databases.

  • Performance / Scalability Testing - to ensure that system(s) can handle the full load of users, imports, exports, reporting, and other processes.

  • Unit Testing / Integration Testing - when a developer commits any code change.

  • Security Testing - weekly and before releasing a new version of STEP.

Automated Functional Regression Test Coverage

Stibo Systems runs automated functional regression tests daily. These tests verify that features work as required by providing sample inputs, capturing the outputs, and validating that the expected results are returned. This allows Stibo Systems to identify regressions in UI functionality and integrations so that they can be resolved prior to release.

The automated functional regression tests are executed on the entire STEP application and are debugged prior to all software releases, including maintenance patch and hotfix-only releases.

Automated Testing Tools

A wide variety of tools exist for automated testing. Once automation is deemed beneficial for your scenario, when choosing a tool, consider the tester's skill level and experience, the effort required to create the automation, and the frequency that the test should be run.

For functional testing, the Quality Assurance team at Stibo Systems codes automated tests using the following software products:

  • Smartbear TestComplete - to test desktop applications, including STEP Workbench, Excel Smartsheets, the Print Publisher InDesign clients, etc. While various coding languages are supported, tests at Stibo Systems are coded in JavaScript.

  • Selenium - to test the Web UI only for Google Web Toolkit (GWT) screens. Tests are coded in Java and are executed in supported browsers.

  • Cypress - to test the Web UI only for React screens. Tests are coded in Typescript and are executed in supported browsers.

When writing automated tests:

  1. Ensure automated tests do not depend on other automated tests.

  2. Ensue an automated test uses a unique dataset.

  3. Use a function or class for steps to be repeated to reduce number of code lines.

Note: The browsers supported by a specific version of STEP are included in the Platform and Software Support topic under the specific feature release version within the System Release and Patch Notes section of the documentation here.

Regression Test System Recommendations

Automated functional regression tests cannot be successfully run on every system.

For automated regression testing, ideally, use an internal system with a current backup (snapshot) and a unique dataset so that the system can be reset for the next round of testing. This ensures you are in complete control of the database used by STEP, as well as the versions of the testing tools and browsers.

Important: Running functional automated regression tests on a live production environment is not recommended.

For reliable automated functional regression test results, ensure the same starting point for each round of testing with these steps:

  1. Revert your test system to a clean snapshot.

  2. Patch to the necessary STEP version.

  3. Import the unique baseline data for the test.

  4. Run the automated test.

To automate general functional testing, Stibo Systems uses a dedicated environment, where the Oracle / Cassandra keyspace can be replaced with a clean keyspace. Without a dedicated environment, automating general functional testing is more challenging since you cannot easily revert your database to a clean snapshot.