Fault-tolerance testing is an important part of any testing strategy, especially for web applications. By simulating different failure scenarios and testing the application’s ability to handle them, we can ensure that the application remains stable and functional in the face of unexpected issues.
In this blog post, we will explore how to use Katalon Studio to automate fault-tolerance testing for web applications.
Setting Up the Environment
Before we can start creating our fault-tolerance tests, we need to set up our environment. To do this, we need to download and install Katalon Studio. Once installed, we can create a new project and a new test case. We can then add our test steps to the test case.
Creating a New Test Case
To create a new test case in Katalon Studio, we can go to File > New > Test Case. We can give the test case a name and select the appropriate folder to save it in. Once created, we can add our test steps to the test case.
Adding Test Steps
To simulate a failure and test the fault-tolerance mechanism, we can add test steps to our test case that simulate different failure scenarios. For example, we can simulate a network failure by disabling the network adapter or disconnecting the Ethernet cable. We can also simulate a server failure by shutting down the server or stopping the web application’s service.
Here’s an example of a test case that simulates a network failure:
import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable as GlobalVariable
WebUI.openBrowser(‘https://example.com’, FailureHandling.OPTIONAL)
WebUI.setText(findTestObject(‘Object Repository/Login Page/Username Field’), ‘testuser’)
WebUI.setText(findTestObject(‘Object Repository/Login Page/Password Field’), ‘password’)
WebUI.click(findTestObject(‘Object Repository/Login Page/Log In Button’))
// Disable the network adapter
Runtime.getRuntime().exec(“netsh interface set interface ‘Ethernet’ admin=disable”)
// Verify that the network error message is displayed
WebUI.waitForElementVisible(findTestObject(‘Object Repository/Error Page/Network Error Message’))
// Re-enable the network adapter
Runtime.getRuntime().exec(“netsh interface set interface ‘Ethernet’ admin=enable”)
// Verify that the login page is displayed again
WebUI.waitForElementVisible(findTestObject(‘Object Repository/Login Page/Username Field’))
Handling Test Failures
One of the key benefits of automated testing is the ability to quickly identify and diagnose test failures. Katalon Studio provides several features for handling test failures.
The first feature is the ability to take screenshots of test failures. This can be useful for debugging and identifying the cause of the failure.
We can use the WebUI.takeScreenshot() method to take a screenshot of the current page when a test fails.
Here’s an example of how to take a screenshot on test failure:
import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable as GlobalVariable
WebUI.openBrowser(‘https://example.com’, FailureHandling.OPTIONAL)
WebUI.setText(findTestObject(‘Object Repository/Login Page/Username Field’), ‘testuser’)
WebUI.setText(findTestObject(‘Object Repository/Login Page/Password Field’), ‘password’)
WebUI.click(findTestObject(‘Object Repository/Login Page/Log In Button’))
// Disable the network adapter
Runtime.getRuntime().exec(“netsh interface set interface ‘Ethernet’ admin=disable”)
// Verify that the network error message is displayed
WebUI.waitForElementVisible(findTestObject(‘Object Repository/Error Page/Network Error Message’))
// Re-enable the network adapter
Runtime.getRuntime().exec(“netsh interface set interface ‘Ethernet’ admin=enable”)
// Verify that the login page is displayed again
WebUI.waitForElementVisible(findTestObject(‘Object Repository/Login Page/Username Field’))
// Take a screenshot if the test fails
if(WebUI.verifyElementVisible(findTestObject(‘Object Repository/Login Page/Username Field’), 0, FailureHandling.OPTIONAL) == false){
WebUI.takeScreenshot()
}
The WebUI.verifyElementVisible() method is used to check if the login page is displayed again. If the login page is not displayed, we assume that the test has failed and we take a screenshot using the WebUI.takeScreenshot() method.
Retry Failed Tests
Another feature for handling test failures in Katalon Studio is the ability to retry failed tests. We can configure our test cases to retry failed steps a specified number of times using the Retry (Keyword) plugin.
Here’s an example of how to configure a test case to retry failed steps:
import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import
com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable as GlobalVariable
// Set the maximum number of retries for failed steps
GlobalVariable.MAX_RETRIES = 3
WebUI.openBrowser(‘https://example.com’, FailureHandling.OPTIONAL)
WebUI.setText(findTestObject(‘Object Repository/Login Page/Username Field’), ‘testuser’)
WebUI.setText(findTestObject(‘Object Repository/Login Page/Password Field’), ‘password’)
WebUI.click(findTestObject(‘Object Repository/Login Page/Log In Button’))
// Disable the network adapter
Runtime.getRuntime().exec(“netsh interface set interface ‘Ethernet’ admin=disable”)
// Verify that the network error message is displayed
WebUI.waitForElementVisible(findTestObject(‘Object Repository/Error Page/Network Error Message’))
// Re-enable the network adapter
Runtime.getRuntime().exec(“netsh interface set interface ‘Ethernet’ admin=enable”)
// Verify that the login page is displayed again
WebUI.waitForElementVisible(findTestObject(‘Object Repository/Login Page/Username Field’), GlobalVariable.DEFAULT_WAIT_TIME, FailureHandling.OPTIONAL)
// Retry failed steps up to the maximum number of retries
WebUI.callTestCase(findTestCase(‘Test Cases/Retry Failed Steps’), [:], FailureHandling.OPTIONAL)
In this example, we set the maximum number of retries to 3 using the `GlobalVariable.MAX_RETRIES` property. We then call a separate test case (`Test Cases/Retry Failed Steps`) to retry any failed steps up to the maximum number of retries. The `[:]` argument is used to pass an empty map of variables to the called test case.
The `Retry (Keyword)` plugin provides additional configuration options for controlling when to retry failed steps and how long to wait between retries.
Conclusion
In conclusion, Katalon Studio provides several features for automating fault-tolerance testing. By simulating network errors and other faults, we can verify that our applications are resilient and able to handle unexpected conditions.
The `WebUI.verifyElementVisible()` method and the `Retry (Keyword)` plugin are powerful tools for handling test failures and improving the reliability of our test suites. With these tools, we can ensure that our applications are ready to handle anything the real world can throw at them.