Automated Software Testing System
An Automated Software Testing System is a software testing system that is an automation system (for executing automated tests and validating software quality).
- AKA: Test Automation System, Software Test Automation, Automated Testing Platform.
- Context:
- It can (typically) execute Test Cases through test scripts.
- It can (typically) validate Software Quality through automated verification.
- It can (typically) generate Test Reports through result analysis.
- It can (typically) maintain Test Environments through configuration management.
- It can (often) integrate with development workflows through continuous integration.
- It can (often) perform regression testing through automated retests.
- It can (often) support test data management through data generation.
- ...
- It can range from being a Simple Test Automation System to being a Complex Test Automation System, depending on its testing capability.
- It can range from being a Single-Layer Testing System to being a Multi-Layer Testing System, depending on its testing scope.
- It can range from being a Basic Test Runner to being an Advanced Test Platform, depending on its platform sophistication.
- ...
- Examples:
- Testing Levels, such as:
- Unit Testing Systems, such as:
- Integration Testing Systems, such as:
- End-to-End Testing Systems, such as:
- ...
- Testing Levels, such as:
- Counter-Examples:
- Manual Testing System, which requires human testers.
- Test Management System, which handles test organization.
- Development Pipeline System, which focuses on build processes.
- Quality Monitoring System, which performs runtime monitoring.
- See: Test Automation, Software Testing, Continuous Testing, Test Framework, Quality Assurance.
References
2022
- (Wikipedia, 2022) ⇒ https://en.wikipedia.org/wiki/test_automation Retrieved:2022-4-11.
- In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing. There are many approaches to test automation, however below are the general approaches used widely: * Graphical user interface testing. A testing framework that generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct. * API driven testing. A testing framework that uses a programming interface to the application to validate the behaviour under test. Typically API driven testing bypasses application user interface altogether. It can also be testing public (usually) interfaces to classes, modules or libraries are tested with a variety of input arguments to validate that the results that are returned are correct. One way to generate test cases automatically is model-based testing through use of a model of the system for test case generation, but research continues into a variety of alternative methodologies for doing so.In some cases, the model-based approach enables non-technical users to create automated business test cases in plain English so that no programming of any kind is needed in order to configure them for multiple operating systems, browsers, and smart devices.
What to automate, when to automate, or even whether one really needs automation are crucial decisions which the testing (or development) team must make. A multi-vocal literature review of 52 practitioner and 26 academic sources found that five main factors to consider in test automation decision are: 1) System Under Test (SUT), 2) the types and numbers of tests, 3) test-tool, 4) human and organizational topics, and 5) cross-cutting factors. The most frequent individual factors identified in the study were: need for regression testing, economic factors, and maturity of SUT. A growing trend in software development is the use of unit testing frameworks such as the xUnit frameworks (for example, JUnit and NUnit) that allow the execution of unit tests to determine whether various sections of the code are acting as expected under various circumstances. Test cases describe tests that need to be run on the program to verify that the program runs as expected. Test automation, mostly using unit testing, is a key feature of extreme programming and agile software development, where it is known as test-driven development (TDD) or test-first development. Unit tests can be written to define the functionality before the code is written. However, these unit tests evolve and are extended as coding progresses, issues are discovered and the code is subjected to refactoring. Only when all the tests for all the demanded features pass is the code considered complete. Proponents argue that it produces software that is both more reliable and less costly than code that is tested by manual exploration.It is considered more reliable because the code coverage is better, and because it is run constantly during development rather than once at the end of a waterfall development cycle. The developer discovers defects immediately upon making a change, when it is least expensive to fix. Finally, code refactoring is safer when unit testing is used; transforming the code into a simpler form with less code duplication, but equivalent behavior, is much less likely to introduce new defects when the refactored code is covered by unit tests.
Some software testing tasks (such as extensive low-level interface regression testing) can be laborious and time-consuming to do manually. In addition, a manual approach might not always be effective in finding certain classes of defects. Test automation offers a possibility to perform these types of testing effectively.
Once automated tests have been developed, they can be run quickly and repeatedly. Many times, this can be a cost-effective method for regression testing of software products that have a long maintenance life. Even minor patches over the lifetime of the application can cause existing features to break which were working at an earlier point in time.
While the reusability of automated tests is valued by software development companies, this property can also be viewed as a disadvantage. It leads to the so-called "Pesticide Paradox", where repeatedly executed scripts stop detecting errors that go beyond their frameworks. In such cases, manual testing may be a better investment. This ambiguity once again leads to the conclusion that the decision on test automation should be made individually, keeping in mind project requirements and peculiarities.
Test automation tools can be expensive and are usually employed in combination with manual testing. Test automation can be made cost-effective in the long term, especially when used repeatedly in regression testing. A good candidate for test automation is a test case for common flow of an application, as it is required to be executed (regression testing) every time an enhancement is made in the application. Test automation reduces the effort associated with manual testing. Manual effort is needed to develop and maintain automated checks, as well as reviewing test results.
In automated testing, the test engineer or software quality assurance person must have software coding ability since the test cases are written in the form of source code which when run produce output according to the assertions that are a part of it. Some test automation tools allow for test authoring to be done by keywords instead of coding, which do not require programming.
- In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing. There are many approaches to test automation, however below are the general approaches used widely: * Graphical user interface testing. A testing framework that generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct. * API driven testing. A testing framework that uses a programming interface to the application to validate the behaviour under test. Typically API driven testing bypasses application user interface altogether. It can also be testing public (usually) interfaces to classes, modules or libraries are tested with a variety of input arguments to validate that the results that are returned are correct. One way to generate test cases automatically is model-based testing through use of a model of the system for test case generation, but research continues into a variety of alternative methodologies for doing so.In some cases, the model-based approach enables non-technical users to create automated business test cases in plain English so that no programming of any kind is needed in order to configure them for multiple operating systems, browsers, and smart devices.
2022
- https://www.atlassian.com/devops/devops-tools/test-automation
- Test automation is the practice of automatically reviewing and validating a software product, such as a web application, to make sure it meets predefined quality standards for code style, functionality (business logic), and user experience.
Testing practices typically involve the following stages:
- Unit testing: validates individual units of code, such as a function, so it works as expected
- Integration testing: ensures several pieces of code can work together without unintended consequences
- End-to-end testing: validates that the application meets the user’s expectations
- Exploratory testing: takes an unstructured approach to reviewing numerous areas of an application from the user perspective, to uncover functional or visual issues
- The different types of testing are often visualized as a pyramid. As you climb up the pyramid, the number of tests in each type decreases, and the cost of creating and running tests increases.
- Test automation is the practice of automatically reviewing and validating a software product, such as a web application, to make sure it meets predefined quality standards for code style, functionality (business logic), and user experience.
2022
- https://www.testim.io/blog/test-automation-platform/
- QUOTE: A test automation platform is a tool or framework that helps automate your QA efforts. A test automation platform uses scripts to automate the whole process of software testing. This platform includes various inputs and various use cases that your software has to be tested for. It has a prewritten set of steps to simulate user actions.
A test automation platform can be designed to replay recorded and predefined actions. You can also design a test automation platform to compare the desired results with the results obtained while testing. Test automation platforms completely limit human errors and can run throughout the day, which should get you more time for testing. ...
- QUOTE: A test automation platform is a tool or framework that helps automate your QA efforts. A test automation platform uses scripts to automate the whole process of software testing. This platform includes various inputs and various use cases that your software has to be tested for. It has a prewritten set of steps to simulate user actions.