Are you really testing what the requirements specify?
In this blog article, you will learn why it is crucial that those who have formulated requirements also design and execute the corresponding end-to-end tests.
1. Traditional role allocation
In many projects, the assumption still prevails: “Whoever writes the code also tests it. ’ At first glance, this sounds logical - after all, developers know their own implementation best. know their own implementation best. However, this approach harbours a number of stumbling blocks.
Developers usually look at the system from a technical perspective. They focus on whether methods are executed correctly and interfaces are implemented properly. What is often neglected is the end user’s perspective: what steps do they really need to go through? what errors do they need to see and how does the system behave in complex everyday situations?
The original requirements, formulated by the product owner, contain important boundary conditions and business rules. As developers have not defined these use cases themselves, they often lack a deep understanding of them, why certain processes have to be like this. Developers therefore often only test the technical process, but not the complete user workflow, and therefore overlook critical details of the requirement.
Unit and integration tests are ideal for validating individual code components. However, they are only suitable for end-to-end scenarios. If a team tries to have E2E tests maintained by developers alone by developers alone, fragile test scripts are created that break with every UI change and cause high maintenance costs.
Example: Discount calculation
Requirement (PO):
Premium customers automatically receive a 10% discount from a shopping basket value of €50, without entering a voucher code.
- Developer test:
Only checks whether a voucher code passed via API applies a discount and whether the server returns HTTP status 200 is returned. - Understanding gap:
The test does not check that the discount automatically applies to premium customers, because the developer has not fully internalised the business model (premium status, minimum order value).
2. product owners as end-to-end test experts
Product owners have in-depth domain knowledge and fully understand the processes completely from the user’s perspective. They are familiar with all relevant business processes, rules and exceptions and can derive precise test scenarios from this. As they have formulated the acceptance criteria themselves, they are able to clearly describe every desired process and every edge case.
Example:
A product owner specifies that not only a confirmation email is to be sent after the order has been completed, but also that billing must be initiated with an external service provider.
Advantages when product owners take over the E2E tests:
- Tests cover exactly all the acceptance criteria defined in the specifications.
- The tests reflect the actual user experience.
- The coordination effort between the specialist department and development is minimised.
3. The challenges of manual end-to-end tests
Although product owners know exactly what needs to be tested, manual end-to-end tests often prove to be time-consuming and error-prone in practice. Every single test run, from preparation to execution and evaluation, ties up valuable resources and can quickly become a bottleneck with frequent release cycles.
In addition, manual processes can easily lead to inconsistencies. Typing errors, forgotten steps or different interpretations of test scripts can lead to that test results vary and problems remain undetected.
In addition, maintenance of manual tests is a major challenge. Even minor changes to the user interface or workflow require an adjustment of all affected test descriptions and all affected test descriptions and documentation. Without automation, the test portfolio grows with every release, becomes increasingly confusing and loses its reliability over time.
All in all, this means that manual end-to-end tests can be comprehensive in theory, but are hardly scalable or sustainable in practice.
4. Automation of end-to-end test creation as a solution
In practice, it is often the case that even modern test generators can only fulfil part of the defined standard click paths and obvious scenarios are recognised, but important edge cases and complex business rules often remain untested. In the end, the automated script runs green without really checking whether all the defined acceptance criteria have been met.
This is where consistent automation of test creation comes in. Instead of formulating tests manually or relying on black box generators, the acceptance criteria are translated directly into precise E2E test cases. Such an approach ensures that exactly what is actually in the requirements catalogue is validated, and that nothing essential is lost. At the same time, there is no need for time-consuming manual adjustments when UI changes are made, as the tests can be dynamically regenerated from the can be regenerated dynamically from the current requirements. The result is a scalable, low-maintenance E2E test suite, that reliably covers what the product owner has requested.
Conclusion
Reliable end-to-end testing requires the expertise of the requirements owner. Only if acceptance criteria are translated directly into test cases, no important use case remains untested. Automating this translation creates a scalable, low-maintenance test suite and ensures that exactly what is written in the requirements specification is tested, without any additional manual effort.