This blog is targeted at those companies that aspire to advance their software development model from waterfall to agile.
When a company decides to transition from Waterfall to Agile testing, address the following questions:
- What are the most important areas to concentrate on for effective Agile Testing?
- How does Testing in Agile compare to that of Waterfall model?
- What set of activities are important for testers to know and do?
Testing Throughout the Development
The first thing to understand is that in agile development, testing is integrated throughout the lifecycle; testing the software continuously throughout its development.
In Waterfall model, testing is a large effort and is left towards the end of development, whereas in Agile, testing is small but more frequent and happens throughout the development.
Testing throughout the development also means that the software is in releasable condition throughout the development, so it can be shipped whenever it’s appropriate.
In Waterfall model, we are taught to think in phases, such as design phase, development phase and testing phase. Agile development does not have a separate test phase as such. Developers are much more heavily engaged in testing, writing automated repeatable unit tests to validate their code.
Developer Participation in Testing
With automated unit tests, testing can be done as part of the build, ensuring that all features are working correctly each time the build is produced. With a solid foundation of good unit test coverage, developers will feel more confident to refactor code as well.
Testing in Agile also means starting early. That means the QA would have to be involved right from the requirements stage, understanding the features and stories and start preparing and even writing tests upfront.
Another important aspect is Test Automation to be able to execute tests continuously as the product is being developed. This is not only unit automated tests, but API and UI automated tests as well.
Integrated and Cross Functional Teams
Transitioning to Agile is cross-functional project team activity. This joint effort is not limited to the testing activities. The developers should assist with testing frameworks and create features, the business analysts assist with refining the story.
Each team member works on a story until the entire story is done, that means developed and tested. Designers, Developers and Testers work together in parallel so achieve a common goal and they all should know what is necessary to get things done.
Performing as a team is the main key point transitioning from Waterfall to Agile Testing. The Company can decide to change to Agile Testing but the people have to support this change to succeed.
Note: There is no test team in agile.
Quality Mindset, Whole-Team Approach
Aim for pro-activeness by defect prevention rather than re-activeness by defect detection.
With early participation of testers in the project, they can assist in identifying key scenarios that are required to test a story. Quite often the acceptance criteria is written as a joint effort between a Product Owner, a Developer and a Tester – the Three Amigos.
This ensures that whatever is being built is testable and understood by all stakeholders. Also, as more people are involved in defining the acceptance criteria and the “Definition of Done”, mistakes can be rectified earlier and ultimately the right product is built right.
Note: Everyone is involved and responsible for quality of the product.
Less documentation, More collaboration
In Agile development, there is more emphasis on conversation and collaboration to clarify requirements more than the traditional approach of specifications and documentation.
Although requirements can be clarified to some extent in agile development, it is still possible for requirements to be ambiguous and incomplete, and for team members to have different understanding of the requirements.
So what does this mean for an Agile Tester? A common concern for testers transitioning to Agile development is that they don’t know precisely what they’re testing for. They don’t have a detailed spec to test against, so how can they possibly test it?
You don’t need to have detailed documentation to get started with testing. Many times, decent testers can use their judgement and common sense to validate a product. Domain knowledge becomes crucially important.
Testers should be confident to work more from their own knowledge of what good looks like. It’s certainly not just a case of following a test script, making sure the software does what it says in the spec.
How to setup a QA function from scratch?
It is a usual scenario: a startup company has a new idea and hires a number of developers to build a working model of the idea.
Because of the nature of the startups, i.e. not much funding available with a short scale of time to develop the idea, the main effort is focused on building the new product to get it out to public to test the waters, and so naturally, testing and QA is not the top priority for the development team.
After it becomes evident that the idea has been a success, the company wants to expand on the idea and starts hiring more developers, but at the same time, they also want the product to be tested before it goes out to public.
For a while, testing is done by whoever is available in the company, and it is largely ad-hoc with no proper processes to follow.
Then there comes a point when the startup company decides to hire their first senior QA person to start implementing a new QA process for the development team.
Lets consider that the startup is an e-commerce company.
Implementing a QA Process
The main aim of having a Quality Assurance process is to ensure that the right product is built right, the first time. That means, we need to ensure that the requirements are correctly defined and the development team have a solid understanding of the functionality of new features before starting to code.
It is important to note that testing is not a phase, it’s an activity and that testing starts from the very beginning of the development process, right from when the user stories are written.
Testing should support development and so testing activities are in parallel to the development activities, and at every stage of the development process we need to ensure that the code is thoroughly tested.
Before implementing a testing process, we need to know the current development methodology and process and if necessary make adjustments to improve the process.
Regression Testing / Sprint Testing
As you start as the first QA person in the company, chances are that there is no regression testing in place and so as new features are developed, you have no idea if they adversely affect the current working website. Moreover, you need to keep up with the development team to test the new features to ensure they function properly and according to specifications.
There are at least two tasks in parallel: Testing of new stories in the sprint and performing some degree of regression testing.
Testing of the new features takes priority as there is more likelihood of finding bugs in new code than breaking the current working site. But, at the same time, Regression Testing is required to ensure that the existing application keeps functioning as we build new features.
A regression testing pack needs to be executed as soon as there is an update to the application, so development team can get fast feedback on the health of the application.
There is not enough time to write regression tests as well as keeping up with testing of new features. How can we break this cycle?
Usually, during the first few days of the sprint, developers are busy coding and so the new features won’t be ready to test for a while. Here is a good chance to start working on the regression tests.
There are best practices for regression testing, but generally, the approach would be to identify the main core user journeys throughout the website, so that on each new release of the website, we can be confident that the application is still usable by majority of its users.
There doesn’t need to be an exhaustive list of these scenarios, just the main and most important ones will be sufficient to start a small regression pack that can be executed on every build. Later, as the regression pack matures, we can begin to add more scenarios.
Most importantly, these regression scenarios should be automated.
In an agile project, where a sprint typically lasts about two weeks, there is not enough time to do all the testing manually. There is testing of new stories as well as regression testing. While it makes sense to do exploratory testing to test new features, regression tests should be automated to reduce the mundane task of repeatedly executing the same tests manually.
Deployment / Build Pipeline
A deployment or build pipeline in an agile project defines how a story gets from product backlog to live production site. It defines a process and the activities that happen at each stage.
In order to implement a successful QA process that ensures we are frequently releasing quality code, the deployment pipeline must be defined and be adhered to by all stakeholders. The deployment pipeline is the spine of software delivery.
The pipeline should be based on best practices and encompass the activities that occur at each stage.
One of the most important activities in an agile project is frequent story workshop sessions. This is when the product owner, developers and testers gather in a room and start elaborating and fleshing out the details of the stories. This is important because everyone should have the same understanding of the story before starting the development work.
Quality Assurance is about defect prevention rather than detection and so in the story workshops, the team get the chance to ask questions about the details of the story, any technical or design constraints and any blockers to developing the stories.
Here is a great opportunity to start writing out the acceptance criteria for the stories. Everyone should contribute and start thinking about the possible scenarios for each story, as each one will have a different idea, so the more heads on the story, the more scenarios can be thought of and the higher chance of preventing defects getting live.
Once everyone is certain on the detail and scope of each story, development starts.
Developer Testing / Testing During Development
Everyone should be responsible for quality of the product and not just testers. As such, there needs to be sufficient amount of “developer testing” to ensure that the code written is of high quality before being deployed to a test environment for further testing.
Certainly each new piece of functionality should be well unit tested. On top of that, there needs to be integration tests, API tests as well as UI tests.
Peer code reviews or “buddy testing” can put a second eye on the work of the developer. A tester can help in reviewing the unit tests and well as the API tests to ensure correct tests have been written, as well as helping to write the high-level automated UI tests.
Read more on Best Practices for Continuous Testing in Agile projects.
Continuous Integration / Test Environments
In order to effectively test new features, we need to ensure the code works not only on the developer’s machine but also on other environments, and integrated with other developer’s code.
Continuous Integration helps in identifying any build problems early on in the process, so that when the deployment fails we can start to look where the issue is coming from.
Test Environments give testers and other team members the chance to test the new features before going live.
When required, we should also perform non-functional testing, such as performance, load and security testing. Quite often the focus is on ensuring the functionality works well, however non-functional testing should be given the same priority, especially for web applications as they could be subjected to heavy load and / or attacks.
By performing non-functional testing, we can be sure that our application can handle load during peak times and that is not open to security threats.
Other points to consider for setting up a QA function from scratch:
- Cross-browser, Cross device testing
- Mobile and Tablet Testing
- Parallel execution of automated tests
- Exploratory Testing
- Tools, such as Jira, Jenkins, Selenium, etc…
- Continuous Improvement
- Recruitment of Testers