Performance Test Strategy and Approach

Project/Module context understanding:

The performance team will try to understand the project context, And learn about different use cases.
The performance team will start thinking about a performance test strategy.
The team should start creating the scenario keeping performance testing in scope.
Understand the test date needed to perform load testing.
Get business expected response time and number of users accessing systems.
Tentative development completion time.

Project/Module implementation walk through:

Dev team to help performance team to understand the technical implementation details.
Share the architecture walk through to help the performance team identify test data modeling.
API contract discussion.
API dependencies and access details.
Understanding infra details.

Performance test feasibility and challenges:

Based on functional and technical understanding, the performance team will have a discussion with the dev and communicate whether the required feature can be performance testable or not and also discuss the challenges.

Client and server detail:

The performance team provides client system configuration.This is required to generate the desired load.
Egov team to provide the required infra access (client and application server access)
Monitoring system access to check the API and System behaviour.

Performance testing objective and deliverables:

Based on functional and technical requirements, the Performance test team defines the concurrent load, duration of test execution, types of load testing and reporting detail.
Get approval on load modelling from the dev and business team.
Typical performance testing carries 3 types of testing, Load, Stress and Endurance. All 3 might not be needed all the time. The team can discuss what type is needed for the particular feature.
Define success criteria for application performance.

Performance test approach:

Typically we follow two approaches
- Independent API’s performance test or real-time load simulation
  - Identify the API to test as part of the feature
  - Identify dependency between the API’s
  - Remove dependency by setting up test data before the actual load test
  - This approach is recommended and also scaleable
  - Here each API’s throughput can be controlled effectively
  - Designing a test plan using this approach takes more time than the scenario wise simulation.
- Scenario wise performance test
  - A test plan will be created based on the scenario
  - If each API has a dependency with one and another, then throughput control is tough to manage
  - This approach is quicker and can be implemented with less effort
Based on the complexity and timeline, the performance team will follow any one of the above approaches and the same will be communicated to stakeholders.

Performance testing scope:

The current scope is to test the API performance. We can leverage the scope to test environment(POD) auto-scaling.
As mentioned earlier we would be doing Load testing, Stress, Endurance and Spike testing based on the module need.
The team will help run multiple iterations or share performance scripts with the dev team to improve code quality.
Server-side detailing such as CPU and Memory utilization.
With the help of the monitoring tool, the team can trace the higher response API’s and also debug the errors.
Test execution result (Jmeter reporting and Server report matrix obtained from monitoring tools).

Performance testing tool and environment:

Risk Analysis:

Overall performance test activities:

Reporting sample :

Avg response time for different user load

API’s error contribution for a different load

Aggregate report for different runs - Overall performance of an API