Performance Test
Test Objectives
The most frequently used modules by users are Testing and Pipeline. IDA needs to be able to handle continuous requests and provide the industry-standard level of service in terms of response time and error rate. The average response time should be less than 2 seconds, and the error rate should be less than 1%.
We expect IDA to have the following capacity:
- 2 core CPU, 4 GB memory (Concurrent Users 10)
- 4 core CPU, 8 GB memory (Concurrent Users 20)
- 8 core CPU, 16 GB memory (Concurrent Users 40)
Test Scenario
This scenario mainly tests the user’s usage of Testing and Pipeline.
User actions in the scenario:
- Login
- Save Project
- Open Project Edit Form
- Edit Project
- Generate Cases
- View Project Dashboard
- Get Test Case Id
- View Test Case
- Edit Test Case
- Get Project Config
- Get Command Parameters
- Get Command Category
- Get Command Options
- Save Case Step
- Get Test Cases
- Exit Edit Case
- Update Check
- Add Pipelines
- View Pipelines Dashboard
- View Pipeline List
- Get Pipeline Id
- Run Pipelines
Test Results
Test duration: 1 hour;
System | Concurrent Users | Total Requests | Requests/s | Response Min (ms) | Response Max (ms) | Response Mean (ms) | Std Dev (ms) | %CPU | %MEM |
---|---|---|---|---|---|---|---|---|---|
2 core CPU, 4 GB memory | 10 | 24521 | 6.578 | 268 | 4538 | 664 | 507 | 68.7% | 47.4% |
4 core CPU, 8 GB memory | 20 | 32326 | 8.59 | 279 | 29919 | 1477 | 1776 | 69.7% | 40.5% |
8 core CPU, 16 GB memory | 40 | 59101 | 15.59 | 284 | 38860 | 1694 | 2486 | 65.5% | 36.3% |