Log in to become a member of D v Suresh's Fan Club!
This section defines a defect Severity Scale framework for determining defect criticality and the associated defect Priority Levels to be assigned to errors found software.
The defects can be classified as follows:
Critical: There is s functionality block. The application is not able to proceed any further.
Major: The application is not working as desired. There are variations in the functionality.
Minor: There is no failure reported due to the defect, but certainly needs to be rectified.
Cosmetic: Defects in the User Interface or Navigation.
Suggestion: Feature which can be added for betterment.
Defect Priority
The priority level describes the time for resolution of the defect. The priority level would be classified as follows:
Immediate: Resolve the defect with immediate effect.
At the Earliest: Resolve the defect at the earliest, on priority at the second level.
Normal: Resolve the defect.
Later: Could be resolved at the later stages.
Test Metrics in software testing:
Metrics are the most important responsibility of the Test Team. Metrics allow for deeper understanding of the performance of the application and its behavior. The fine tuning of the application can be enhanced only with metrics. In a typical QA process, there are many metrics which provide information.
The following can be regarded as the fundamental metric:
Functional or Test Coverage Metrics.
Software Release Metrics.
Software Maturity Metrics.
Reliability Metrics.
Mean Time To First Failure (MTTFF).
Mean Time Between Failures (MTBF).
Mean Time To Repair (MTTR).
Functional or Test Coverage Metric . It can be used to measure test coverage prior to software delivery. It provides a measure of the percentage of the software tested at any point during testing.
It is calculated as follows:
Function Test Coverage = FE/FT
Where,
FE is the number of test requirements that are covered by test cases that were executed against the software
FT is the total number of test requirements
Software Release Metrics
The software is ready for release when:
1. It has been tested with a test suite that provides 100% functional coverage, 80% branch coverage, and 100% procedure coverage.
2. There are no level 1 or 2 severity defects.
3. The defect finding rate is less than 40 new defects per 1000 hours of testing
4. Stress testing, configuration testing, installation testing, Nave user testing, usability testing, and sanity testing have been completed
Software Maturity Metric
Software Maturity Index is that which can be used to determine the readiness for release of a software system. This index is especially useful for assessing release readiness when changes, additions, or deletions are made to existing software systems. It also provides an historical index of the impact of changes. It is calculated as follows:
SMI = Mt - ( Fa + Fc + Fd)/Mt
Where
SMI is the Software Maturity Index value
Mt is the number of software functions/modules in the current release
Fc is the number of functions/modules that contain changes from the previous release Fa is the number of functions/modules that contain additions to the previous release
Fd is the number of functions/modules that are deleted from the previous release
Reliability Metrics
Reliability is calculated as follows:
Reliability = 1 - Number of errors (actual or predicted)/Total number of lines of executable code
This reliability value is calculated for the number of errors during a specified time interval.
Three other metrics can be calculated during extended testing or after the system is in production. They are:
MTTFF (Mean Time to First Failure)
MTTFF = The number of time intervals the system is operable until its first failure (functional failure only).
MTBF (Mean Time Between Failures)
MTBF = Sum of the time intervals the system is operable
MTTR (Mean Time To Repair)
MTTR = sum of the time intervals required to repair the system
The number of repairs during the time period .
D V Suresh is the author of blog MICROSOFT DOT NET and SOFTWARE TESTING
» left by Steven from Germany (6 hours 20 minutes ago.) New Comment!
A very simplified and incomplete defect classification. You might want to consider the following Critical Situations:
Loss of or corruption of data. I am sure you would not be very happy if your bank details suddenly went missing or were corrupted.
Security Breach. What if it was possible to breach system security and access sensitive data.
Personal Injury or Death. What if the defect could lead to personal injury or even death.
In all the above cases the application is running but they are all critical cases, contrary to what you have defined.
To your Metrics:
The Mean Time Metrics relate to the effectiveness of the development process.
Mean Time to Repair The focus is how quick are the development team turning round problems. This cannot reflect in any way the quality of the application because the developer could turn problems around and the application still be defect.
Function Test Coverage what use is the test coverage if I do not know how many test cases are passed. 100 Percent coverage is not much use when 95 percent is not working. Functional Test Coverage is not just one metric but many. Overall Test Coverage, Test Coverage of all Passed Tests, Test Coverage of all Failed Tests, Test Coverage of all High Risk Test Requirements and so on.
Software Release Metrics I hope this is just an example, because if your working on a large application with hundreds of functional requirements, and your backs against the wall in terms of resources and time, then your not going to reach this goal. The focus will be amongst others on high priority, high risk test requirements with the low priority stuff being nice to have, informally tested and so on.
Disclaimer: All information on this site is provided for informational purposes only! By no means is any
information presented herein intended to substitute for the advice provided to you by any health care or other professional
or organization.
tweet this!
