SOFTWARE IN PRACTICE
(Alias: physical proof)
One test result is worth 1000 expert opinions.
Objective evidence is any documented statement of fact, other information or record, either quantitative or qualitative, pertaining to the quality of an item or activity, based on observations, measurements or tests which can be verified.
Examples of Objective Evidence
Using Objective Evidence in IT Audits
The objective of an audit is described by ISO 90011 clause 8.2.2 Internal audit:
"The organization shall conduct internal audits at planned intervals to determine whether the quality management system a) conforms to the planned arrangements (see 7.1), to the requirements of this International Standard and to the quality management system requirements established by the organization, and b) is effectively implemented and maintained."
Making Audit Findings Credible
Any anomalies identified by an audit are documented in non-conformance reports. For a non-conformance report to be credible it must be backed up by objective evidence (refer example in glossary item: Non-conformance Report). Anomalies that lack credibility are unlikely to be followed up by management thus defeating the main purpose of an audit, that is: to identify problems and trigger corrective action on the quality management system.
Getting to the Truth
When conducting audit interviews, wherever possible, professional auditors seek out objective evidence in support of the assertions of auditees.
Example: Auditor: Did you test the system? Auditee: Yes. Auditor: Sounds good, can you please provide me with your test plans, test designs, test cases, test results, test reports together with records of the resolution of all test anomalies identified.
Case study: An IT auditor reviews the design process in a software development shop. In his audit report he concludes that: "... the design process is inadequate and produces poor quality design descriptions." In the absence of objective evidence to back up this statement these assertions are open to dispute as they are based on one individual's personal opinion. In this case it is unlikely that any corrective action would be taken by management because the auditor's pronouncement lacks specificity and proof.
A more professional approach would have the auditor providing objective evidence in support of his conclusions. For example he could provide measurements of the degree to which: