Welcome to the second post in a series on Computer System Validation.
In our previous entry, we gave an overview of the fundamentals of computer validation and the system development lifecycle (SDLC).
During the development or upgrading of Production-grade systems (particularly regulated ones which contain financial, medical, proprietary or other critical data), a robust validation and testing plan must be kept current and followed. Failure to do so can lead to massive fines as just a start and continue to damage a company’s reputation or brand, earnings, and many other sensitive areas.
See Paragon's first blog in this series: What Is Computer System Validation? The Fundamentals
Good computer system validation practices start with a plan and virtually always include the execution of test cases which ensure that the system is adhering to the approved requirements.
In today’s piece, we will look at a few snippets of imaginary test cases for the upgrade of an ATM’s operating system and how they fit into an overall testing initiative.
To recap, there are generally at least three phases of validation testing on a given project:
- Installation Qualification provides instructions to ensure that the system is installed correctly in the target environment(s).
- Operational Qualification verifies the system’s functional requirements. These are generally quite detailed and touch every aspect and functionality of the system covering both positive and negative conditions.
- Performance Qualification tests the user requirements and often follows predefined use cases or real world scenarios that have been created by the client.
An additional phase of testing that we will look at below is Regression Testing, which verifies that previous system functionality (i.e. not something which is a part of the current upgrade or development) continues to perform as expected with the addition of the new features. For the sake of readability, the sample test case tables below will be simplified and streamlined. The Test Instructions and Expected Results columns will be merged into one, and the Test Step # and Actual Results column will be omitted.
Sample IQ requirement: The system shall run on version 5.2 of the “CashPlus” operating system.
Sample user requirement 1: The system will dispense cash following authorized transactions.
Sample user requirement 2: The system will dispense cash in the form of US dollars.
Sample functional requirement 1: The system must require a valid ATM card to grant a user access.
Sample functional requirement 2: The system must require a specific four-digit personal identification number to grant a user access.
Sample functional requirement 3: The system will display the error message: “Invalid PIN. Please try again.” if an incorrect PIN is entered.
In our upgrade scenario, we are going to assume that the modifications to the ATM software will only impact the withdrawal functionality and not anything regarding deposits. However deposits are a critical part of ATM transactions and therefore must be part of the regression test case.
Finally, we will take a brief look at negative testing, which is generally performed during the Operational Qualification phase. In the functional requirements above, it is stated that the system will grant access if a valid card and PIN are entered. But there are always a number of ways that a bank customer could use an ATM that do not follow a straight A to B path. It is the responsibility of the test manager to anticipate scenarios where a user will attempt (maliciously or not) to make the machine to do something that it is not intended to do.
Although extremely short and simplistic compared to the actual verification which would be necessary for a financial system, the examples should give an introduction to the thought behind the testing process. In addition to merely meeting the stated requirements, it is essential that any testing initiative also considers regression and negative testing to ensure that our hypothetical ATM system not only performs as expected, but is not exploitable by anyone.