Validating Cloud Software Tools
Do you remember which computing tools
you were using in 2002?
I would guess, if you were old enough to use computing tools, you had an IBM PC clone or were carrying a laptop that made your shoulder ache. You had a Motorola, Samsung, Nokia, or similar cell phone that was definitely not a “smart phone”. Your Internet access was from your PC / laptop using Internet Explorer, and your Internet content was millions of static web pages that loaded quickly enough if there weren’t too many images on the page. Contrast this with today’s tools and experience – iPhone, smart watch, tablet, and the ability to stream any content, anywhere, anytime, on any device.
So, what does 2002 have to do with anything, you ask? Well, 2002 was the date of the most recent guidance publication from the FDA on software validation. I cannot say whether the FDA had cloud computing in mind when they authored “General Principles of Software Validation; Final Guidance for Industry and FDA Staff”, but I can say that the guidance was carefully written to avoid referencing any specific technology or prescribing any particular computing techniques for validation and, because of this, the document is still relevant in today’s era of cloud computing. However, what is dated, from a technology standpoint, is how firms have applied the validation guidance from the FDA.
Current Validation Approach Incompatible with Cloud Computing
Validation of software is typically a costly, time-consuming process, with many firms spending 4-6 months to validate a specific software product. This has led to the most common validation approach being one of “locking down” the software being validated to prevent any version changes that could force having to re-validate the target software product. This approach limits firms to using only software that can be installed locally and that can be controlled by an internal IT staff. Cloud computing, by contrast, was invented to avoid the cost and complexities of installing and managing software locally. And, while there are still many software products being developed for local installation, the growing majority of software development and innovation is focused on cloud computing.
Before discussing how to take advantage of cloud computing while also following the FDA validation guidance, let me describe the significance of validation. As a person knowledgeable with regulated software, you probably understand or have read the difference between software verification and software validation, but, if not, search in your browser for “software verification vs validation”. Regarding validation, there are 3 distinct areas where validation is needed when building a medical device:
Validating software that is wholly a medical device or is part of a medical device
Validating software that is used to build/test/deploy medical device software
Validating software that is used in maintaining records related to the medical device
The focus of this article is primarily on validating software of the 2nd and 3rd types listed above, but the concepts are applicable to validating software that is part of a medical device, as well.
Figure 1 – Verification/Validation Evidence
Cloud Computing Tools and Validation
The current and upcoming generations of software developers have grown accustomed to cloud-based tools for developing software, as evidenced by the popularity of tools/systems like github.com, bitbucket.com, Jira, and the tools/suites included in cloud vendor products like Amazon AWS, Microsoft Azure, and Google Cloud. If you hire a developer with cloud tool experience and ask them to use your internal, locked-down software development tools don’t be surprised if you find them passively protesting and using their favorite cloud tool for the majority of their work, and occasionally syncing up their work into the internal tool. This is not ideal from a productivity, developer morale, or source code confidentiality perspective, so, it is best to find a way to incorporate the best of breed tools (i.e. cloud computing tools) into your validated tool set.
One of the main benefits of cloud computing tools – that someone else controls the installation, updating, and maintenance of the software – is the biggest impediment to overcome when validating cloud software. The cloud vendor has complete control of when new software versions are introduced, generally providing little or no notice to the end-user of the cloud software when a new version is released. Based on this inability to lock-down the software, there are two general approaches for validating cloud software tools:
Continuous validation
Periodic validation
Continuous validation, as the name suggests, is validation that occurs on a continuous basis and is characterized by having automated tests that can be executed or triggered on-demand and can be executed in a short timeframe (e.g. minutes or hours). The FDA guidance on validation does not state how testing has to be performed, as in manual or automatic test execution. The guidance specifies only that testing evidence is generated to prove that all tests pass.
Periodic validation is differentiated from continuous validation in that it typically takes longer to execute, likely due to some portion of the validation being done through manual testing, and is scheduled to occur on a regular, but not continuous basis. Periodic validation would be suitable for situations where the cloud software validation needs to be updated based on a significant event, such as releasing a new version of medical device software that the cloud software was used to create, to prove the tool used to create the medical device software is still functioning as expected.
A key element of validation for both continuous and periodic validation is that requirements exist that document the user needs for the software being validated. The tests – be they automated, manual, or through code inspection – need to test and prove that the software is meeting the defined requirements.
Criteria for Selecting a Validation Approach
Selection of continuous versus periodic validation depends on several factors. One factor that would drive selection of continuous validation would be that the software under validation is wholly or part of a medical device. For example, if medical device software is built using cloud computing, then it is not sufficient to only periodically (re-)validate, as cloud software version changes can occur at any time and such changes, if not validated immediately, could have an adverse effect on the medical device’s behavior if a breaking change in the cloud software occurred. Likewise, if the cloud software under validation is responsible for managing recording keeping data about a medical device, an automated approach would be preferable to a periodic approach, so that there is never any risk of compromise to the integrity of the records due to a cloud software change.
Another factor in determining the validation approach is the technical skills and technical feasibility of writing completely automated tests of cloud software. If your company does not have software developers on-staff who can write automated tests of cloud software, but does have verification testers capable of writing manual tests, then periodic validation could be a logical approach. This assumes that the cloud software to be validated is not part of a medical device, as described in the previous section, such that it would require continuous validation.
A third factor in deciding the validation approach is cost, in terms of the cost to develop the initial test suite and the on-going costs to re-execute the validation test suite. Generally, creating automated tests can have a higher up-front cost, but will have little or no cost of re-executing the tests, other than small on-going costs for the automated testing infrastructure. Developing manual tests that are part of periodic validation can have a lower up-front cost, but, given the human labor costs to re-execute, can be more costly in the long-term, depending on how frequently re-validation occurs.
Conclusion
Adoption and use of cloud computing for software development, including for medical device software, is here to stay. Fortunately, and due to the foresight of the validation guidelines provided by the FDA, use of cloud computing for medical devices software is possible, so long as risks are evaluated, and an appropriate validation strategy is established.
Firms like the RND Group are familiar with use of cloud computing tools and the validation of such tools. RND Group can provide advisory or complete out-sourced services for validating both cloud-based and non-cloud-based software tools. Please contact RND Group today for any questions or to discuss how we can help your company.