The FDA wants your software to work right, and if you're in a regulated environment, they have the force of law behind them. But what do you do when you want to apply modern, "agile" software development methodologies in a validated environment? And how do you get regulatory to believe you when you tell them that it's all going to be ok?

The FDA, of course, is charged with ensuring that products under its jurisdiction are both safe and effective. FDA regulations, including those that define computer validation, stem from the goals of safety and efficacy. In fact, the FDA defines systems validation as follows:

Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes.

Note the critical aspects of that statement: the FDA wants clear-cut documentation that shows that you have a process, that you follow it, and that the process will produce a quality product. In fact, that statement can be applied to drug compounds, medical products, and safety documentation just as easily as it can be applied to software.

The FDA has a history of providing high-level objectives and then permitting industry to find the best way to meet those objectives. For instance, the regulations in 21 CFR 11, which define the requirements for electronic records and electronic signatures, do not specifically address individual products or protocols. Instead, the FDA describes the qualities of a compliant system, and then lets industry demonstrate how specific systems possess those qualities. This approach permits the FDA to maintain safety and efficacy standards without micro-managing the free market, and it has worked fairly well.

In fact, the FDA's validation definition is so high-level that there would seem to be wide leeway in how software might be developed, and in fact this is true. However, one piece of that definition is problematic:

will consistently produce a product meeting its predetermined specifications and quality attributes.

Yes, the FDA assumes that, whether you're producing pills or programs, you have predetermined, clear-cut specifications.

Now, this makes absolute sense in most cases. Of course we want that pill's makeup to be clearly defined before it's made -- how else will the FDA know what goes into that pill? Even in software development, this approach makes sense under most methodologies, including the waterfall methodology that is popular at most pharmaceutical companies. Indeed, Edward Yourdon's The Decline and Fall of the American Programmer, one of the "bibles" of the waterfall development methodology, was published at about the same time that the FDA was becoming aware of the need for computer systems validation.

So what's wrong with following a waterfall methodology? In many cases, nothing at all: the waterfall methodology has many strengths, including a very predictable sequence of steps that produces a very concrete and well-defined set of deliverable documents.

However, as the state of the art of software development has moved forward over the last ten years, those of us in the field have come to realize that the waterfall methodology (and other "heavy" methodologies that depend on a complete understanding of requirements up front) has significant weaknesses. To quote Martin Fowler, Chief Scientist at ThoughtWorks:

Methodologies impose a disciplined process upon software development with the aim of making software development more predictable and more efficient. They do this by developing a detailed process with a strong emphasis on planning inspired by other engineering disciplines.

These methodologies have been around for a long time. They've not been noticeable for being terribly successful. They are even less noted for being popular. The most frequent criticism of these methodologies is that they are bureaucratic. There's so much stuff to do to follow the methodology that the whole pace of development slows down. Hence they are often referred to as heavy methodologies, or to use Jim Highsmith's term: monumental methodologies.

As a reaction to these methodologies, a new group of methodologies have appeared in the last few years. For a while these were known a lightweight methodologies, but now the accepted term is agile methodologies. For many people the appeal of these agile methodologies is their reaction to the bureaucracy of the monumental methodologies. These new methods attempt a useful compromise between no process and too much process, providing just enough process to gain a reasonable payoff.

The result of all of this is that agile methods have some significant changes in emphasis from heavyweight methods. The most immediate difference is that they are less document-oriented, usually emphasizing a smaller amount of documentation for a given task. In many ways they are rather code-oriented: following a route that says that the key part of documentation is source code.

However I don't think this is the key point about agile methods. Lack of documentation is a symptom of two much deeper differences:

• Agile methods are adaptive rather than predictive. Heavy methods tend to try to plan out a large part of the software process in great detail for a long span of time, this works well until things change. So their nature is to resist change. The agile methods, however, welcome change. They try to be processes that adapt and thrive on change, even to the point of changing themselves.
• Agile methods are people-oriented rather than process-oriented. They explicitly make a point of trying to work with peoples' nature rather than against them and to emphasize that software development should be an enjoyable activity.

The rest of Fowler's essay is available at http://www.martinfowler.com/articles/newMethodology.html#N401.

So permit me a leap of logic and let's assume that we agree that agile methodologies can help us produce better software for the end users. How then do we reconcile ourselves with the FDA's definition of validation? It clearly states that specifications are predetermined, yet the very heart of the lightweight methodology is the concept that a clear understanding of requirements is achieved throughout the development cycle.

The primary thought-shift that makes this reconciliation possible is this:

Writing code is part of the specification process, and the software itself is part of the documentation.

If you're a regulatory affairs specialist or someone else who is responsible for ensuring quality in software, you probably fell over in your chair as visions of cowboy programmers running rampant danced in your head. For many programmers, documentation is an evil chore to be avoided at all costs. For many regulatory specialists, project managers, and business people, documentation is the only tool with which to try and understand just what the heck it is those crazy programmers are up to.

Agile methodologies don't dispose of documentation. Instead, they recognize that a three-inch thick functional requirements specification may not be the optimal tool for programmers and users to use in coming to a common understanding of the system requirements, and instead attempt to use the software itself as the tool.

How is it possible that the software can be a tool in its own creation? Through an iterative process. This is the basis behind Joint Application Development (JAD) and Rapid Application Development (RAD) methodologies: involve the users early and often, and give them something they can touch, use, and react to. This is the lifeblood of an agile methodology.

This is all very well and good, but we need concrete steps to be able to make this fly in a validated environment. The FDA is concerned with process and product: you can define your own process, but you'd better follow it and it had better produce a quality product.

So how would we go about it?

• First, develop a Standard Operating Procedure (SOP) that describes your particular agile methodology. It needs to clearly define the process in detail, including the deliverables that come out of each part of the process. Include the software as a specification and design deliverable, not just as the final output of the process. Have the SOP reviewed and approved before making a significant commitment to an agile methodology.
• Produce the same set of deliverables as a more traditional waterfall methodology would produce, including a work plan, requirements specifications, design documentation, a test plan, unit test scripts and system test scripts. You still need all of these things to pass muster; agile methodologies are not an excuse to eliminate documentation.
• Clearly define how each of those deliverables is built. In a waterfall, the requirements specification comes at the beginning and becomes immutable as the project moves forward. In agile methodologies, the initial requirements specification is developed the same way, but then is modified as the system moves forward through design and development.
• I encourage lots of screen shots and process flows in the requirements and design documentation, harvested from user sessions and prototype software.
• Pick a point at which the requirements and design really do become frozen and subject to rigid change control. In an agile methodology this is going to be significantly further along in the process, but it's not at the end. Depending on the size of your project, it?s going to be a few to several weeks prior to system testing.
• Pay special attention to your test plan. Testing should occur throughout the entire lifecycle of the project, and needs to be documented thoroughly, even when the testing is automated.
• Take the time to explain agile methodologies to your regulatory specialists and work hard to gain their understanding and agreement, because the burden of proof will be on them and you if an FDA auditor decides to take a peek under the hood.

The end goal of every methodology is the same: to produce a quality software product. Agile methodologies accomplish this goal with greater flexibility and user input, often resulting in shorter development times, fewer defects, and greater user satisfaction with the end product. By taking the time to understand the FDA's objectives and applying them to agile methodologies, you can develop better software in a validated environment.

2008 addendum: I originally wrote this article in 2002, when agile methodologies were still relatively new.  Since then, methodologies like Scrum and Extreme Programming have become popular in pharmaceutical companies, and my company alone has developed a half dozen major validated systems using this approach.  Every once in a while another practitioner of the engineering arts will challenge the points I've made in this article in a good-natured way, especially with regards to development of hardware or medical devices - prototyping is much more expensive and time-consuming in the hardware world.  If you've had an experience with using agile methodologies in a validated setting, good or bad, feel free to This e-mail address is being protected from spambots, you need JavaScript enabled to view it and let me know how it went.

Copyright (C) 2002, 2008 by Frank Jacquette. Permission to redistribute this document is granted provided that it is unaltered and attribution, including this copyright notice, remains intact.