Effective Risk Assessment

Risk assessment is a fundamental part of quality assurance. When used proactively, it can detect errors before they occur both in products and processes. At its core, risk assessment is a formal method for identifying risks and making business decisions whether to address them. In practice, risk assessment is a boring chore that is often neglected in many organizations. The focus of this article is to assist the regulatory affairs professional in identifying when risk assessment should be done, how it can be completed, and what role the regulatory affairs professional has in risk assessment.

When to Perform Assessments
Risk assessments should be done during new product development or after significant process changes. In both of these situations the product or its method of manufacture is no longer clearly understood and a systematic method for reviewing and assessing its risks is appropriate.

In new product development, risk assessment should be completed before the first product specification has been completed. Typically, when the first product specification is approved, development quickly proceeds to verify and validate the design and design changes become more difficult to implement. As an example, silicone liquid injection molded medical devices often have an upper specification limit on nitrosamines. If this is known prior to completing the product specification, then heating processes to eliminate the contaminants can be developed and implemented before the complete manufacturing process is finalized.

Risk assessments should also be done or re-examined prior to the initiation of a clinical trial. These are often significant project milestones, and the severity of the application justifies revisiting risk assessments. Particularly for complicated designs, it is important to continually inject learning derived from bench testing, pre-clinical trials, or early human feasibility studies into the risk assessment process. If it is discovered during a feasibility study that the device is incorrectly used by the physicians, then the labeling may be inadequate and should be revised since labeling is often used as a control during risk assessment.

Risk assessments are also appropriate prior to the initiation of substantial changes to manufacturing processes. Examples of major process changes would be relocating the process to a new facility or a different area of the same facility, technology upgrades, or new software installation. Moving a process usually includes a validation master plan, equipment qualification, and the training of new supervisors and operators. The probability for tribal knowledge to be lost is larger, so a risk assessment at this point would be prudent to ensure that all of the quality and process controls are still in place and effective after the relocation. Technology upgrades would include moving to a new sealing technology (radiofrequency to heat, for example) or upgrading hard coded logic with programmable controllers. These are essentially new processes to the business and the old controls and inspections may no longer be appropriate. Finally, new software should almost always have a risk assessment. The software industry is still in the stone age from a quality perspective. Indeed, many software vendors release error – ridden codes as a matter of course figuring they will fix them in the next release. For the software managing or verifying product quality, this is unacceptable and a risk assessment is fundamental to assuring the quality of the software.

Types of Risk Assessment
Three types of risk assessment are commonly used to manage risk: the informal assessment, failure modes and effects analysis (FMEA), or fault tree analysis(FTA). Informal assessments are the catch-all category much risk assessment falls into. It itemizes risks and compares them to industry standards or generic categories of cost, safety, quality, and customer satisfaction. For example, for the risk assessment of a heat sealing technology for a sterile product, the risks may be informally addressed by:

Risk

Safety

Quality

Cost

Customer

Service

Overall

Product Unsealed

Major

Minor

Major

Heat Sealer Breaks

None

Moderate

Major

The informal process is typically performed by a few people or even one person. Since it does not systematically address every possible failure, some judgment is used as to which risks to include or exclude. Useful for very simple changes, the informal method is not really appropriate for more complicated risk assessments.

The failure mode and effects analysis is probably the most common formal risk assessment used today. Pioneered by the automotive industry, FMEA has spread to many FDA regulated industries and is recognized by most auditors. FMEA is a bottom-up approach, where every step in a process or feature of a product is listed and its possible failure modes identified. Each failure mode is then assigned a severity on a scale from 1 to 10 with 10 being the most severe. The methodology then proceeds to identify possible causes for each failure mode and assign them a probability from 1 to 10 with 10 being highly likely. After this, the team usually looks at the current process controls to determine how easy it is to detect each failure mode. The detectability for each failure mode is also estimated from 1 to 10 with 10 as perfectly detectable with the current inspection systems or controls. An overall risk priority number (RPN) is then calculated by multiplying the severity, probability, and detectability. With risks then running from 1 to 1000, a cut-off is established and risks above the threshold are addressed with a combination of design changes to reduce frequency or controls to improve detectability to bring down the overall RPN. An example of a FMEA for the previous heat sealer would be:

Feature	Failure Mode	Severity	Possible Causes	Frequency	Controls	Detectability	RPN
Heat sealer platen	Platen too cool to seal bags	10	Platen damaged	9	Preventive Maintenance Schedule.	5	450
Heat sealer platen	Platen too cool to seal bags	10	Defective part	2	Temperature verified prior to use.	5	100
Heat sealer platen	Platen too cool to seal bags	10	Dirty	7		5	350

Therefore, if the cutoff was established at 400, then actions would be taken to reduce the probability of the platen being damaged with sealing or improve the detectability. This could include a visual inspection of the platen condition prior to sealing, or designing a fixture to hold the bag so as to protect the delicate part.

The disadvantage to FMEA is that they can be extremely time consuming. While very thorough, the multidisciplinary nature of the assessment can absorb a tremendous number of resources to complete a relatively routine product assessment. Despite its shortcomings, its popularity for risk assessment is justified as it is a solid methodology for managing risks.

Another type of risk assessment not seen as frequently is fault-tree analysis. More common in reliability engineering, FTA is a top-down approach that first looks at system failures and tries to decompose the failures into individual sub-system failures assigning each a probability. At the end, probabilities are calculated for each branch to determine to possibility of a system failure. Typically diagrammed in cryptic logic diagrams, they can be understood by reviewing the English translations. Using the heat sealer example, FTA might look something like this:

Sterility failure = (Heat sealer failure OR Raw material failure OR Package Breach)

Heat Sealer failure = (Platen damaged AND NOT Detected during preventive maintenance) OR (Defective Platen AND NOT Detected during installation)

It can get fairy complicated, but it is still sometimes seen.

The RA Professional's Contribution
With risk assessment typically executed by the engineering or quality assurance functions, it is imperative for the regulatory affairs function to be involved in the assessments. Engineers typically focus on the safety and functionality of the design, while quality is often targeting controls and inspections. The RA professional can incorporated regulatory risks by accurately assessing the compliance severities and ensuring controls do not affect any regulatory submissions.

Different regulatory risks are routinely underestimated by manufacturing such as product mislabeling or mixup. Often seen as a minor violation of the Good Manufacturing Practices (GMP) since no actual product was lost, the RA professional can train the team performing the risk assessment in understanding the implications associated with the distribution of adulterated products.

Proposed controls also may compromise the quality or integrity of a submission. If the manufacturing section of a submission indicates that a process is under statistical control and no inspection will be performed, and the quality department is recommending 100% inspection to mitigate a risk, then there is a discrepancy between what is said and what is done. The process is not truly under control with respect to all parameters and the submission is inaccurate. Likewise, controls often include additional labeling for internal use only; therefore, additional controls or procedures must be implemented to ensure that this unapproved labeling is not left on the packaging and sent to the customers.

Conclusion
Risk assessments should be done during new product development and before major process changes. Using informal assessments or the formal failure mode and effects analysis and fault tree analysis can provide systematic and repeatable determinations of risk and their impact on consumers. Regulatory affairs professionals should be involved in this process to ensure that compliance risks are accurately determined and that controls do not undermine current submissions.