FDAQualityMainImage.jpgParagon’s Mike Shafer shares his perspective on the evolution of FDA’s Quality Metrics Initiative, along with insights gained at the PDA Metro Chapter Quality Metrics Event.

The FDA’s Quality Metrics Initiative has sparked active two-way dialog between the agency, industry, and non-profit/professional interest groups. Despite its extended period of germination, the initiative is a driver of significant change in the way life sciences companies monitor and evaluate operational and quality performance. Firms that are able to build key foundational capabilities around information management, data visibility, and business process harmonization will be best prepared to contribute to and evolve with the agency’s evolving stance and requirements. What’s more, those same firms will be able to capture significant value beyond compliance in areas both directly and indirectly related to Quality Metrics.

A Brief History
A lot has changed since the FDA first announced its intention to collect Quality Metrics in February 2013. The initiative began as part of Title X of the Food and Drug Administration Safety and Innovation Act (FDASIA), and it was primarily focused on preventing and mitigating drug shortages. In the intervening four years, the scope and impetus behind the initiative have shifted. While signal detection for potential drug shortages (as well as other potential patient/public health risks) is still a component, the FDA has positioned the metrics initiative as a way to gauge and promote Quality Culture, as well as an essential tool in its new Risk-Based Inspection approach.

Industry Reaction, Challenges, and Refinement
The FDA has published two versions of its Draft Guidance for Industry on Quality Metrics. The first, released in August 2015, took into account input from the Brookings Institute, the Parenteral Drug Association (PDA), industry white papers, and an ISPE Quality Metrics Pilot. Eighty-three responses were received on this initial draft from pharmaceutical firms, non-profits, industry-specific professional organizations and interest groups, and individuals. Several key challenges emerged from these responses:

  • Lack of clarity in metric definitions and method of calculation
  • Limitations imposed by current QMS data structures and business practices to produce the required metrics
  • Potential for biased results when comparing metrics as absolute numbers across the industry
  • Possibility that metrics as defined can be “gamed,” resulting in misleading information and potentially harmful unintended consequences.
  • Actual utility of selected metrics (i.e. Do they truly provide the insight FDA is looking for?)
  • Accountability for reporting as well as data ownership and integrity when dealing with hybrid (i.e., CMO/CPO) supply chains
  • Undefined data format and transfer requirements 

The FDA digested these comments―along with inputs from public meetings,PDA’s Quality Culture Metrics Survey, Metrics Conferences, and Assessment Pilot, as well as a 2nd iteration pilot by ISPE―to refine their guidance. The second version was published in November 2016. In this edition, the number of metrics to be reported was reduced, and the definition of each of the remaining metrics was refined. As a corollary to the second edition, the FDA also published a Technical Conformance Guide that detailed how the data elements supporting the metrics are to be submitted to the agency. 

The industry has since had the opportunity to comment on these new guidance documents. While the FDA has taken steps to address several key areas of concern, challenges still persist. Key themes from the second round of commentary are as follows (including some repeats):

  • Defining a clear and comprehensive metric that can be applied across industry may not be possible.
    • A metric can still be “gamed,” which means that creating a very small set of critical metrics can actually make each one less meaningful.
      • However, creating a network of metrics to combat “gaming” and unintended consequences is not feasible because a complex network cannot be applied consistently across the industry.
    • The importance of trends as opposed to absolute numbers was again reinforced.
    • Even the refined definitions do not solve the problem of standardizing approaches and calculations across the industry
    • Current QMS technology solutions, as well as business practices remain focused on evaluating quality performance by Site and/or Department/Function, rather than by Product family.
      • This means companies will have to take one of two paths:
        • Significant manual intervention to transform the data, which leads to data integrity and human error risks
        • Substantial investment in business process transformation, coupled with leading-edge Information Management and Analytics technology solutions
    • While the Technical Conformance guide provided much needed detail, several technological and IT Process hurdles remain:
      • Most QA Compliance groups (who will own reporting responsibilities) do not currently have the technical resources to build the quality metrics XML submission package as defined by the FDA.
        • Additional time will be required to make these resources available and have them properly trained.
      • Currently there are no FDA Electronic Submission Guidance documents specific to Quality Metrics data, as there are for clinical data or adverse event data, as an example.
      • There have been no redress or amendment procedures yet established should a manufacturer need to submit corrected or additional data points.
      • Processes for FDA’s review and approval of the submitted data, as well as any potential responses from the agency, have not been defined.
      • The strategy to maintain data integrity, security, and privacy once submitted has not been shared.

Key Implications
As a result of the iterative refinement of guidance and active public commentary on the Quality Metrics Initiative, the FDA has reevaluated its timetable for implementing the requirement. After publishing the second edition of its guidance, the FDA had planned to delay implementation of the program and instead open up a voluntary submission program in calendar year 2018. However, due to the additional challenges revealed through review of the technical conformance guide, among other factors, even this voluntary submission period has been postponed.

This delay will allow both the agency and the industry to better prepare for the submission of meaningful quality data elements, and it will provide the opportunity to continue to refine the metrics currently under consideration. 

The progress achieved over the last four years has made it clear that the FDA and industry are aligned on fostering a data-driven quality culture across the industry. Both groups support a risk-based approach to inspection management and quality monitoring. However, it is clear that the insights necessary to support this model can only come by taking advantage of digital age technological advances that support process harmonization, and revolutionary data visibility/information management practices. The Quality Metrics Initiative is, in effect, driving the industry towards a common master data management approach (starting with the 14 master data elements currently requested) and a harmonized view of key quality, operations, and distribution transactions.


Conclusion: Creating a Connected Quality Environment
Firms that prioritize creating a foundational connected quality approach that integrates quality and operations data across their business systems in a validated integration layer―and who invest in a scalable business intelligence capability on top of that layer―will be best positioned to contribute to the continued refinement of the Quality Metrics Initiative. In addition to this regulatory benefit, not to mention the added agility to respond to future FDA Metrics Requirements, the companies who create a connected quality environment can reap the following significant business benefits:

  • They can gain new and powerful insights into how quality records and processes impact their supply chain and supplier risk profile.
  • They can provide both tactical and strategic tools and levers to drive quality culture and right first time performance.
  • They can vastly accelerate their internal and external reporting timelines, leading to more timely and better informed business decisions.

Building the foundations for connected quality does not have to happen in a wholesale manner. Firms can approach it in a modular fashion. By adhering to a scalable information architecture platform and integrating the highest risk and most critical data sources first (including those requested by FDA), companies can take a stepwise approach that can support more organic, incremental change management, as well as the realities of budget, resource, and time constraints.

In the end, firms will have to invest to comply with the FDA Quality Metrics Initiative, but they can use that investment as a foundation to build a competitive advantage beyond compliance that will serve their business for years to come.


5 Key Criteria for Selecting a Quality Document Management Platform


1.  US Department of Health and Human Services: FDA: CBER, CDER. Submission of Quality Metrics Data—Guidance for Industry. Rev. 1. Rockville, MD: Food and Drug Administration, November 2016.Paragon’s Mike Shafer shares his perspective on the evolution of FDA’s Quality Metrics Initiative, along with insights gained at the PDA Metro Chapter Quality Metrics Event.