Wednesday, January 9, 2013

To Certify or not to Certify, that’s the question.

The recent buzz around the announced new IHE certification caused me to step back and think about the value of certification and how it might impact and/or better the implementation of healthcare imaging and information technology. 

If you look up the definition of certification, it means: confirmation that some fact or statement is true.” Therefore, it appears that we are looking for an entity, supposedly un-biased and objective, to prove that a specific claim such as a person, product and service or, in the case of IHE, functionality and/or interface specification is true. 

Some of these certifications are well established and accepted by the industry, for example, we have professionals who are “Certified PACS Administrators” (CPAS) or “Certified Imaging and Informatics Professionals” (CIIP). We have products that are “Certified EMR.”  Another semi-certification, is the FDA approval process. The FDA would not want you to claim that you are “FDA certified,” but, rather a device would have received 510(k) clearance, but it is still the same process: confirm that a medical device meets certain requirements with regard to safety and effectiveness. There are also voluntary certifications such as ISO 9000, and its equivalent for medical devices, ISO 13485, which certifies the presence of a quality management system.

First a few words about certifying professionals. There are quite a few certifications that a healthcare imaging and IT professional can pursue after he or she has completed a basic education. Universities are offering numerous certificate programs as additional training, and there are many certifications by the major hardware and software vendors (Cisco, Microsoft, Oracle). Most application software vendors offer their own certification (EPIC certified), professional organizations offer certification (ARRT to become a certified CT/MRI tech), and non-profits such as HIMSS (CPHIMS), PARCA (CPEMS, CHEA and CPSA) and ABII (CIIP) offer certifications as well.

In general, there is a trend to require more and more certified professionals by institutions as well as vendors. In addition, there is also a trend by the states to require certifications. As an example, the Texas legislature in the previous session actually had a bill pending that would require all healthcare technical professionals to be certified. In addition to these trends, there are sometimes heated discussions about the benefits and/or requirements of being certified as a healthcare imaging and IT professional, although these seem to have calmed down and it has become accepted.

Second, with regard to certifying products and services by ISO, FDA and others, there is no question that this has caused a major improvement in the quality of the products and services offered. The FDA clearance process is not quite perfect, which is witnessed by the number of discrepancy letters that the FDA generates as a result of their site inspections, and the number of worst case interventions resulting in recalls and closing down plants.

Many of you might recall the shortage of C-arms several years back as GE was forced to close down its manufacturing plant (as per the press release: “FDA's most recent inspection of the Utah facility, conducted between July 31 and August 29, 2006, revealed CGMP deficiencies, including failure to establish and maintain adequate procedures for validating the device design and failure to establish and maintain adequate procedures for implementing corrective and preventive actions”).

These products did all have FDA clearance, which shows that certification can only provide a higher level of certainty that a product meets certain specifications, but cannot guarantee it. As a company, we provide FDA “mock-audits” and it is very unusual for a typical company to pass this without any minor, or in many cases, major discrepancies, therefore proving the point.

I have seen a similar behavior at the annual Connectathon meetings, where vendors show up to test their IHE profile implementations, and, with all good intentions, their implementations are sometimes quite far from meeting the connectivity requirements as defined by these profiles. Needless to say Connectathon testing definitely provides a major benefit to both the vendor and ultimately the user community. However, the question that is now raised, is whether or not we also need certification in addition to the testing that has taken place ever since the late 1990’s. IHE USA definitely thinks they should add a certification component, as per the recent announcement to provide this as an option at the 2013 Connectathon in Chicago.

I have been fortunate to be able to participate in the Connectathon effort as a monitor for the past several years. The way this effort operates is probably best described by David Clunie in his blog on this subject, “it provides the opportunity to gather and learn and test and even experiment at the Connectathon, in a relatively informal manner that encourages iteration, improvement and collegiality between competitors.” One item I like to add to this statement is that the Connectathon provides a level-playing field for any vendor, regardless of its nationality, size or perceived importance in the market.

As an example, I have watched integrations between small Asian companies who developed workstation applications with major players of PACS and imaging modalities. These small companies would never have been able to get a chance to test their implementations against any of these other parties except in the field, which is definitely not the ideal test environment. Second, I think that we have started to realize that worldwide specifications such as IHE and their corresponding DICOM and HL7 standards are still subject to interpretation especially by engineers whose primary mother language is NOT English. The Connectathon provides a forum for interpretation.

If certification of IHE profiles is going to be reality and replaces the Connectathon, it will almost certainly use internet-based tools. That means that the face-to-face interactions with monitors who are domain experts, and the iterative improvement process that has been taking place, will be killed. It will also mean that some of the profile testing will become impossible, especially when there is a human (visual) interpretation required. As a case in point, how would a radiologist-monitor be judged as to whether the CAD marks match the actual findings on a mammography image in a virtual internet-based test environment? In addition, I am not sure about the additional benefit that an “ISO-based” certification process would provide, except for additional cost and effort. As experiences show with FDA approval, unless there is an “inspection process” that actually validates the implementation in certain products, and an enforcement process, the additional benefit of the rubber stamp might not justify throwing away the baby with the bath water.

In conclusion, I believe that certification makes sense for professionals, for medical devices as required by the FDA, for implementing quality systems, such as done through ISO, and for functionality as required by ONC for the EMR’s, however for IHE, I can’t find any sensible reason to require certification. Will it happen regardless? Probably yes, but I can only hope that there will be a fair amount of discussion around the adoption of certification and that the IHE leadership will be listening to all stakeholders involved and take their concerns into account.