Thursday, May 2, 2013

Middle East PACS Success Factors

Kingdom tower in Riyadh,
Site of HIMSSME 2013

Although I have traveled several times to the Middle East while doing training classes in Dubai, Egypt and Kuwait, this was my first visit to Riyadh, Saudi, to attend the HIMSS Middle East conference. While visiting healthcare facilities and talking with quite a few PACS administrators, it was again amazing that major manufacturers do not make an effort to find out what the local requirements are for a different culture such as that of Arabic countries. It is critical for medical devices and PACS systems to support Arabic names as there are no on-to-one translations from Arabic to English, therefore, a person would have to show up in a worklist with both the Arabic name and its English translation.

Some of the major vendors are not making an effort to implement this, which by definition rules them out from any opportunity in this huge market. I believe this is one of the reasons that smaller companies such those based in Egypt or Turkey can be more effective in this region, in addition to the fact that their support can be better as well.

In addition to not meeting local requirements such as supporting the native character set, there are also many complaints from the user community about the poor service and support by European and US healthcare imaging and IT vendors. Again, it is amazing that these companies don’t make the effort to properly train and recruit engineers and professionals with the same level of competence and skills as in their mother countries. I have had the opportunity to meet and train many professionals from the Middle East and I can assure you that these are among my best students, many of them trained at some of the top US universities. Therefore, they have the same technical skills as engineers in western countries.

To be successful in the countries of the Middle East, and I assume it applies to other regions as well, such as the Asia Pacific region and possibly Eastern Europe, vendors need to learn what the local requirements are, customize their systems to meet those requirements, and train a local technical staff that can support these complex systems. As the market for some of the healthcare imaging and IT systems, such as PACS, are maturing in the US and Europe, the opportunity for growth is definitely among the emerging countries.

IHE Projectathon in Saudi Arabia


Interoperability Showcase at HIMSSME
in Riyadh
Saudi Arabia is not at the forefront of implementing Healthcare IT, but this country of 26 million inhabitants is making every effort to catch up quickly. It has started a national e-health program, which sets aggressive goals to double the number of beds, add another 50 new hospitals to the existing 260 and connect them all seamlessly with every provider using a unified medical heath record by 2020.

To facilitate this, it joined the IHE organization in early 2013, hosted the HIMSS Middle East conference in Riyadh, and is making major investments in setting up a health information exchange allowing electronic health records to exchange information for Saudi citizens, emigrants and visitors. This effort will be accompanied by many new hospital information systems as well as PACS installations and healthcare infrastructure improvements. Every effort is being made to apply the lessons learned by other countries, notably the US and Canada, and leveraging  the standardization efforts by the IHE organization.

The Saudi Ministry of health (MOH) is responsible for 60 percent of the healthcare in Saudi Arabia while 40 percent is split by other ministries such as the ministry of interior taking care of the security forces, army, and the private providers. The MOH is in the process of specifying which IHE profiles make sense based on their specific-use cases, some of which require minor adjustments. For example, the country has a central registration of persons who all have a unique ID (including visitors, who are identified by their visa numbers), which can be used to query from the yet-to-be implemented registries for medical records, which can then be retrieved from the several repositories. The very aggressive and ambitious healthcare IT effort will connect the hospitals to three major data centers, all connected through the Saudi Health Information Exchange (HIE). This HIE will not only function as the central repository and registry but also be able to run national e-health applications such as e-referrals.

Critical to this effort is to standardize vocabularies and terminology for procedures, results, etc. For example, if a request for a lab test is to be sent to another lab, and the results sent back to yet another hospital, one better make sure that there is agreement about the test specification and that there is clear definition of the results so that they can be interpreted electronically by the receiving EMR. The implementation of HL7 version 3, CDA release 2 will be a part of the required functionality to exchange documents between the facilities and the respective HIE.

The facilities themselves are divided into four different layers. The first layer is the primary care physician. He or she will refer a patient to a hospital if needed, which is the second layer. The third layer consists of specialist hospitals for specific diseases and specialists, with the top layer being the so-called medical cities, which deal with the most difficult cases.

It does not make sense to repeat the connectathons to test the different IHE profiles to be used to connect all these systems, as most of the vendors have tested these rigorously at locations in the USA, Europe or other connectathon venues. However, to make sure that the local extensions are correctly implemented, the idea of a “projectathon” has surfaced. This activity will retest the necessary extensions based on the use cases applicable to the local environment.

The Saudi government is very serious about the requirement to support the standards as defined by the IHE modified sub-set. It has plans to set up a testing laboratory, and it might even put import controls in place to prevent any non-standards compliant devices from entering the country.

There is still a lot of work to be done by the Saudi government, such as finalizing the IHE subset of the profiles with their local extensions and modifications, setting up the certification labs, and obviously selecting the vendors. Procuring all the necessary systems and managing the implementation is a major effort. The good news is that in many cases, the hospitals are not burdened by the need to migrate data as they can start from scratch. In addition, applying lessons learned from other countries and using an extensive set of profiles from the well-proven and tested IHE specifications should also help. This should allow the Kingdom of Saudi to be able to jumpstart these implementations and hopefully meet their aggressive goals to be totally health-connected by 2020.

Monday, April 15, 2013

Top ten things to consider when replacing your PACS system.

Many institutions are at a point where they need to replace their PACS systems, which for several, means changing the vendor as well. There can be multiple reasons for changing vendors ranging from being unhappy with the current vendor’s service level, to missing PACS functionality and/or reliability. In many cases, the reason may be because the hospital corporate headquarters decides to single-source all of their imaging equipment from a different vendor. This often occurs when a hospital is acquired and the corporate office wants to have a single vendor to improve information exchange between the institutions. In the rare case that one is not replacing a PACS system, but rather, buying a new one for the first time, most of the recommendations listed below equally apply.

When replacing a PACS system, it is highly recommended that you consider revisiting the fundamental image and information system architecture and workflow, as this is a prime opportunity to redesign it. In particular, one should consider the following:

1.       Use a Vendor Neutral Archive (VNA) – To avoid multiple data migrations in the future, it is strongly recommended that you disconnect the PACS database or image manager from its archive and use a different vendor for the archive from the PACS vendor. It is possible to purchase a VNA from the same vendor as the PACS; however, it is very hard if not impossible to prove that they are not very tightly coupled. For example, some vendors do not physically delete images from the archive but, rather only set a deletion flag in the database, which is not acceptable. The reason is that if one ever is going to migrate the archived images to a different vendor, the deletion-flagged images will almost certainly re-appear. One could argue that deletion of images is not a common scenario, however, it is necessary if the age of the images is past the retention rules, or if they are simply incorrectly labeled or have no diagnostic value. Regardless, just for the sake of being able to go with other vendors in the future without massive, lengthy and costly data migrations, a VNA from a different vendor makes every sense in the world.

2.       Consider an enterprise PACS solution – Another major advantage of using a VNA is that it allows a relatively easy transition to connect multiple PACS systems from several departments. The first one on the list is cardiology, however, this is probably also the most complicated as cardiology requires storage of not only images but waveforms and electrophysiological data as well. Other specialties ranging from gastroenterology to dermatology or dentistry have a need to archive MPEG files in addition to JPEG static images. A “true” VNA should be able to take care of these additional image format storage requirements.

3.       Determine what information should be migrated – Many PACS systems store so-called key image information and annotations in the database instead of encoding this as a DICOM object that can be migrated. It is possible to decode this vendor-proprietary information and create new DICOM objects, however, this goes beyond a “simple” migration and can become quite involved and costly. Therefore, one should decide what information is clinically important and critical, which depends on how the previous PACS system was used. For example, the chance that a five-year-old study is going to be retrieved is small to start with, and if a measurement resulting in an annotation has to be redone, that might not be a big deal. However, if technologists were using annotations to correct left and right positioning indications, this has to be recovered and recreated. In addition, migrating is a good time to revisit retention rules to avoid migrating information that exceeds the retention requirement. Lastly, some PACS systems store diagnostic reports, some don’t. If the new PACS doesn’t and if the old PACS does, these have to be either migrated from the old PACS to a RIS or EMR.

4.       Revisit document management – Many institutional systems have been scanning documents such as requisitions, release forms, and anything that was part of an order, as a DICOM object which is stored with the images. This made a lot of sense when the only access to electronic information for a radiologist was the PACS workstation. However, as most institutions can be expected to either have an EMR, or be in the process of installing one this year, it does not make sense to continue this practice, as it can be kept in the EMR. In addition, some of these documents, such as the requisition, should be available as an electronic order in the EMR, which includes the reason for the study and clinical history in electronic format, eliminating the need to scan requisitions.

5.       Phase out CD import and export – Burning CDs and importing images from them has become a logistical issue in addition to the fact that some CD’s are still created with image resolutions that simply have no diagnostic value such as simple JPEG’s, or are stored in a proprietary format. There are other options for importing and exporting images using file sharing or cloud services, some of those provided by the PACS vendor, some of them provided by a third party. Eventually, images will be exchanged through Health Information Exchanges (HIE’s), therefore, using a file-sharing service may serve as a logical step into that direction.

6.       Apply rules learned from IHE scheduled workflow – I am convinced that the majority of PACS systems don’t fully use the IHE scheduled workflow features, and thus do not take advantage of the efficiency and workflow benefits of those features. In particular, implementing a Modality Performed Procedure Step (MPPS) allows modalities to exchange exam status, the image count, and procedure changes with PACS and RIS, but MPPS has been sparsely implemented. Changing a PACS system is a good opportunity to review the bidirectional RIS/PACS interfaces and activate DICOM MPPS as well as Storage Commitment at all of the modalities. Storage Commitment (handing off archive responsibility) should be implemented between the PACS and VNA anyway to prevent information loss.

7.       Look beyond RIS/PACS to EMR/PACS – RIS systems might become obsolete as sophisticated Computerized Physician Order Entry (CPOE) systems are becoming available as part of an EMR. A radiology department would still need software to manage its resources, finance, supplies, and other typical department management items, however, the traditional RIS features that include ordering, scheduling and report management and distribution can be centralized in an EMR. An integration of a PACS with an EMR on the front-end might make more sense than integrating it with a RIS. If nothing else, a plan to phase out the RIS should be part of the PACS replacement objectives.

8.       Revisit the enterprise imaging solution – Images from the PACS have been made available throughout the enterprise by using web-servers that were part of the PACS system. However, when all images are going to be available at a VNA or enterprise archive, it makes more sense to have a viewer connected to the VNA instead of to the PACS. Most VNA’s support a web version of DICOM called WADO (Web Access to DICOM Objects) and there is actually a newly defined IHE profile that specifies how plug-ins would work with EMR’s (see www.ihe.net for more information). A temporary solution is for the EMR to have a PACS “plug-in,” which allows the EMR access to the primary PACS archive, however, one should design the system architecture to eventually get the images from the VNA.

9.       Make sure there is support for all recent DICOM enhancements – There is a new generation of DICOM objects, which I refer to as DICOM 4.0, which are based on multi-dimensional object definitions with DICOM headers that are much richer, better defined and encoded. This family of so-called DICOM SOP Classes includes the enhanced MR and CT, and the new digital mammography, tomosynthesis. There are even new versions for cardiology, angio and RF. Instead of having to use dedicated modality workstations to view these new DICOM SOP Classes, one should make sure that there is support for them in the new PACS system. Other groups of SOP Classes are the Structured Reports that encode ultrasound measurements and CAD results, which should be supported and interpreted as well as displayed correctly. Lastly, there is an increasing demand to record dose information in the form of DICOM Structured Reports, which have to be managed in a PACS, i.e. recorded and exchanged.

10.   Consider the cloud – Outsourcing storage might make sense for some institutions, depending on available in-house IT resources. Concern about privacy, security and availability, and whether or not the information is considered to be of such an important strategic asset may be reasons that an institution would want to keep it in-house. If for no other reason, one should consider maintaining a copy of the data in the cloud to provide disaster recovery. If there are concerns with security, organizations could consider creating their own clouds, which is rather common for either large geographical areas (e.g. in Canadian provinces), or for large provider groups. For a small institution that does not have any affiliations, a commercially available cloud service might make sense.

In conclusion, replacing a PACS with the same or different vendor without redesigning the architecture of the system would be a major lost opportunity. It is strongly recommended that you review the current architecture, workflow and standards support, i.e. how open the system is and how it can be improved, to make sure the new system can serve the next generation of hardware and software.

Monday, April 1, 2013

Lessons learned from a humanitarian trip.

For the past nine years, I have traveled to Nicaragua as part of a team of Rotarians to build clinics, schools, libraries, pharmacies, and to provide many supplies such as computers, clothes, etc. I just came back from my spring break trip there and am still suffering from PNS, or “Post-Nicaragua-Syndrome.” It is hard to refocus on work in our non-tropical temperatures, and get re-accustomed to the silence in my office. When driving, there are no tricycle taxis with merchants going to the local market, or horse-powered buggies selling firewood, or mini-taxis the size of a smart car.

When I start my trip I always have those grandiose ideas about getting computers down there so they can use electronic health records or use smart phones to transfer vitals from remote clinics. However, rightfully so, they always start laughing at me when I suggest these technologies as the Internet is very expensive and slow, electricity is unreliable and prone to unpredictable surges that “fry” the power supply of computers. There is no A/C so overheating can be a real problem as this country is close to the equator.

Despite all these problems and issues, it is remarkable (or maybe not) that this country still seems to operate and function. It causes me to ask, do you really need a smart phone or is a simple pre-paid phone OK as well. Should I call someone or can I just text him or her?  Do I really need an iPad or tablet to look at my emails or would an old fashioned PC with a keyboard suffice? And what about the cultural impact of having A/C? Now, everyone is always outside in the evening, talking with their neighbors, and the kids are playing on the streets. Imagine everyone being inside to cool off, the streets would be deserted. And imagine more people having cars; remember a teacher makes $300 a month; no way the “middle class” can afford cars right now, no more walking to the market?

Sometimes it does not hurt to re-calibrate ourselves and realize how good we have it, or maybe not… We could do a lot more with much less, save some resources and money, or, share some of what we have with the less fortunate. At least once a year, during my trips to do work in developing countries, I am reminded of that. I encourage everyone to take that opportunity as well. We could make a difference one school, one clinic, one teacher, one student, one doctor at a time.

The most common DICOM connectivity problems.


The most common DICOM connectivity errors are related to addressing issues and/or a mismatching of the negotiated capabilities. Addressing issues can be categorized as follows:


  • IP address issues The IP address might be incorrectly configured, or DHCP has created a new IP address different from the one that is used in the configuration file. One of the quirks with DICOM connections is its reliance on fixed IP addresses, which shows that it was defined in the early 1990’s, prior to the popularity of dynamic IP addressing. Any router can be configured to assign a fixed IP address, but IT maintenance or replacement of the router without reconfiguring can assign a new IP address to a host, which causes the fixed IP settings in the DICOM devices to be incorrect. Using the ping feature will allow a simple check of the availability and configuration of the IP address.
  •  Net mask or domain incorrect − Although the IP address can be correctly configured, if the domain or net mask settings do not match between the source and destination, there will be no communication. Checking the settings at both the destination and source will show if there are any discrepancies.
  • Port number incorrect − The initial port number that was commonly used was port 104, however this caused problems in some devices that did not allow low numbers to be assigned. The correct port number to be used instead is 11112, which is officially assigned to DICOM by the Internet Assigned Numbers Authority (IANA). Mismatching port numbers will show up in a sniffer log as multiple “SYN/ACK” sequences that cannot be resolved.
  • AE Title Incorrect − Note that AE-Titles are case-sensitive, which is probably the most common error made. Mismatching AE-title errors can easily be identified by a sniffer or DICOM error log as it is a standard status code.
  • Firewalls − Many computers and routers have firewalls configured. Make sure that the applicable ports are open.

The DICOM Association has always negotiated between devices using a specific presentation context which consists of an abstract syntax and one or more transfer syntaxes. An example of an abstract syntax is the CT Image Storage SOP Class. An example of a transfer syntax might be JPEG compression. If a proposed presentation context does not match the capabilities at the destination, the message that comes back will indicate a complete or partial rejection of the association. Examples of new SOP Classes that might not be supported are Dose structured reports (SR) from a CT, comprehensive structured reports containing measurements from an ultrasound, CAD results, or new image SOP Classes such as the enhanced CT, MR or digital mammography or tomosynthesis images.

These errors are easy to simulate by using a modality simulator that has the capability of sending exactly the same presentation contexts and investigating the log files and sniffer logs. The resolution is for the destination to be upgraded to support the new SOP classes or having a service engineer simply adding this to the configuration. Another option is to “downgrade” the SOP classes at the source, for example, instead of a CAD result or ultrasound measurement in a SR, send the information in a screen capture, or send the images in the “old” format.

New transfer syntaxes, which are often lacking, are MPEG for video clips, and new wavelet compression techniques. The solution is also upgrading the destination or downgrading the source.

In conclusion, with the right tools, it is relatively easy to visualize the reason why a DICOM connection cannot be made. A visual presentation on using the tools and how the data would look is available on YouTube, see link.

Monday, March 18, 2013

The impact of lacking IT expertise to support PACS/RIS systems.

Digital medical imaging systems such as Picture Archiving and Communication Systems (PACS) are complex and non-trivial to support. It requires PACS system administrators with expertise in both clinical and IT subject domains to maintain image and data integrity. Preferably, these imaging and information professionals are certified through either one or both of the certification agencies (PARCA or ABII).


There are many installations that rely very heavily on support by their imaging vendor. If the vendor has an on-site service engineer, that is fine, however, this practice is getting less common, and many of these systems rely heavily on remote support or an engineer who might take up to several hours or even a day to get on-site. The impact of the decreased support by vendors is that a hospital has become heavily dependent on in-house expertise.

What are the components of IT expertise for an imaging and informatics professional? The components include knowledge of information communications, which includes the network and application level protocols (DICOM, HL7). This will allow a professional to diagnose, and resolve any integration and communication issues. It requires possession of computer hardware and software skills to maintain and support these systems, which could include both Windows® as well as UNIX®. The professional also should be able to use troubleshooting tools to diagnose any potential issues (see related blog).

Most importantly, these professionals are responsible for the specification and adherence of PACS policies and procedures. There are many policies ranging from whom and where to import images from CD’s, to who is merging duplicate patient studies or fixing missing identifiers in an image header. IT-related PACS policies are critical, for example, who is doing back-ups, how often is a backup done, where is the back-up saved, etc. This does not only refer to backing up images or the corresponding database, but also configuration information, hanging protocols, audit trails, monitor calibration records, etc.

Policies of who has access to what external information is critical as well, as these mission-critical systems have to be extremely well protected from downloading viruses or other malware. In many institutions, the use of external memory sticks is forbidden, and computers are configured to disable these external inputs. Virus scanners, firewalls and intrusion detection systems are critical as well as the definition of Virtual Local Area Networks (VLANs) to restrict traffic, and VPN’s for any external connection. If an external connection is used by a vendor, for example, it should always be routed through an internally managed router, which supports audit trails, preferably using standards as defined by IHE (there is a so-called ATNA integration profile defined for audit trails).

In the US, I have heard of a couple of instances where a system was brought down for several hours or even one or two days because of virus infection, but that seems to be less and less common. I am convinced that the decrease is related to strict enforcements of policies, due to a heavier influence of IT in imaging departments. More PACS administrators are now reporting to IT, if only with a “dotted line” on the organization chart, and PACS administrators have also increased their skill sets, witnessed by the thousands of certified professionals.

By contrast, if I ask in my PACS training classes that I teach in emerging and developing countries, the number one PACS support issue is the constant fight against viruses and malware. There is no question in my mind that this is due to a lack of skills, too few certified professionals, and the absence or lack of enforcement of sound IT policies.

It is obvious that sub-optimal operation impacts the availability of timely and accurate information and ultimately patient care. To resolve this requires building awareness among the decision-makers for the implementation of these imaging and information systems, and stepping up their support for their professionals who seek to obtain these skills. Top-down management support is needed to enforce sound IT policies such as limiting the downloading of information from the Internet, disallowing the use of external media, and other critical policies affecting safety and security. Maybe it takes an incident, such as the case of a hospital in the Northwest United States that lost the images of hundreds of patients who had to be called back to the hospital to redo their exams.

In the meantime, the best thing I can do is keep on writing about this subject, promote the use of policies and procedures, and keep on teaching. Hopefully, I will be heard and people will start taking notice.

Friday, March 8, 2013

HIMSS 2013: My top Ten list of What’s in and What’s out.

The HIMSS2013 bustling exhibition

This year’s HIMSS meeting was held in New Orleans, a city that is still struggling to recover from Katrina as was evident from the many empty houses that are still around. This was especially clear when taking the back roads to the airport at peak traffic hour. According to the cab driver, 300,000 people never returned to the city, which ends up being close to 30 percent of its population. But, walking downtown, there was not much evidence of that as it was swinging with music, although it was a little bit hard to find authentic Jazz or the traditional Cajun music with the washboards and accordions.


The HIMSS meeting was bustling with health care IT professionals listening to a good line-up of speakers and educational courses. There were many demonstrations of new products and technologies; here is my (very subjective of course) list of significant developments as well as cool gadgets.

1.       The McKesson interoperability deal is in, Epic is out: Mckesson and Cerner announced a non-profit entity called CommonWell Health Alliance, and were joined by Allscripts, Athenahealth and Greenway. This organization intends to create standards allowing data sharing for patient information. This activity poses two questions: why is Epic, which has the major share of the EMR market absent from this alliance? Second, why replicate work that is already being done by the IHE organization that provides a platform and venue for testing for anyone not just five vendors. To me, this sounds a little bit like the agreement that Philips and Siemens made many years ago to support a mutual imaging format standard called SPI, which, fortunately, has been replaced by an open standard, i.e., DICOM. Another possibility for this alliance could be to counteract the IHE USA organization contracting with ICSA labs in a push for certification, which resulted in a meager eight vendors being certified at the recent Connectathon. It will be interesting to see if this effort will become more of a political than a technical endeavor.

Proximity badge and fingerprint scanner
2.       Fingerprints are in, passwords are out: Everyone agrees that passwords are a necessary evil. While being cumbersome to remember and maintain, combined with audit trails, they effectively meet HIPAA security and privacy requirements. Many institutions have implemented single sign-on, either using the CCOW mechanism as defined by IHE for that specific purpose, or using commercially available semi-proprietary solutions. However, it is still a pain, especially when they expire, have to be reset[MU1] , or, heaven forbid, are shared among practitioners. A better solution is using proximity detectors and fingerprint scanners. A problem with these fingerprint scanners has always been that they could not read through latex gloves, however the new scanners can do that. So, out with the passwords!

      3.       FHIR is in, MINT is out: The Medical Imaging Network Transport (MINT) standard got a bit of attention last year as an alternative to the DICOM communication standard. However, this seems to be fizzling out especially as new additions to the DICOM standard are being defined to facilitate internet based transports. In contrast, there is still a major issue with querying information using the HL7 protocol. HL7 is based on trigger events, for example a patient admission, arrival, or order placer, and not until recently was there a query capability. And, the query capability added to the later version of V2 has been poorly implemented, if at all. The Fast Healthcare Interoperability Resources (FHIR) initiative has developed a draft standard to address some of these limitations and a mini-Connectathon has shown good results. This standard uses web 2.0 and RESTful principles. As it is still a draft, there is quite a bit of work to be done, but this might actually facilitate extended functionality allowing EMR’s to communicate.

Virtual Hospital demonstration
4.       IHE showcase is in, virtual hospital is out: The IHE showcase was a blast this year; its location was in the middle of the conference rooms, instead of last year being in the basement, and it had its own dedicated ballroom. I had the honor of being one of the docents, [MU2] and was able to shuttle many groups from all over the world through the various use cases. The reactions from attendees ranged from “amazing” to “when can I have this in my product.” For example, one of the use cases showed how three EMR’s from a PCP, specialist and another PCP, are able to exchange information crossing two different domains (simulating state boundaries) and thereby using the appropriate HIE gateways, while exchanging documents with all kinds of patient information. If you missed it this year, make sure to look it up next year. The virtual hospital demonstration was also an excellent demo, however, it was more driven by commercial vendors showing applications such as in an ICU, OR, or other settings, and it was at the end of the exhibition hall, much harder to miss than the interoperability showcase. It seemed to get good attendance as well, but there was much less emphasis on the impact of open standards and interoperability.

    5.       Scooters are in, the sports cars are out: I remember when I first started to go to the HIMSS meeting, the give-aways were really over the top. It was even hard to schedule any meetings because you had to work around the various drawings for sports cars and other high-tech gadgets. Getting a chance to participate in a drawing for an i-Pad for allowing a company to get your business card seems to be the current trend right now, however most people already have a tablet (unless they like the i-Pad for their kids, or, maybe parents).  In any case, I typically don’t participate in the drawings, but this time I could not resist taking a chance for a Vespa scooter. It brought back good memories for me, driving with my spouse on the back from Amsterdam to Italy when I was young. If this sounds crazy, yes it is, but then ... when you are young you sometimes do stupid things.

Video of gait in left window,
clinical images on right
6.       Video is in, DICOM is out: Storing images in a DICOM format is simple and seems to have become a commodity now. PACS systems have been widely implemented, at least in the USA and Western hemisphere, and many institutions are already in the second or even third generation and/or vendor. The next challenge is to capture all those video’s that are being created and stored in many departments in a hospital. Some of them can be encapsulated in a DICOM format, such as those generated by endoscopy or surgery devices, however, many of the departments that create videos have no clue what DICOM is all about, and merely store videos on DVD’s. A good example is shown on this picture where a video is shown of a person’s gait, for example, taken before and after hip or back surgery or as part of a physical therapy program. Properly identifying these video clips with sufficient metadata so they can be managed, retrieved and displayed is somewhat of a challenge, and is typically done by using an enterprise archive (VNA) with viewers that have the capability to retrieve and display them. This is an area that will get a lot of attention over the next few years.

7.       Big DATA is in, MU is out: Last year it was all about Meaningful Use (MU), this year this seems to be “old” and it was all about the Big DATA and how to manage it. The US EMR Adoption model which is based on the HIMSS survey of EMR implementations and their functionality still shows that the majority[MU3]  of the hospitals are at stage 3 (38.3 percent as of the fourth quarter of 2012), which includes functionality such as nursing documentation (flow sheets), clinical decision support, and proliferation of PACS outside radiology. Stage 3 means the support of stage 2, 1, and 0 as well, which includes HIE connectivity, the use of medical vocabularies and lab radiology and pharmacy being integrated electronically. Therefore, most hospitals now  have at least some encoded data that can be exchanged. One also would find that some of the long neglected areas suddenly become very valuable data sources. For example, let’s take long-term care. Compared with a hospital, which has only information about the patient from the hospital stay, the long-term care facility will have many years of data. In addition, some of the academic institutions that have kept their patient data forever, instead of aging and deleting it after let’s say seven years, are now sitting on a gold mine of data (think the Mayo clinic). Now the question is, what we can do with this massive amount of data to improve the way we practice healthcare in the USA? I have not seen a good answer yet, and this is definitely going to be an area where we need expertise from other industries and domains. Big DATA, defined as too large, complex and dynamic for any conventional tools to analyze, will definitely be a major topic of discussion.

8.       Structured Report is in, retyping text is out: Almost all of the ultrasound devices, especially the high- end ones, are now creating the measurements in a standard electronic format using the DICOM Structured Report (SR) encapsulation. DICOM includes many templates that are geared towards specific applications, such as cardiovascular, cardiology, OB/GYN and many more. The problem is that the interpretation of these well-defined templates is not supported widely by the many reporting systems, therefore defeating the purpose of SR support. This is despite the major potential efficiency improvement provided by better-defined vocabulary and codes that prefill the information from these SR’s into the templates. To me this interfacing seems to be a no-brainer, but I guess that is not what these big reporting companies think. The good news is that there are other companies (PACSGear for example) that are filling this gap and creating interfaces to, in this case Powerscribe to make this happen.

9.       Kiosks are in, receptionists are out: Healthcare could use some of the innovations used by the airline industry and even some hotels and fast food chains by implementing kiosks for patients to enter their basic information. Many institutions are also starting to use tablets for a patient to enter his or her admission information, but kiosks are a little bit more versatile because one could also implement an identity card scanner. They are also less prone to be stolen as it would be harder to walk off with a kiosk, compared to a tablet, which might just fit in a big pocket or purse. I am probably not alone in not liking the impersonal nature of these devices but it sure is more efficient and supposedly more accurate. If the use of Personal Health Records (PHR’s) becomes more widespread, these  could be used to “check-in” and authorize the PHR access, instead of having to enter all of the patient information again. Another issue is the “digital divide,” between those who are computer illiterate and those who are fluent with computers. This will continue to require personal attention regardless of the data capture, but that divide is shrinking rapidly, if for no other reason than the widespread use of smartphones. The challenge is vendors will have is to make their user interfaces look like an i-Phone from which they still can learn a lot with regard to usability.

10.   And last but not least, Blackouts are in, water is out: Well, we did not have a blackout during the HIMSS meeting, but when I checked in at my hotel on Sunday morning there was no water. It took a few hours for the water to be back up and running, but then it was not safe to drink the water for another two days. The worst thing was that all of the coffee shops were not serving any fresh brewed drinks either. So, for the StarBucks addicts, including myself, it took some getting used to running on a low-caffeine diet. The official explanation of the event was that the power problem was caused by a fire at a sub-station. I think it was good old Voodoo telling all those high-tech professionals to make sure that we are not going to be too dependent on high-tech and making sure we always have a back-up (fresh water bottles in this particular example).

In conclusion, HIMSS 2013 was a blast, Orlando next year will definitely not have the same ambiance, we will miss the jazz and good food. There were no spectacular new products and services that I saw, but more of a continuation of the massive effort to implement EMR’s that have decent usability and make sense of all the information overload we are creating. We’ll see whether there are going to be some more intelligent solutions next year to deal with this.

Tuesday, February 26, 2013

A primer on troubleshooting tools for healthcare imaging and IT.


Many healthcare imaging and IT professionals have to deal with troubleshooting, testing, and validating connectivity and interoperability. There can be multiple objectives for this testing:

·         A software engineer might test his or her newly developed software,
·         An integration engineer needs to test connectivity between different devices,
·         Application engineers test interoperability,
·         Service and support people try to determine why something does not work or stopped working, and
·         System administrators need to deal with finger pointing between different vendors and locate the problem source.
Another important test activity is acceptance testing by a user, often represented by a consultant, to determine whether the system works as specified and meets the initial requirements.
A common categorization of the different test tools is as follows: Test systems, simulators, validators, sniffing tools and test data sets. Many of them are available for free or as open source, some require a modest licensing fee. Test data is generated by either standards organizations or trade associations. Below outlines the characteristics of these tools – when they are used, what tools I recommend followed by a list of where to download them and find tutorials on how to use them.

1.       Test systems:

Test systems are either a copy of the system to be diagnosed or a system with equivalent or very similar behavior. Specifically, if you have a PACS, you might have a “test-server,” which is another license of the same database as used for the production PACS. The test system could be running on a high-powered, stand-alone mini-server, which could possibly store images for a week or so. Most users purchase or negotiate a test system to be available as part of the new purchase. A recent OTech survey showed that about 40 percent of the PACS users have a test system. Another option, in case you don’t have a test system, is to use a free or open source PACS application such as Conquest or DCM4CHEE.

In addition to the PACS “back-bone,” one should always have at least one and preferably two additional test viewers from different vendors for displaying images.  Examples of these viewers would be KPACS, ClearCanvas, Osirix for Mac’s and several others, all of these are freely available.

For EMR’s , I found that it is uncommon for users to have a test system available, which is kind of surprising. In many cases, a production server might be loaded with test data at system installation time, but as soon as the user training is complete and the system goes operational, this information is typically wiped to get ready for the production data. EMR’s are also quite different with regard to their functionality and interfaces; therefore a free or open source EMR might not be as useful as a test PACS system. But one could use the CPRS/VistA EMR, which is developed by the Department of Veterans Affairs and is available as open source.

The best-known interface engine that is available for free is Mirth. This device maps several different interface protocols, but is at its best when being used for HL7 Version 2 message mapping. I found it somewhat hard to use, but there is (paid) support available for someone who needs to configure the mapping rules.

One can use a test system to test new modality connections to a PACS, new interfaces (e.g. lab or pharmacy) to an EMR, or for reproducing certain errors. In the case of a new image acquisition modality connection, one could create test orders that would show up at a test worklist (the DCM4CHEE PACS has this capability), and query the worklist from the test system. This allows for the proper mapping to be tested from the orders to the DICOM worklist, and to tune any additional configuration needed to make sure that the worklist does not have too many or too few entries. The same applies for external interfaces, e.g. for lab or pharmacy to an EMR. It is usually better to test connectivity prior to actually going live.

There are those who use these test systems as a basis for their production, i.e. as their primary clinical system. For example, it is not inconceivable to use the VistA as an EMR, the Mirth interface engine as the HL7 router, DCM4CHEE as a PACS and modality worklist provider, and ClearCanvas for image viewing. However there are potential liability issues for using non-FDA approved and/or non-certified software for medical purposes, especially if used for primary diagnoses of humans. But, for veterinary use, these test PACS systems are relatively widespread in clinical use. I would not recommend using any of these in a production environment unless you have a strong IT background or can rely on a strong IT department or consultant.

2.        Simulators:

A simulator is a hardware and/or software device that looks to the receiver to be identical to or similar to the device it is simulating. An example would be a modality simulator that issues a worklist query to a scheduler, such as provided by a RIS, and can send images to a PACS. If the simulator assumes the same addressing (AE-Title, port number, IP address) as the actual modality, such as a MRI, and sends a copy of the same images, the receiver would recognize the data the same as if the transaction came from the actual device. The same can be done for a lab simulator to an EMR, exchanging orders and results, and for a CPOE simulator sending orders and arrival messages. The advantage is that these simulators provide a “controlled“ environment, while providing extensive logging.
These simulators are typically used to test connectivity prior to having an actual operational system available in order to simulate and resolve error conditions and troubleshoot connectivity issues. They can also be used for stress testing and evaluating performance issues. One should note however, that a simulator does not exactly reproduce the behavior of the device it is intended to simulate. If there are timing related issues, or there are semi-random problems, one would try to keep the original configuration intact as much as possible and use sniffers instead to find out what is going on. I use an HL7 CPOE simulator and DICOM modality simulator, available from OTech. One could also use the various DVTK simulators, but these are not trivial to use and are therefore almost exclusively used by test and integration engineers. DVTK simulation tools also are programmable using a proprietary script, which makes them very useful for exception, performance and error testing and simulation.

3.       Validators:

 A validator is a software application that validates a protocol or data messaging format against a standard set of requirements or good practices. These are extremely useful for testing by development and integration engineers, especially for new releases and new products.
I am amazed by how many errors I find when running a simple DICOM image against a validator. I personally believe that there is no excuse for these errors as these tools are available for free in the public domain. DICOM protocol and data formats can be validated using DVTK.

Another useful tool provided by DVTK is “file compare.” If there is any suspicion about data integrity, i.e. whether a vendor adds or removes information from a header, which could cause problems, one can simply compare the original and “processed” one to see the differences. In addition, this compare tool can be configured to filter certain attributes and highlight the ones that one is looking for. I have used this tool to determine any changes in software, which supposedly did not impact the data format by running this against the image before and after the upgrade.

The HL7 messaging workbench, developed by the late Pete Rontey of the VA, is a great test simulation and validation tool for HL7 Version 2 messaging.

For information exchanges between EMR’s, the CDA data format is emerging as the standard. This is an area where we might expect a lot of potential issues in the near future as these EMR’s are being rolled out. Its data format and compliance with the required templates can be verified on-line on the NIST website.

4.       Sniffing software: 

Sniffing software requires access to the information that is exchanged, which can be done by having the software installed at one of the devices that interacts with the connection to be monitored, which could be the device sending or receiving the information, or at a network switch, or connecting the sniffer to the link to be intercepted by a simple hub. This could be somewhat of an issue as many institutions clamp down on their networks and do not allow for a “listening” device to be connected as they are afraid that it compromises the network integrity. The de-facto standard for sniffing and analyzing DICOM connections is Wireshark, which used to be called Ethereal.

However, one could use Wireshark for the sniffing only, and asking the network engineer to provide you with the so-called .cap file, which can be captured on any of the available commercial sniffer and network management applications. The analysis can then be done separately using Wireshark. Sniffers to detect any semi-random and not easily reproduced errors, or to troubleshoot in situations where the error logs are either non-comprehensible or non-accessible, or to prove that changes are made in data before sending the information. A combination of a sniffer and validator is especially powerful, for example, one could upload a capture file into DVTK analyzer/validator and analyze both the protocol and data format.

Using a sniffer is often the last resort, but it is an essential tool for those hard to diagnose problems. As examples, I have been able to diagnose a device that randomly would issue an abort, which would cause part of the study to fail to be transferred, or to determine the errors as exchanged with the status code of the DICOM responses, and to find that query responses do not quite match the requests, and to resolve many other semi-random problems. One can easily configure the sniffer to capture all of the traffic from a certain source or destination, store it in a rotating buffer and when the problem occurs, start analyzing the information.

5.       Test data

If a problem occurs with clinical data, it is often hard to determine whether the problem is caused by corrupt or incorrectly captured data, or whether it is a result of the communication and processing of the information. Therefore, having a “gold standard” of data is essential. Imagine a radiologist complaining that an image looks “flat” too “dark or light” or just does not have the characteristics he is used to seeing.  In that case, being able to pull up a reference image is invaluable. There are not only sample images, there are also sample presentation states, structured reports and CDA documents available.

Most of the test data objects are created by IHE to test conformance with any of their profiles. For example, there are extensive data sets available to test proper display of all of the different positions indicators (and there are quite a few) on digital mammography images, together with correct mapping of CAD marks.

The same applies for testing the imaging pipeline, for which there are more than a hundred different test images which are encoded using almost every possible combination and permutation of pixel sizes, photometric interpretation, including presentation states. The nice thing is that the data is encoded such that the effect of the image display always is identical. For example, one might have an image for which the header says that the information should be inverted with the data to be inverted so that the ultimate end effect is that the image looks the same as the non-inverted test sample.

It is easy to load all of these images onto a workstation where you’ll see almost immediately for which image the pipeline is broken. This is a great test tool to be used when doing an acceptance test or to check after a new software upgrade is installed at your workstation, and you would be surprised how many systems do not render all of these correctly.

For verifying display and print consistency, the AAPM has created a set of recommendations and test images, both clinical and synthetic, which are invaluable to determine whether or not your display or printer supports the Grayscale Standard Display Function, also referred as “the DICOM curve,” and, if so, if it is properly calibrated according to that standard. A simple visual check will make sure that certain parts of the test pattern are visible and indicate compliance or the potential need for recalibration. Even if one uses non-medical grade displays, there is no reason NOT to calibrate a monitor or printer according to this standard (there are after-market devices and software available to do this) and to make sure they stay in calibration.

In conclusion, I am convinced that any connectivity issue can be visualized, located and resolved by using the right set of test, simulation, and validation tools using a wide variety of test data. It is just a matter of learning how to use these tools and applying them for the appropriate circumstance. In addition, they are also invaluable for acceptance testing and to prevent potential issues. Healthcare IT systems are not plug-and-play and never will be, and a healthcare imaging and IT professionals therefore will need to master these tools to ensure data integrity.

Where to find information and how to access the tools mentioned above: (all of them are free or open source unless mentioned otherwise)

Monday, February 4, 2013

AA: Risks of a make-over.


Living in the Dallas Metroplex, I am highly dependent of American Airlines for all my air travel and therefore follow its developments closely. When I left recently for my meeting, I spotted one of the first airplanes at the DFW airport that had gone through a make-over, i.e. with new colors and logo. The rumor has it that the flight attendants also will be outfitted with brand new uniforms. I guess the reason for the make-over is to create a perception that this is a new beginning and supposedly resulting in a more customer focused corporation.

Here are some of my recommendations for AA getting more customer focus:

-Be focused on leaving on-time: One time, there were not enough meals, which caused the crew to call the catering representative, who had to have his supervisor re-count the meals, and then the supervisor of the supervisor recount it as well, till finally 45 minutes later 3 additional meals were brought in and we could leave. This migth have caused some of my fellow travelers to almost miss their connection. I experienced with another airline that the captain himself ran out across the hall and got a few hamburgers from McDonald to cover for the missing meals and we left in time.

-Keep your bathrooms clean: Anyone who ever has flown a 10+ hour flight knows that the toilets look like a battlefield at the end of the flight. If you are lucky, there is still toilet paper left, and the sink is not disgusting, but the chance is small. Now take Japan Airlines: at any time during the flight, the toilet is spotless, the paper is even nicely folded. What a difference.

-Get rid of those ancient planes as soon as you can: imagine a large screen monitor with more movie channels than I have on my cable at home, nice leather seats, excellent food, and free beer and wine on international flights, pretty much most other airlines, but Delta and Emirates are the best in my opinion.

There are many more recommendations; bottom line is that the risk of trying to create a new perception top-down is that if there are no actual changes “inside”, it might actually backfire. This applies to any company. To be honest, in this particular case, I have not noticed any difference flying this week flying American, and am skeptical that changes are “in the air”. But hopefully I am wrong.

IHE Connectathon, standardization over the top?

The ultimate plug-fest for healthcare IT

The IHE (Integrating the Healthcare Enterprise) has organized so-called Connectathons since 1997. These events, also sometimes called plug-fests, provide an opportunity to test and demonstrate device and system interoperability, and to resolve compatibility issues. 

This particular event has grown steadily, and this year in Chicago, more than 500 engineers and monitors were working diligently for a week to test 163 healthcare IT systems representing 101 vendors from all over the globe. The number of tests that were verified were close to 3000.

This year however, the number of attendees was flat compared with last year, which begs the question, why? The healthcare IT industry is definitely booming, there is a big emphasis on implementing healthcare IT as part of the $20 billion American Recovery and Reinvestment Act (ARRA) incentives for deploying electronic health records, and healthcare is one of the fields where new innovations are definitely happening. Here are my guesses for its reasons for its stall:

1.    We have reached the max with regard to standardization ­­– As of now there are twelve domains ranging from radiology to pathology, including eye care, oncology, lab, cardiology, dentistry, pharmacy, infrastructure, patient care coordination and devices, and patient care coordination and public health. Each domain has numerous profiles based on specific use cases. The IHE website states that there are 350 members with 2,000 volunteers who work on various committees. Maybe IHE has become too big and the effort has become too much too soon, too fast.
2.    Vendors can’t keep up – Imagine you are a vendor and you have to update your software pretty much annually to meet the various IHE integration requirements. Market-driven companies might decide that there are other priorities that should be addressed before interoperability. The only way this would ever change is through customer pressure, i.e. if supporting an integration profile is either high on the requirement list, or, even better, a make-or-break purchasing requirement. However, despite the initiatives by RSNA and HIMSS to promote and advertise the need for interconnectivity and, in particular the need to support new IHE profiles, interconnectivity still appears to be low on the list of customer requirements.
3.    The need for a formal certification – It might be that the idea of a volunteer-driven, loosely managed activity is getting outdated and is being replaced by a more formal testing using standard guidelines resulting in official certified systems. The pros and cons of certification vs the more or less informal plug-fests are discussed elsewhere (see link). So maybe it is time to switch gears.
4.    Standards have become too complex – If you have ever looked at a sample clinical document (CDA), for example the one used to capture information for discharge from an emergency department, it is a very verbose XML document including ten pages of coding, which is not easy to interpret. It literally requires one to either be involved with the HL7 standardization effort or go through intensive training to be able to work with these. One could argue that other standards (DICOM comes to mind) are not that easy either, but people seem to have learned to work with it through the years. However, these standards bring complexity to another level, which definitely creates a barrier for implementation and deployment.
5.    Standards are still not tight enough – Despite its complexity and the extensive encoding used, there is still a connectivity problem as semantic interoperability is still missing. It is not sufficient to merely exchange pieces of data, if there is a different interpretation of the context, adding this information to a database might even be dangerous. If that means that we can only convey certain information in its original context and information model, we could resort to just sending e.g. a PDF document, which makes the current encoded information exchanges a massive overkill.
6.    Vendors are not motivated – It is not always in the commercial interest of a company to support standards, but, rather to keep their systems proprietary. A good example is the evolution of PACS systems. Even though some of the components, notably the image archive in the form of a VNA, were extracted from the monolithic PACS systems, the workstations are still very tightly connected with the image manager of these same systems, despite the presence of workstation work list standards. Except for a few open-source products, vendors have been very hesitant to implement that and want to protect their turf from third party workstation vendors.

Back to the Connectathon, I noticed the actual testing and validation process has definitely improved. There are relatively robust validation tools and plenty of samples available. In addition, we used to look for information structures in the past, now new tests and checks are available that also evaluate the content. This is overall a major positive development. However, the questions still remain whether or not we have reached a plateau or even the top.  Of course, I could be wrong and next year’s attendance could be back up proving that 2013 was just a fluke or temporary dip. But, I don’t think so; maybe we will get a better idea of the reason behind this by then.