Complexities Inherent in Building an Application Requiring Medical Image Visualization

by
John F. Kalafut PhD

There are a myriad of challenges and considerations that you should keep in mind when developing a product that involves medical imaging data visualization and manipulation. Inevitably, you’ll want to compare Off-the-Shelf (OTS) solutions that contain most of the viewing functionality required in your solution against the option of developing your own custom solution using toolkits/SDKs, libraries, or reference applications. “Rolling your own” application using SDKs/frameworks/libraries can offer benefits, but they also come with limitations and risks. Using an existing application (with extensible APIs for application integration) might incur licensing fees and support costs. You should carefully assess your product requirements and goals before deciding.

Before delving into the main challenges for innovators, let’s take a look at the broad challenges to any stakeholder of a solution that consumes and manipulates medical images. Then I’ll highlight some key complexities inherent in building an application requiring medical image visualization.

When it Comes to Medical Imaging, Change is Constant

We in the imaging community – vendors, physicians, or researchers – are accustomed to adapting to continual technological change and then realizing those changes into clinical reality. Radiology as a sub-specialty was predicated and founded on the translation of physics and engineering discoveries for the betterment of medicine. In the past 25 years, we have brought to life the wide-scale digitization of radiology to the global medical community. In a relatively short amount of time, we have seen the near universal adoption and deployment of image management IT and innovations in most developed countries and now see developing and under-served regions embracing the same capabilities, albeit through various deployment mechanisms.

Today, medical images are reviewed not only by radiologists but are commonly referred to and interpreted by other members of the care team. Medical imaging visualization solutions must support several digital data types like video, digital pathology slides, ophthalmology scans, and even smart device photos acquired at the point-of-care. A clinical user wants intuitive, repeatable, and seamless access to these data regardless the care delivery setting. To enable this "anywhere" secure access of medical data – including imagery and video – we are seeing the early adopters’ embrace of public cloud for the storage and manipulation of medical imagery. A complimentary and positive side-benefit of the broader adoption of cloud in clinical routine will continue to accelerate the adoption and access of various machine intelligence technologies. This new era presents both opportunities and challenges for healthcare organizations that need to manage and share medical imaging across different settings and stakeholders. Medical imaging is not just a radiology thing but a strategic asset that requires enterprise-wide coordination and leadership, thus the phrase “enterprise imaging.”

Considering Off-the-Shelf Imaging Solutions for your Innovation

In our experience, many innovators without imaging expertise who need to implement IT solutions for image viewing and manipulation use-cases may have misconceptions about medical imaging. They may be surprised to discover that radiology studies are usually more complex than a single X-Ray image; DICOM, the universal digital standard for medical imaging, can be difficult to master; DICOM is not just a file format standard but also a network protocol; and what works in a proof-of-concept using an open-source tool may not scale well in a production environment with thousands of users, hundreds of concurrent users, and a wide variety of datasets from different sources and systems.


If you are developing an application or an IT system that needs to display and manipulate medical imaging data, you will inevitably consider using toolkits or libraries from vendors or open-source projects. However, you should be aware of the complexity and challenges involved in integrating such solutions with your product. Here are some key points to consider:

  1. Medical imaging data is not just pixel data nor is it just 2D. It includes data from various modalities such as ultrasound, MRI, CT, PET, SPECT, etc. Each modality has its own characteristics, parameters, and formats that need to be handled properly.
  1. Medical imaging data is governed by the DICOM standard, which defines how images are stored, exchanged, and displayed. DICOM is a complex and comprehensive standard that covers many aspects of medical imaging, such as metadata, compression, networking, security, and more. You need to understand the DICOM standard and its implications for your product development and compliance.

    DICOM stands for Digital Imaging and Communications in Medicine, and it is the international standard for storing, transmitting, and managing medical imaging information and related data. DICOM covers various imaging modalities, such as radiography, ultrasound, CT, MRI, and radiation therapy, and it enables the interoperability of medical imaging devices from different manufacturers. DICOM also defines protocols for image compression, 3-D visualization, image presentation, and results reporting.

    However, DICOM is not a perfect standard, and it has some limitations and variations that can cause engineering challenges for medical image visualization and manipulation. It is also a constantly evolving standard, and new classes and features are added regularly. This means that not all toolkits, libraries, and framework support the latest versions of DICOM, and some may have compatibility issues with older versions. You should check the DICOM conformance statements of the OTS solutions you are considering for validating the versions and SOP classes they support, and how they handle unknown or unsupported classes.
  1. Medical imaging data is usually volumetric, meaning that it consists of thousands of individual images that form a 3D representation of the anatomy or function of interest. These images are organized in a hierarchical data model that reflects the acquisition process and the patient information. Your application needs to navigate, visualize, and process this data efficiently and accurately.

    Many use cases require that multiple sets of images be displayed and navigated together in a comparison view. For example, you may want to compare a current image with a previous one, or a CT image with an MRI image. However, not all reference applications or toolkits support this functionality, or they may have different ways of synchronizing the images. You ensure to evaluate and test components (libraries and SDKs) with your own data sets to see how they perform in comparison view scenarios.
  1. Regulatory standards may play a role. Some applications that manipulate and process medical imaging data may be considered as software as medical device (SaMD), which means that they need to comply with certain regulations and standards. For example, if your application performs diagnosis or treatment based on the imaging data, or if it modifies or enhances the imaging data in a way that affects clinical decisions, it may be classified as an SaMD. You should consult with regulatory experts to determine if your application is an SaMD, and what steps you need to take to ensure its safety and effectiveness.

    There are many free and open-source (FOS) toolkits, libraries, and applications available for medical image visualization and manipulation. Some of them are very popular and widely used, such as ITK/VTK/GDCM/DCMTK/Orthanc/Weasis/OHIF Viewer. However, FOS solutions may have some drawbacks compared to commercial solutions, such as less documentation, support, testing, or updates. You should evaluate the quality and reliability of the FOS solutions you are considering, and whether they meet your requirements and expectations.
  2. Medical imaging data is increasingly moving to the cloud, where it can be accessed and shared more easily and securely. However, not all OTS solutions are cloud-native or compatible with cloud platforms. You need to evaluate the performance, scalability, and interoperability of your chosen solution in a cloud environment. Keep in mind that DICOMWeb (the RESTful variant of DICOM transfer) can be tricky to use in low-latency settings.
  3. Medical imaging data is a valuable source of information for artificial intelligence (AI) applications that aim to improve diagnosis, treatment, and outcomes. AI solution developers need tools that can annotate, validate, and evaluate medical imaging data in a way that is compliant with regulatory requirements (such as FDA Part 11) but also user-friendly for clinical domain experts. You need to consider the specific needs and expectations of your AI target audience when selecting or developing such tools. Selecting a vendor partner with a regulatory-cleared component or viewer solution will alleviate many of these issues.

    Use-cases that require dynamic interrogations of the imaging data, such as making measurements, markups, and annotations, are usually integral to clinical applications that embed AI/ML capabilities. However, there is no standard way of storing or exchanging these data in DICOM, and different solutions may have slightly different implementations or interpretations of the standard. Some may use private tags or extensions to store the data in the DICOM files, while others may use separate files or databases. You should check how your framework or library handles these data, and how they can be integrated with your product.
  4. Medical imaging data should be easily accessible by patients who undergo the imaging procedures. Patients have the right and the expectation to access their electronic health records (EHR), including their imaging data. You need to design your product with patient-centric features that enable easy and secure review and exchange of medical imaging data.

The ability to display and manipulate medical imaging data brings a multi-dimensional problem to the already complex challenge of product development. While complex, this is not insurmountable if the right questions are addressed in the early stages. Asher Orion Group is activating medical AI for improved outcomes by providing expert guidance and development services, such as product and solution management, for vendors of AI and enabling technology. Leverage our experience and expertise in bringing new technologies to market to make AI real. Let’s have a conversation.