In recent times, we have witnessed an astounding surge in the adoption of artificial intelligence (AI) and deep learning (DL), particularly in the realm of medical applications. These technological advancements have sparked an intriguing discourse surrounding the utilization of the copious amounts of healthcare data available, allowing computerized systems to process it effectively. However, when it comes to healthcare, we must place paramount importance on maintaining a high level of patient data privacy and security. Additionally, there is a growing emphasis on obtaining appropriate consent from patients, clients, or participants, which poses additional challenges in gathering retrospective data.
Another crucial concern revolves around the unstructured and non-standardized nature of much of the health-related data, despite its abundance in healthcare. The ownership of medical data also plays a pivotal role in this discussion, as various ownership rules come into play concerning original, de-identified, anonymized, and processed data. This prompts pertinent questions, such as determining which data still qualify as personal for individual patients or participants in clinical trials, and who ultimately owns the data generated by self-learning computer systems.
Once we address and resolve these issues and inquiries, allowing for the collection and utilization of data, we often encounter scenarios where big data is amassed with a specific objective in mind, placing greater emphasis on quantity rather than quality. Unfortunately, this approach can impede proper implementation and even lead to erroneous data processing. The quality assessment and curation of (imaging) data for AI and DL are touted as the most time-consuming tasks for data scientists. This chapter delves into the critical issue of data quality, exploring the process of curating medical images and other associated data, while considering the diverse facets involved as we advance in the AI era.
When we talk about curating medical images and data in the era of AI, we must first tackle the fundamental question: What exactly is artificial intelligence? Different sources provide diverse answers, but let's explore three commonly heard and read definitions.
In the realm of machine learning and deep learning, the algorithms are no longer solely crafted by human observers. Instead, machines utilize input data and known outcomes as training data to develop these algorithms. As a result, data quality becomes a critical factor as the algorithm's development is directly linked to the quality of the collected data. It's important to note that the results provided by such systems are always preliminary since each new piece of data entered into the learning system has the potential to alter the algorithm. Consequently, maintaining a consistently high-quality dataset over time becomes imperative to prevent algorithm degradation due to new data and evolving knowledge. This necessitates not only high-quality data collection but also a meticulous process of data curation to enhance its value and usability.

Now, let's define data curation as the active and ongoing management of data throughout its life cycle, ensuring its relevance and usefulness in the realms of scholarship, science, and education. Data curation encompasses activities such as data discovery, retrieval, quality maintenance, value addition, and long-term preservation, enabling its reuse over time. This meticulous data curation process is essential to establish an imaging biobank or data repository that is easily discoverable and reusable.
As we delve further into this topic, it's crucial to understand the exponential growth of data in our world. Current estimates suggest that the total amount of data doubles every 2-3 years. Moreover, the percentage of digital data collected has significantly increased over the past two decades compared to analog data. While specific long-term data growth figures are not available, we can assume similar trends within the medical imaging field.
In the 1990s, the digitalization of imaging took off with the introduction of standardized data structure and communication through DICOM (Digital Imaging and Communication in Medicine). Additionally, the development of picture archiving and communication systems (PACS) facilitated a more convenient and standardized collection of imaging data. These advancements also ensured long-term storage and accessibility of imaging data, provided appropriate storage mediums and migration strategies were implemented.
The surge in data was predominantly driven by the growing demand for high-quality imaging data, fueled by advancements in computer tomography (CT) and magnetic resonance imaging (MRI) technologies.
The rise of digital data collection has not only revolutionized the accessibility of data but has also enabled higher sampling frequencies and comprehensive datasets, further fueling the exponential growth of data. This, in turn, has led to the accumulation of multi-terabyte PACS archives over the years, encompassing a wealth of patient information from various modalities, sequences, protocols, and more. Additionally, post-processed data derived from analysis and review using a variety of tools and workstations can be integrated into the patient data stored in the PACS, alongside reports and other crucial meta-information.
When conducting retrospective data collection from such a complex PACS environment, the challenge lies in selecting the relevant data from the vast dataset that can truly drive meaningful analysis and yield the desired insights. Determining what data to collect and at what frequency is still a human decision, making it susceptible to errors, variations, and personal or institutional preferences. Consequently, there is a looming risk of amassing collections of largely useless data, which unfortunately often become the datasets utilized in artificial intelligence and deep learning applications.
Machine learning and deep learning systems are rapidly evolving, with frequent developments in both supervised and unsupervised approaches. The selection of an appropriate environment or network becomes a crucial part of the profound learning challenge, as it needs to align with the specific characteristics of the data collection used for training and testing the network. Regardless of the system chosen, the availability of an appropriate training dataset remains paramount.
As I mentioned earlier, the data selected for training artificial intelligence environments must meet high-quality standards. It needs to be accurate, properly labeled and up-to-date with current standards. But quality alone is not enough. The data also needs to be of sufficient size for AI to yield significant findings. Small datasets lack the statistical power necessary for meaningful insights. It is crucial to find and, if necessary, combine the right data collections to obtain a large, unbiased dataset encompassing all possible variations.
In the healthcare industry, the discovery and retrieval of imaging data, particularly in AI radiology, present unique challenges. This data is predominantly personal health data from individuals, which complicates the process. When collecting retrospective health-related data from electronic medical records (EMRs) or picture archiving and communication systems (PACS), or acquiring prospective data through clinical trials or population studies, multiple factors come into play. Unfortunately, the tools needed to extract structured, meaningful, and easily accessible data from these clinical systems are often lacking, despite their essential role in obtaining useful datasets for AI applications.

Identifying the correct cohorts and obtaining the necessary approvals are only the beginning. The variability in data collection can be vast. Medical imaging equipment lacks standardization, leading to incomparable image presentations and diagnostic quality. Differences in equipment generation, protocols, and technical implementations further contribute to the variability. To address these issues, guidelines for data acquisition and protocol standardization are imperative to construct large and valuable data collections.
The imaging data collected today primarily consist of processed DICOM images, while the raw data, such as k-space data of MR and sinograms of CT, which could be valuable for computerized analysis, are not stored. Exploring the inclusion of raw data could potentially offer less variation and enhance analysis capabilities.
Ownership and control of data are critical considerations. We have observed a shift from a hospital-centric data model to a more patient-centric model, integrating new information through the Internet of Things (IoT) and wearables. Open science and open data initiatives promote large data collections stored in the cloud, providing sandbox environments for software training and validation. Entrusting data to the cloud poses risks, as it can lead to loss of control and hinder the discovery and retrieval of relevant data due to dispersion and lack of standardization across multiple cloud solutions.
With all these considerations, even if we manage to collect enough high-quality data, finding the right data becomes a challenge. Retrieving information from disparate systems and combining different sources into a single dataset is complex. While data merging can lead to richer information, it may also raise privacy concerns. Combining multiple datasets could potentially identify individuals through data linkage, compromising their privacy.
Ensuring the privacy of patients or participants is a legal obligation of utmost importance when gathering data in the healthcare domain. Existing techniques for anonymization and de-identification often prove insufficient in terms of effectiveness. We must develop novel algorithms capable of concealing identities effectively, while still preserving the scientific value of de-identified datasets and removing any personal health information. Achieving a balance among these three aspects—de-identification, privacy, and data value—can be challenging, particularly given the diverse nature of data and the absence of standardization, thereby impeding the automation of this process.
Presently, research data repositories for AI radiology rely on a workflow wherein curators visually examine and, if necessary, rectify each DICOM file entered into their database to ensure data privacy and appropriate handling.
One specific hurdle arises from DICOM headers, which may contain proprietary details not encompassed within the standard DICOM format. While such information is critical for advanced data processing, it can also encompass references to personal health information (PHI) or other sensitive data that may infringe on individuals' privacy. Similarly, comment fields within the DICOM headers, allowing free-text content, vary depending on local conventions and may include manually inputted PHI by technicians or radiologists.
In addition to header information, the actual image content presents potential privacy risks. Certain procedures like secondary captures, topograms, and ultrasound examinations may embed sensitive information in the images. Although it is possible to remove this information, automating the process poses challenges due to the varying locations of sensitive data within the images. Furthermore, DICOM containers can be constructed wherein the DICOM header exists. Still, the file includes a different file type, such as a PDF, which could contain complete patient reports including PHI.

Another type of DICOM file requiring careful handling is the DICOM Structured Report (SR). SR files typically contain reports from radiologists describing image reviews and conclusions, potentially disclosing sensitive information based on local policies or the radiologists' reporting methods.
Lastly, facial features can be easily derived from MR and CT datasets of the head. Surface or volume rendering reconstructions of these datasets can reveal the subject's face. Facial recognition technology can combine these reconstructions with images from social media profiles, potentially leading to the identification of the person in the images. This concern is amplified by the direct association between individuals' names and facial features through contemporary social media platforms' tagging of pictures.
The integration of artificial intelligence in radiology brings forth tremendous potential for advancements in healthcare. Safeguarding patient confidentiality, addressing data quality concerns, and implementing robust anonymization techniques are critical steps in leveraging AI for radiological applications ethically and effectively. By striking a balance between innovation and privacy, we can harness the power of AI while ensuring the protection of sensitive healthcare data.
Here are our key points to remember:
Images reference: Parker W, Jaremko JL, Cicero M, et al. Canadian Association of Radiologists White Paper on De-Identification of Medical Imaging: Part 1, General Principles. Canadian Association of Radiologists Journal. 2021;72(1):13-24. doi:10.1177/0846537120967349

Discover how Rayscape works