Digital health Guide to the use of artificial intelligence in medicine

Artificial intelligence (AI) is becoming more prevalent in many areas of life. AI holds huge potential for medicine and the entire field of healthcare. It is predicted that AI will ultimately do much more than change and simplify the work that is done by medical staff. AI applications that can be used to help patients understand their symptoms are growing increasingly popular. AI can also serve to drive medical research forward at a faster pace. If used shrewdly, AI is a tool that can improve the medical care of the future.

Some AI applications are already being used in everyday medical practice, for example, to analyze CT scan images or to schedule appointments in doctors’ practices. Many AI applications are currently in development and growing numbers of AI-supported programs are constantly coming onto the market. 

AI has already helped uncover new findings in medical research, such as identifying some causes of cancer. AI can support the development of new medications. It's also much easier to analyze vast datasets using AI. AI looks set to transform research and accelerate it significantly.

The term “artificial intelligence” (AI) has now become a normal part of everyday language for many people. However, there are various ways of defining what exactly it is. As a result, different people and organizations may use it to mean different things. For example, the term “artificial intelligence” may be used when machines mimic the cognitive abilities of humans. These abilities include thinking, learning, planning and creativity. For this purpose, the machine derives the probabilities of certain statements or behaviors from underlying data or commands. Artificial intelligence systems work independently to a certain extent. Artificial intelligence is most commonly referred to by the acronym “AI”. 

Artificial intelligence is an umbrella term for different concepts. It also includes certain methods used for data analysis and pattern detection. These methods are known as machine learning. With machine learning, a computer program is trained using a large volume of data, for example, a large set of x-ray images. The program is also provided with relevant information during training. For example, it may be told whether or not a knee fracture is visible in the image. From looking at the images, the AI program derives the common features of knee fractures so that it can recognize the again when presented with new images. 

One special type of machine learning is known as deep learning. With machine learning, the way in which the human brain works is mimicked, in other words, the complex interactions between many individual brain cells (neurons). This creates “deep” networks of many interconnected layers of artificial neurons. 

Deep learning applications are also trained with data, for example, so that they can recognize patterns in new data. Some deep learning applications are able to independently identify the relevant common features within the data with which they are trained, without receiving any additional information. Larger datasets are needed for deep learning than for other AI applications. Through the networking of large numbers of artificial neurons, deep learning can perform more complex tasks than other AI applications. However, it is virtually impossible to verify the solution pathway hat was used by a deep learning program because of the intricate architecture involved.

Some artificial neuronal networks are capable of independently generating complex texts or complete images. AI possessing this capability is known as generative AI.

The healthcare system is facing major challenges at the present time. New expensive therapies and demographic changes are leading to continuously spiraling costs. A shortage of trained staff is making it difficult to provide all patients with the care they need. 

Digitization in general and the use of AI can help find solutions to these problems. In the area of medical care, programs that can perform the following tasks are currently being tested or are already in use: 

• administrative tasks, such as coordinating doctors’ appointments
• documenting, summarizing and translating consultations and clinical reports
• helping patients to assess their symptoms
• supporting the analysis of test results
• providing doctors with expert medical knowledge

Documentation

Using AI can save time – for example, by partly or fully automating time-consuming tasks. For example, AI could document the results of an examination or a medical consultation and then draft a doctor’s letter or medical report. In an ideal scenario, the time saved in this way would allow the medical staff to spend more time having direct contact with patients.

Analysis of test results

AI can help to detect disease by analyzing images and other data. For example, AI programs can detect brain bleeds (hemorrhages) in CT scan images. This means that AI has the capacity to speed up the diagnosis process and to improve diagnostic accuracy. Moreover, the use of AI programs could offset lapses in concentration and other types of human error in decision-making to avoid mistakes. AI could be used in specific situations where human error is more likely, for example, during night shifts.

Medical research

The use of AI is delivering very promising results in the area of medical research. One breakthrough was made, for example, by the use of an AI-based computer program that predicts the three-dimensional structure of proteins. If this structure can be detected, it is then easier to determine the function of the proteins in the cells and furthermore to develop new active ingredients for medicinal products.

The increasing use of AI holds huge potential in many areas. However, in order to leverage this potential, the challenges associated with using and developing AI must be overcome.

To ensure reliable AI applications for the medical field, they must be trained using sufficient volumes of suitable data. Users should be trained in the use of AI so that are able to respond appropriately to any mistakes in assessments made by AI applications. Once AI becomes more established, it will also be important to minimize its effects on the environment.

Very little research has been conducted to date into the question of whether the use of AI in medicine is economically viable for healthcare systems. Consideration needs to be given, for example, to the question of the total cost of developing and establishing AI in the healthcare sector and how funds should be deployed in the interests of patients. 

Data basis

Both in research and in AI applications, a solid data basis is essential to achieving accurate results. The use of unsuitable data can distort the results. This type of distortion is known as bias. Biased results should not be used, or only used to a limited extent, to make real-life medical decisions. 

An unwanted bias may also occur with AI models that were trained with data that was not fit for purpose. This can result in misjudgments by AI. To avoid these, large volumes of high-quality data are needed to train an AI model.

New organizations have therefore been established to make high-quality data available while also complying with strict data protection regulations. In Germany, these include the Network of University Medicine, the German Portal for Medical Research Data and the Health Data Lab. In the future, the European Health Data Space (EHDS) will also enable shared use of health data within the European Union.

It will also be possible for AI applications to be trained with data that was originally collected for other purposes, for example, data from electronic patient records. If the data used reflects the full breadth and diversity of society, the impact of data bias can be reduced.

Transparency

The more complex an AI application becomes, the more difficult it is to understand where its results come from. In the medical field, humans are responsible for making some far-reaching decisions. It is therefore important, for both medical staff and patients, that the results and analyses supplied by AI are verified. Judgments made by AI applications in the area of medical care must be sufficiently traceable and transparent. 

Wrong decisions

It has been demonstrated that some AI applications make fewer mistakes than doctors. For example, one AI application was found to be more reliable than human experts in detecting anomalies in images from CT lung scans. Moreover, safety and reliability are prerequisites for the certification of medical AI applications that are brought to market as medical devices. Despite this, the possibility of error is not excluded. Both medical staff and patients need to be aware of this when using AI. Ultimately, the responsibility for medical decisions still lies in human hands. However, manufacturers are still liable in the event of technical errors or incorrect instructions for use.

The human-machine relationship

AI applications can help improve human decision-making by providing objective, comprehensive information. Under certain circumstances, people are more likely to trust the assessments made by an AI application than those made by another human being.

This trust is justified if the AI is capable of delivering better judgments than human experts. However, placing too much trust in AI puts people at risk of becoming increasingly reliant on AI to make decisions that are best made by humans. People may also begin to lose certain skills if decisions are increasingly made by AI.

Both healthcare professionals and patients need to learn how to make proper use of AI. This includes being able to evaluate the strengths and weaknesses of an AI application.

AI cannot replace the special relationships that exist between healthcare professionals and patients. Personal care and attention promote healing and is an important part of the treatment process. AI applications may actually strengthen this special relationship by creating more time for personal contact.

Sustainability

Energy is needed in order to store large datasets, develop AI systems and to send each and every individual prompt to an AI chatbot. Digitization and the increasing use of AI are therefore expected to cause a steep rise in power consumption by data centers worldwide. Other resources are also used to run AI applications, including water to cool data centers. In addition, a massive increase in the volume of electronic waste is expected due to AI hardware requirements. At the same time, AI applications could also help to optimize the consumption of valuable resources in the healthcare system – for example, through more efficient development of medication.

AI systems normally have a long road to travel from the concept stage to use in daily clinical practice. AI systems intended for a specific medial purpose are generally considered to be medical devices. Medical devices must meet European Union legal requirements before they can be sold on the European market. The requirements that must be met in order for medical devices to be certified depend largely on the risk that could potentially be associated with use of the product (product risk).

After an AI application is developed, the product is clinically evaluated. In a clinical evaluation, the manufacture tests the product for safety and performance. This includes, for example, comparing the performance of the AI application with the capabilities of specialist doctors.

Before manufacturers can bring AI medical devices to market, they must verify that they meet all legal requirements. A conformity assessment procedure (CAP) is used for this purpose. An independent testing and inspection body is also involved in this procedure. If the product meets all relevant requirements, it is awarded the CE mark. 

The manufacturer is also obliged to establish a quality management system. This system ensures, for example, that:

• potential risks associated with the product are identified and reduced (risk management)
• the product continues to be tested following its market launch

AI Act of the European Union

In the future, many AI products in the European Union will also have to meet the requirements of the Artificial Intelligence Act, or AI Act for short. As a matter of principle, the AI Act prohibits the introduction of certain AI systems, such as systems that serve to manipulate people or enable social control. The AI Act also places restrictions on certain medical devices that work with AI. 

Some provisions of the AI Act overlap with the legal requirements that apply to medical devices. For example, manufacturers of medical devices and of high-risk AI products are obliged to identify potential risks to safety, health and fundamental rights and to try to mitigate these. KI products, like medical devices, continue to be monitored after they are launched on the market.

The requirements of the AI Act are more specific than the Medical Device Regulation with regard to certain points. These include requirements relating to:

• data processing and storage and data protection
• logging of AI system activities to enable the tracking of errors where necessary
• oversight of AI systems while they are being used by users

In the medical field, decisions and test results can have far-reaching consequences. Staff and patients need to be able to rely on AI applications that are used in this field. Therefore, medical AI applications must be developed with particular care and monitored to ensure that they always deliver correct results.

Programs with artificial intelligence process sensitive patient data while they are being developed and used. 

Statutory provisions seek to ensure that only AI systems that offer security and adequate transparency are used within the European Union (EU). These binding provisions are also intended to make the development of AI products more attractive, as manufacturers will be able to align themselves with the legal framework. This is why the AI Act has been in place in the EU since 2024, with provisions that are now coming into effect in phases.