AI in the service of patient therapy

Background

The range of pharmaceuticals available on the Polish market can make your head spin. Keeping abreast of such a wide range of options, while at the same time keeping up with the latest research on the efficacy of individual active ingredients, is a challenge for both doctors and pharmacists. On the other hand, patients are often prescribed combination therapies that contain several active ingredients in different medicines, which also has an impact on convenience and ease of use, and therefore on the effectiveness of the therapy.

Goal

The client's objective was to create a solution that would assist professionals in the therapy appointment process in order to reduce the number of prescribed medications as much as possible.

Reducing the number of medications taken by the patient:

Simplifies the daily medication routine
Reduces the risk of errors
Is safer due to potential drug-drug interactions

In the course of the work, it became clear that it is equally important to provide the patient with such support - so that they can consult their doctor and facilitate the creation of a dedicated and optimal therapy.

What challenges did we encounter

Scanning a medical prescription case study with AI

One of the major issues in the implementation of this project was the process of entering data into the application, because:

It was not intended to integrate with any IT systems of medical institutions.
The solution was also not intended to store and process patients' personal data.

In order for the doctor to be able to effectively enter the patient's medication data, in addition to manual entry, it was necessary to be able to enter this data in the form of:

a photograph, e.g. of a hospital discharge note or prescription, or any other form of document/photo that can identify the list of medicines taken
images of the packaging of medicines (sometimes patients go to the doctor with the packaging of the medicines they are taking)
a screenshot of the system that stores medical information.

In addition, we have agreed that we do not intend to support handwriting recognition.

During our analysis, we concluded that because input documents and images can vary in form, quality and content structure, implementing an algorithm based solely on OCR mechanisms and recognition of individual document formats appeared to be a costly solution, requiring a great deal of work and constant updating for new document formats.

Ensuring data security and compliance with pharmaceutical legislation also proved to be a challenge to which we paid particular attention.

Solution

In collaboration with the Client, we went through the full design process - from analysing the problem, to validating potential solutions and selecting the most optimal one. As a result, we proposed to the client the use of generative AI (GenAI) models to process the content of user-entered documents or images.

Model selection process

We tested several available GenAI models in order to estimate their performance in text classification and parsing of the unstructured textual input processed by OCR application layer in order to ensure that the response format of the AI system is predictable and consistent, regardless of the quality and format of the input image. This is a prerequisite for further processing of the GenAI response and its use in classical algorithms. In the end, the choice fell on a GenAI model that met the budget requirements and the expected speed of response.

Process for defining prompts and verifying correctness

In order to effectively use the capabilities of the GenAI model, it is necessary to provide it with appropriate prompts. They are a key element in the process of working with the AI model and determine the quality of the answer. They are like a well and accurately asked question together with the provision of all contextual knowledge - so that the answer is as complete and accurate as possible. That's why our engineers gave them a lot of attention.

User path definition process

In parallel with the analysis and validation of the technology solution, we worked on designing the best user experience for the solution. The PWA service was chosen as the best way to adapt the tool to the circumstances in which it would be used and the needs of the users. During the discovery process we also explored areas related to the constraints and opportunities presented by the market and industry specifics. During the workshop, we developed separate user flows for the two user types: patient and doctor/pharmacist. We also defined priorities for the search algorithm and the rules by which it would interpret the data provided.

Results

In the course of the iterative work, our solution gained several new features. These have effectively improved the quality of the experience and added value to the use of the service.

Based on a photo of a document or pharmaceutical packaging, we were able to obtain structured medical data on drug names, active ingredient names and dosages. This relieves the doctor of the need to complete the data manually, leaving him free to check the data he has read and correct it if necessary. This speeds up the work and the decision-making process regarding the choice of drugs for therapy.

The whole process is also designed to work in the natural conditions of a doctor's visit, so we have added to the solution the possibility of uploading a print screen from the medical records currently being viewed, or manually completing the active substances and their doses for which we are looking for a single pill equivalent.

The developed solution was enthusiastically received by the audience. Thanks to the access to the target group, we were able to validate our solution and get constructive feedback on the usability of it. Finally, the developed and tested tool was launched at a prestigious industry event, allowing the Client to present it to a wide audience.