Is the Clinical Implementation of In-House Artificial Intelligence–Developed Algorithms Happening?

THE IN-HOUSE RULE: A CATALYST FOR INNOVATION AT HEALTH INSTITUTIONS

The in-house rule is a provision within the European MDR that permits health institutions to develop and use their own medical device software, provided they comply with specific regulatory requirements (2). This rule allows health institutions to create local solutions for clinical needs not addressed by commercial entities (4). In our recent work, we successfully integrated an AI algorithm for tumor delineation into our clinical pipeline, making prospective clinical testing possible. However, authors and developers should be aware early on that several requirements must be met before the in-house rule can be applied. First, a document of user requirements must be predefined before requesting approval for use under the in-house rule. Developers must have a clearly defined specific intended use and conduct a risk classification of the product in accordance with MDR. Additionally, a market analysis should be performed to assess the availability of MDR-approved software with similar functionality. The in-house rule typically applies when there is a need for a solution that is not available on the market or when an existing solution requires customization to support the institution’s specific workflows and processes for patient care in a particular context.

ESTABLISHING A ROBUST AND ITERATIVE QMS

A critical requirement for using the in-house rule is the establishment of a robust QMS tailored to in-house-developed devices (5). The QMS must ensure that all necessary processes, risk assessments, and testing procedures are in place to comply with the MDR. Although the specific structure of a QMS may vary across institutions, certain key themes are essential for ensuring the safety and effectiveness of the medical device.

At the core of a robust QMS are comprehensive design, risk management, and testing processes, which are inherently iterative. The design process begins with a clear definition of the intended use, along with specifications that align with clinical needs. Risk management involves identifying, assessing, and mitigating potential risks associated with the device. These steps are interdependent: after conducting a risk analysis, the design may need to be revisited and refined to address identified risks, requiring updates to the product and associated documentation, including requirement specifications, test plans, and test reports.

The iterative nature of the QMS ensures that each revision of the product is tested and validated. Testing strategies must be developed to verify that the device meets its specifications and performs as intended. According to the RELAINCE guidelines, specific QMS measures such as periodic monitoring and postdeployment assessments play a critical role in ensuring ongoing safety and efficacy of AI algorithms in clinical environments (6). This includes planning, executing, and documenting tests that demonstrate the device’s reliability, safety, and effectiveness under expected conditions of use. The exact testing requirements depend on the MDR risk classification and the specific implementation of regulations at each institution.

For example, in our case, the QMS process led to the initiation of a prospective noninferiority study involving 150 patients, testing whether physicians become over-reliant on AI. This clinical test aims to provide evidence that the algorithm is safe and effective and that clinicians maintain appropriate clinical judgment when using AI, which is critical for the safe implementation of AI in clinical practice.

Overall, an appropriate QMS must be tailored to the specific requirements of the device and the institution while ensuring compliance with MDR. The design, risk, and testing processes must work in concert, with each cycle of refinement and validation bringing the product closer to clinical readiness. Each organization must interpret the requirements of Article 5(5) and implement a QMS that meets regulatory standards, supports quality assurance, and facilitates the safe introduction of in-house-developed medical devices into clinical practice.

ADDRESSING NICHE CLINICAL NEEDS

The creation of the in-house rule under European regulations enables innovation and allows institutions to address specific clinical challenges promptly (4). By leveraging this rule, health institutions can implement and clinically test their AI developments, ensuring they meet quality and safety standards, and ultimately implement these into clinical routine. This regulatory option is crucial for advancing medical science and improving patient care, especially in areas lacking commercial solutions.

Moreover, the in-house rule supports the development of AI solutions where commercial interest may be limited. For instance, AI models for rare diseases or small patient populations often do not attract commercial developers because of limited market potential. However, these areas are where innovative solutions are needed. As a health institution, we can acquire the necessary hardware and use open-source AI methods to train models tailored to these specific needs. This capability allows clinical researchers to advance medical care for conditions that might otherwise remain under-researched (7). In addition, it enables showcasing of AI solutions that could have a large clinical impact and potential commercial interest.

PRACTICAL IMPLEMENTATION AND FUTURE DIRECTIONS

We have implemented our AI method in our clinical pipeline and are testing it prospectively on incoming patients in a shadow testing phase. This parallel testing helps identify initial flaws and areas for improvement, leading to the next iteration of the product. Subsequently, we plan to evaluate the impact of the AI tool on clinical decisions through masked assessments by oncologists. This prospective evaluation is crucial for validating the tool’s effectiveness and ensuring its safe integration into clinical practice (2,4). However, the path to clinical implementation has challenges. Establishing a QMS requires significant organizational commitment and resources. In the Capital Region of Denmark, the information technology department has a department dedicated to Digital Regulation with a team for handling QMS of products approved under the in-house rule. The documentation process is extensive, demanding meticulous attention and rigorous adherence to regulatory standards. Institutions must develop documents of user requirements that precisely define the intended use and performance expectations of the AI tool. Additionally, institutions need to ensure consistent documentation of development stages and any incidents or deviations, ensuring traceability and accountability throughout the development process.

PRODUCT EXPLANATION AND INTEGRATION TO CLINICAL SOFTWARE

Our product is a deep-learning algorithm designed for tumor delineation on ¹⁸F-FDG PET/CT scans of the head and neck (Fig. 1). The intended use of the product is “to automatically generate a qualified estimate of the ¹⁸F-FDG PET/CT-positive malignant area (abbreviated as PET-GTV, PET-positive Gross Tumor Volume) in scans of Head and neck cancer patients. The intended use is limited to decision support for nuclear medicine specialists in defining PET-GTV. The output PET-GTV is specifically intended for guiding the final gross tumor volume definition, which is subsequently used to define the Clinical Target Volume that forms the basis for radiotherapy treatment planning.” The algorithm leverages convolutional neural networks to analyze PET/CT images, providing precise and consistent tumor boundaries. The effectiveness of this AI tool was validated through a thorough clinical study, which demonstrated its equivalent performance compared with traditional methods. By integrating this AI solution into our clinical workflow, we hope to partially automate workflows, to reduce interobserver variability in tumor delineation, and to ultimately improve patient outcomes (1).

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1.

Visualization of function and quality of algorithm in axial plane of 50-y-old man with cancer of rhinopharynx. (Reprinted from (1).) HU = Hounsfield units.

A program was developed specifically for implementation in our clinical pipelines (available at https://github.com/Rigshospitalet-KFNM/HeadNeckPETGTV). This program provides a comprehensive pipeline for segmenting tumors in ¹⁸F-FDG PET/CT images of head and neck cancer using deep-learning techniques. The primary functionalities include data preprocessing, model application, and format conversion. The application utilizes a trained convolutional neural network previously published (1) to delineate tumor boundaries on the preprocessed images. The model used for segmentation, which takes the preprocessed PET/CT image as input and produces the delineations as output using DICOM Radiotherapy Structure Sets, is available online (https://rigshospitalet-tumor-segmentation.regionh.dk/). Using this pipeline, the AI algorithm can be effectively incorporated into the clinical workflow using the treatment planning system of choice, ensuring that tumor delineation is both precise and efficient.

DISCLOSURE

David Petersen received research funding from Brødrene Hartmanns Fond. Barbara Fischer received research funding from Aage and Johanne Louis-Hansen Fonden and has received support from Siemens Healthineers to attend meetings. No other potential conflict of interest relevant to this article was reported.