Clinical Evaluation in Detail

What is a Clinical Evaluation?

The Medical Device Regulation EU 2017/745 provides a clear definition:

“‘clinical evaluation’ means a systematic and planned process to continuously generate, collect, analyse and assess the clinical data pertaining to a device in order to verify the safety and performance, including clinical benefits, of the device when used as intended by the manufacturer; [MDR, Article 2 (44.)”

It should be noted that by definition, this is a process – something ongoing or recurring, not a one-time event. As one of the general obligations of manufacturers, Article 10 mentions that a clinical evaluation must be carried out in accordance with Article 61 and Annex XIV, including post-market surveillance. Further in the text of the regulation, minimum contents of the quality management system are described in the general obligations. At this point, clinical evaluation and clinical post-market follow-up are addressed again.

A clinical evaluation must therefore be prepared for all medical devices and standalone accessories.

As part of the clinical evaluation, the intended clinical performance, clinical benefit, and risks associated with the device, as well as the state of the art or state of medicine, are considered. The problem is that these terms are not defined under MDR.

DIN EN ISO 14971 on risk management systems defines the state of the art as follows:

“developed stage of technical capability at a given time as far as devices, processes and services are concerned, based on the relevant consolidated findings of science, technology and experience”

The state of the art is thus described, for example, by relevant technical standards.

The state of medicine comprehensively describes the disease or clinical condition relevant to the intended purpose of the device and what current therapeutic options exist. It can be thought of as a holistic view of regular clinical practice regarding the respective disease and includes, in addition to other medical devices, a certain consideration of treatment processes. In many medical fields, treatment guidelines exist that should also be consulted in determining the state of medicine. The state of medicine should also include quantitative and qualitative data on safety and performance against which the device to be evaluated should be compared. Ultimately, the clinical evaluation is about demonstrating the benefit-risk profile of the device and its intended purpose. If the benefits outweigh the risks, the device may be placed on the market (after successful conformity assessment). If this is not the case, the intended purpose or the device must be changed and a new assessment must be carried out. The clinical evaluation is therefore indispensable on the path to approval of a medical device or accessory.

The Process

The MDR requires that clinical evaluation be understood as a process. To this end, Article 61 of the regulation demands a precisely defined and methodologically sound procedure based on these principles:

a critical evaluation of the relevant currently available scientific literature relating to the safety, performance, design characteristics and intended purpose of the device […];
a critical evaluation of the results of all available clinical investigations, […];
a consideration of currently available alternative treatment options for that purpose, if any.

Annex XIV defines further requirements:

Establishing and updating a clinical evaluation plan, […];
Identification of available clinical data relevant to the device and its intended purpose, and any gaps in clinical evidence through a systematic scientific literature review;
Appraising all relevant clinical data by evaluating their suitability for establishing the safety and performance of the device;
Generating, through properly designed clinical investigations in accordance with the clinical development plan, any new or additional clinical data necessary to address outstanding issues; and
Analysing all relevant clinical data in order to reach conclusions about the safety and clinical performance of the device including its clinical benefits.

Those familiar with MEDDEV from the days of 93/42/EEC (MDD) will quickly realize that the legal requirements for clinical evaluation compared to the requirements of MEDDEV and also the expectations of notified bodies only partially align. The legal requirements are very vaguely formulated, but in some details are significantly stricter than under MDD.
Currently, the MDCG is preparing a document to provide guidance on clinical evaluation according to MDR. Basically, the wheel is not being reinvented in it, as many processes of clinical evaluation are sensibly defined under MEDDEV and are established in the industry.

Relevant Terms

Many technical terms are used in clinical evaluation, the meaning of which must be known in detail. Only when you understand the nuances of the different definitions will you understand what possibilities the legal requirements allow and how meaningful evidence can be provided.

The Process

Under MEDDEV 2.7-1 rev. 4, which defined extensive requirements for clinical evaluations under MDD, various phases (“Stages”) were described:

Stage 0: Clinical Evaluation Plan (CEP)
Stage 1: Identification of data and data gaps
Stage 2: Appraisal of data
Stage 3: Analysis of data
Stage 4: Clinical Evaluation Report (CER)

When understanding clinical evaluation as a process and considering that the MDR strengthens the concept of clinical data and thus clinical investigation, these stages lack the generation of necessary clinical data. In this context, it has been established that before the creation of the clinical evaluation report, a phase takes place in which a clinical investigation is conducted, if necessary.

The clinical investigation takes place if, after analyzing existing data, it is determined that clinical data are necessary to demonstrate conformity with the applicable general safety and performance requirements.

This can also be seen in Annex XIV of the MDR. There, the following steps for clinical evaluation are roughly required:

Creation of the plan;
Identification of data and data gaps;
Appraisal of data;
Generation of data;
Analysis of data;
Report (only mentioned at the end of the annex).

In the following chapters, we therefore present the somewhat more modern or adapted structure with Stages 0 to 5. The stages presented differ in the order of data analysis from points 1 to 6. This is because, according to Annex XIV, an assessment of relevant clinical data regarding their suitability for demonstrating safety and performance is carried out; then the phase of data generation follows, and only then is it analyzed whether conclusions on safety and clinical performance as well as clinical benefit are possible. In practice, one will try to avoid generating clinical data as part of the clinical evaluation, as conducting clinical investigations is very time-consuming. Therefore, it makes sense to try to reach conclusions before starting a clinical investigation. Only if this is not possible will a clinical investigation be initiated.

This different approach is permissible as the MDR does not prescribe an order, and we use it to keep costs as low as possible for our customers.

Stage 0: the Plan (CEP)

The clinical evaluation plan (CEP) should generally include the following points:

Description of the device to be evaluated;
Description of all variants and configurations;
Description of any accessories;
An introduction to the state of the art;
Equivalence consideration, if relevant.

Additionally, the minimum contents of the clinical evaluation plan from Annex XIV, Part A, (1.a) must be covered:

Identification of the general safety and performance requirements that require support from relevant clinical data;
Specification of the intended purpose of the device;
Clear specification of intended target groups with clear indications and contraindications;
detailed description of the intended clinical benefit for patients with relevant concrete parameters for the clinical outcome;
Specification of the methods to be applied for examining the qualitative and quantitative aspects of clinical safety, with clear reference to the determination of residual risks and side effects;
non-exhaustive list and specification of parameters for determining the acceptability of the benefit-risk ratio for the various indications and the intended purpose or purposes of the device, based on the latest medical knowledge;
Indication of how to address questions regarding the benefit-risk ratio for specific components such as the use of pharmaceutical, non-viable animal or human tissues and
clinical development plan: from exploratory studies, such as first-in-man studies, feasibility studies and pilot studies, to confirmatory studies, such as pivotal clinical trials, and post-market clinical follow-up according to Part B of this Annex, indicating milestones and describing possible acceptance criteria;

Stage 1: Identification of data and data gaps

The identification of available clinical data differs between Legacy Devices (i.e., devices that are still approved under the MDD and are in the transition period to the MDR) and devices that are new or initially approved under the MDR or are already approved.

In principle, clinical data are those that meet the definition of clinical data from Article 2 of the Regulation. There are different levels of evidence for clinical data that can be specified in different ways. MDCG 2020-6 recognizes the following hierarchy:

High-quality clinical trials without gaps;
High-quality clinical trials with certain gaps;
Results from high-quality clinical data collection systems, such as registries
Results from studies with potential methodological deficiencies, but where the data can be quantified and acceptance justified;
Equivalence data (reliable / quantifiable);
Assessment of the state of the art, including the assessment of clinical data from similar devices;
Complaints and vigilance data; curated data;
Proactive PMS data, for example from surveys;
Individual reports on the device in question;
Compliance with non-clinical elements of common specifications deemed relevant to the safety and performance of the device;
Simulated use / animal trials / cadaver trials involving medical professionals or other users;
Preclinical and laboratory tests / compliance with standards.

According to MDCG 2020-6, Class III Legacy Devices and implantable Legacy Devices that do not represent well-established technologies should have at least sufficient clinical data at level 4.

For devices that represent well-established technologies, conformity with the relevant GSPRs can be demonstrated by assessing the cumulative evidence from additional sources in ranks 5 to 12.

It is explicitly stated that relying solely on complaints and vigilance data is not sufficient!

Stage 2: Appraisal of data

When evaluating the data, their suitability for demonstrating the safety and performance of the device should be determined. Aspects of safety, for example, are listed in various GSPRs in Annex I. Should the clinical evaluation now demonstrate all aspects of safety?!

Fortunately not. The clinical data should demonstrate the safety and performance of the device within the scope of its intended purpose – primarily the safety for patients and users. For active medical devices, for example, electrical safety is demonstrated through relevant standards and associated tests. Clinical data are not necessary to demonstrate electrical safety, and therefore the clinical evaluation should focus on the GSPRs that need to be substantiated with clinical data.

It’s somewhat different when the clinical evaluation is prepared according to Article 61 (10) because demonstrating conformity with the GSPRs based on clinical data is not considered appropriate and is justified accordingly. In the absence of clinical data, the demonstration of proof is naturally not based on this data. Since Article 61 (10) is a separate and very specific topic, we will not go into it at this point.

Stage 3: Analysis of data

As part of the data analysis, the necessary conclusions about the safety, performance, and clinical benefit of the device being evaluated are drawn. These conclusions are based on a comparison of the device data with the state of the art or medicine previously described extensively in the clinical evaluation.

GSPRs that typically need to be substantiated with clinical data are GSPR 1 and GSPR 8, as they relate to the clinical safety and clinical performance of the device. For the analysis, reliable methods for evaluating the data must be used and documented in a traceable manner. These include:

Systematic literature review
Qualitative or quantitative methods for data analysis
Weighting criteria for datasets

In Stage 3, it should also be checked whether further clinical data need to be generated – for example, as part of a clinical trial or as a PMCF measure.

Stage 4: Data Generation

Pre-market – Before market introduction:

If insufficient data is available to complete the clinical evaluation with a positive result, corresponding clinical data must first be generated.

Depending on the maturity level of the device, different types of data generation can be used. In pilot studies, a small number of subjects are used to test the device at an early stage of maturity. Here, initial findings on safety and performance can be derived with limited effort, which may need to be used for device improvements before a larger study is meaningful.

Pivotal studies serve as comprehensive evidence of safety, performance, and clinical benefit. The device should have an appropriate level of maturity at the beginning, as failure of the study causes very high costs: The device will likely need to be changed in terms of design or intended purpose. Changes to the device design at such a late stage of development are very time-consuming, as extensive reviews and possibly adjustments to other development results are necessary.

Since these studies to evaluate safety and performance are conducted with human trial participants, they already meet the definition of a clinical trial. Clinical trials in the EU must meet the requirements of Article 62 and Annex XV. In addition, there are national regulations in the member state where the clinical trial is conducted, as well as EN ISO 14155 as the standard for good clinical practice.

It should be particularly emphasized that there are enormous differences in terms of documentation effort between a clinical study (research) and a clinical trial (demonstration of safety and performance of a medical device). The documentation of the clinical trial is much more extensive and must be done very conscientiously. A notified body checks the data of the clinical trial very carefully and can open deviations that then have to be closed by the manufacturer. If the deviations relate to aspects of the clinical trial that are no longer traceable or if there were methodological errors, it may happen that the clinical trial is formally not sufficient to demonstrate safety and performance. Another clinical trial would then be necessary, and the approval is delayed accordingly.

Post-market – After market introduction

The MDR generally requires post-market clinical follow-up (PMCF), in which clinical data should be generated with devices during the market phase to update the clinical evaluation. There are different measures with which data can be collected, such as user surveys and PMCF studies.

PMCF studies are systematic investigations to evaluate safety or performance and are therefore also clinical trials. Depending on the goal of the PMCF study, it can be considered a clinical trial or other clinical investigation. A clinical trial that is not conducted for the following purposes is considered other clinical investigation:

Determination and verification that a device is suitable for its intended purpose and delivers the stated performance;
Determination and verification of clinical benefit;
Determination and verification of the clinical safety of the device and to determine any adverse side effects that may occur and to assess whether these represent acceptable risks compared to the benefit provided by the device.

Stage 5: the Report (CER)

The results of the clinical evaluation are documented in the clinical evaluation report (CER). This means that many evaluations do not necessarily have to take place in the clinical evaluation report – this can significantly improve readability. The CER should present all conclusions and the process of clinical evaluation in a comprehensible manner.

Many notified bodies require that the clinical evaluation documents are independently readable. A mere collection of references is therefore not accepted. What content a notified body examines in the clinical evaluation, for example, can also be derived from various MDCG documents, which should accordingly also be considered when preparing the clinical evaluation.

The CER ultimately serves to document the clinical evidence and thus provide proof that the device is safe and fulfills the clinical benefit when used as intended. The CER is thus also part of the evidence listed in the GSPR checklist to prove compliance with the applicable requirements.

Update

Article 62 (11) requires that the clinical evaluation be updated throughout the entire product lifecycle based on the clinical data resulting from the implementation of PMS and PMCF activities. For most devices, there is no legal requirement to update annually, for example. But when should one update the clinical evaluation?

As a rule of thumb, the clinical evaluation should be updated when data can have an impact on the existing benefit-risk analysis and conclusions on the safety or performance of the device.

Discovery of new risks that could affect the benefit-risk ratio;
new device variant is introduced;
new clinical data has been collected (e.g., PMCF study);
device change that could potentially affect the benefit-risk ratio;
use of new claims;
change in the state of the art;
change in the state of medicine.

Class III medical devices and implants are legally required to update the clinical evaluation annually. For other medical devices, updates are generally less frequent but should not be forgotten. It is therefore important to link the PMS and PMCF processes accordingly with the clinical evaluation.

Worth Knowing

Lectus nullam diam posuere tincidunt nam morbi leo scelerisque nulla imperdiet dictum ac fermentum urna. Lectus nullam diam posuere tincidunt nam morbi leo scelerisque nulla imperdiet dictum ac fermentum urna. Lectus nullam diam posuere tincidunt nam morbi leo scelerisque nulla imperdiet dictum ac fermentum urna. Lectus nullam diam posuere tincidunt nam morbi leo scelerisque nulla imperdiet dictum ac fermentum urna. Lectus nullam diam posuere tincidunt nam morbi leo scelerisque nulla imperdiet dictum ac fermentum urna.

Important Documents

In the section on clinical trials and clinical evaluations of the MDCG publications alone, 18 documents are listed, many of which one should be familiar with for clinical evaluation. While MDCG documents are not legally binding, notified bodies still strictly require compliance. Much of the content is helpful in understanding the expectations and preparing accordingly. However, even with these documents, it is hardly possible to create a compliant clinical evaluation within a reasonable timeframe without proper training and experience.

Although MEDDEV 2.7-1 rev. 4 is no longer fundamentally applicable under the MDR, it still offers a lot of helpful information on how a clinical evaluation can be carried out systematically and in a structured manner.

You should definitely be familiar with the following documents when working on clinical evaluations:

(EU) 2017/745 Article 61 and Annex XIV
MEDDEV 2.7-1 rev. 4 (MDCG 2020 – 6 shows which chapters are still valid)
MDCG 2020-5 Guidance on clinical evaluation – Equivalence
MDCG 2020-6 Guidance on sufficient clinical evidence for legacy devices
MDCG 2020-7 Guidance on PMCF plan template
MDCG 2020-8 Guidance on PMCF evaluation report template
MDCG 2020-13 Clinical evaluation assessment report template
MDCG 2024-10 Clinical evaluation of orphan medical devices

Literature Search

A systematic literature search must be conducted in every clinical evaluation – even if a clinical trial for the device is available. The literature search serves on the one hand as a possibility to determine clinical data for one’s own device, but also as a basis to describe the state of the art in technology and medicine as well as acceptance criteria for clinical safety and performance.

When documenting the literature search, make sure that it is fully reproducible. It must therefore be possible for a third party (e.g., auditor, notified body, authority) to find sufficient information in your documentation to repeat the search. It should be possible to achieve the same results as through your search.

Regarding PubMed, it must be said that the indexing of publications is changed from time to time. This means that with the same search protocol, a fully reproducible search is not always possible if some time has passed in between. This is now known to many people, but unfortunately not to everyone. Nevertheless, PubMed remains a popular database for literature searches because it is comprehensive and easy to use.

The use of different databases, but also different search terms within a database, usually leads to duplicates. For the clinical evaluation, each publication may only be used once. Therefore, check your results for duplicates!