IVDR
Performance Evaluation

IVD performance evaluation – Current situation under IVDD

The IVD Directive 98/79/EC sets out in Annex I requirements for the performance of in vitro diagnostic medical devices. Devices should be designed to be suitable for their intended purpose and to meet performance parameters. Parameters to be mentioned in this context include the following:

• analytical sensitivity,
• diagnostic sensitivity,
• analytical specificity,
• diagnostic specificity,
• accuracy,
• repeatability,
• reproducibility,
• including control of known relevant interference, and limits of detection,

But how do you prove it? How is a performance evaluation under the IVDD carried out? The EN 13612 standard provides some assistance in this respect. It describes the performance evaluation of in vitro diagnostic medical devices and gives different ways of carrying it out. The proof of performance can be provided by existing data, scientific literature or data from performance evaluation studies. Such studies may be conducted in a clinical or otherwise appropriate setting.

EN 13612 provides further information and framework conditions for conducting a performance evaluation study. General requirements, such as responsibility and provision of resources, guide the reader to more specific topics, such as necessary preconditions, study and test design, through to the report on the study. A general set of rules for studies is provided.

The requirements of the IVDD and EN 13612 are currently still very general and therefore interpreted differently. In the current situation this is quite sufficient - however, something fundamental is about to change in the industry. In May 2022 the IVD Regulation (IVDR) will enter into force. There, much more extensive requirements for performance evaluation will be set. It describes much more clearly what is expected.

IVDR performance evaluation – What will change in the future?

Have you ever read MEDDEV 2.7-1 Revision 4 on the clinical evaluation of medical devices? Do you think that this is a very extensive, very formal procedure? IVDR performance evaluation is developing in exactly this direction.

The performance evaluations for in vitro diagnostics, which have been handled very differently up to now, will be subjected to a more uniform procedure under the IVD Regulation. Article 56 and Annex XIII of the IVDR provide a whole new set of requirements for performance evaluation.

As in the clinical evaluation according to MDR, the performance evaluation according to IVDR is divided into two parts:

• A performance evaluation plan is required. The necessary contents are listed in Annex XIII, paragraph 1.1.
• The performance evaluation report shall contain at least the contents of Annex XIII, paragraph 1.3.2.

The performance evaluation report shall include the methodology used for the literature search, the literature search protocol and a literature evaluation report. How such a literature search is conducted and documented has already been established in the field of medical devices and MEDDEV 2.7-1 Revision 4.

The principle of performance evaluation according to IVDR

An IVDR-compliant performance evaluation follows a methodological principle. It is thorough, objective and takes into account both favourable and unfavourable data. Decisive for the thoroughness and scope is the product and the associated risk class, or risks, performance and purpose.

An IVD manufacturer must identify relevant available data for the product and its intended use. If unresolved questions or data gaps subsequently arise, a systematic evaluation of scientific literature must be carried out to address these questions. If this is not sufficiently possible, appropriate data must be generated. All data generated must be evaluated for their suitability for determining safety and performance.

The performance evaluation is based on 3 pillars:

• Proof of scientific validity
• Proof of analytical performance
• Proof of clinical performance

The scientific validity of an analyte „means the association of an analyte with a clinical condition or a physiological state [IVDR]". The route via scientific literature is an easy way to prove this aspect. If available, other sources such as results from clinical performance studies can be used.

The analytical performance describes “means the ability of a device to correctly detect or measure a particular analyte [IVDR]". For this purpose, analytical performance studies must always be carried out. The parameters to be demonstrated are set out in Annex I, Section 9.1(a). The following parameters, among others, are described there:

• analytical sensitivity,
• analytical specificity,
• trueness (bias),
• precision (repeatability and reproducibility),
• and others.

The clinical performance describes „the ability of a device to yield results that are correlated with a particular clinical condition or a physiological or pathological process or state in accordance with the target population and intended user [IVDR]". In Anhang I Abschnitt 9.1 Buchstabe b werden relevante Parameter der klinischen Leistung wie folgt genannt:In Annex I, Section 9.1(b), relevant clinical performance parameters are listed as follows:

• diagnostic sensitivity,
• diagnostic specificity,
• positive predictive value, negative predictive value,
• Likelihood-ratio
• expected values in normal and affected populations

At this point it is interesting to see how proof of clinical performance can be provided. It is possible to use one or a combination of the following sources:

• clinical performance studies,
• scientific peer-reviewed literature,
• published experience gained by routine diagnostic testing.

If there are sufficient reasons, clinical performance studies may be waived. Here it becomes clear that the path via clinical performance studies is preferred. However, this is often not in the interest of the manufacturers. The way out: The necessary reasons against a study are shown via scientific literature. This in turn strengthens the importance of literature research in the IVDR performance evaluation. The similarity to clinical evaluation becomes clear again.

Literature research on IVDR performance evaluation

• Step 1: Analysis of the intended use and clinical claims. Data on the aspects identified here are determined in the literature search.


• Step 2: Identification of data sources from which clinical data can be obtained. For example PubMed, Cochrane, Embase.


• Step 3: Creation of search terms and objective filters to obtain data on the in vitro diagnostic medical device to be evaluated


• Step 4: The identified literature is first pre-filtered at the level of abstracts and checked for potential relevance.


• Step 5: Potentially relevant literature is now analysed in depth at full text level.


• Step 6: The clinical performance data obtained in the analysis of the full texts will now be used to demonstrate clinical performance and, if necessary, to justify that no clinical performance study needs to be conducted.

State of the art

The performance evaluation plan shall address the state of the art, including reference to relevant standards, common specifications, guidelines or best practice documents. In addition, it shall require the indication and specification of the parameters for determining, on the basis of the state of the art in medicine, the acceptability of the risk-benefit balance for the intended purpose and for the analytical and clinical performance of the device.

It is clear that the in vitro diagnostic medical device to be evaluated is not considered in isolation, but also how the product relates to the state of the art and the state of the art in medicine.

The performance evaluation report shall then state that the demonstration of clinical performance is acceptable in the light of the state of the art in medicine. Literature research is one way of identifying useful information for this purpose.

Interface: Post-market surveillance

The post-market surveillance system can actively and systematically collect data on the quality, performance and safety of the IVD throughout its lifetime. The information collected here can be used to update the performance evaluation.

Interface: Post-market performance follow-up

PMPF under IVDR is an ongoing process that updates the performance evaluation. This requires a plan in which the activities to be performed are planned. The contents for this are described in Annex XIII, Part B, 5.2. The IVDR takes the approach that, on the one hand, data is collected throughout the life cycle through the post-market surveillance system and, on the other hand, data is generated continuously through the post-market performance follow-up.

If post-market performance follow-up is not defined as appropriate for an IVD, justification for this will be provided in the performance evaluation report. A post-market performance follow-up plan is nevertheless necessary. However, it will refer to the justification in the performance evaluation.

Conclusion

In the future, IVD performance evaluation will be much more comprehensive than most manufacturers have implemented to date. This will result in a large additional effort to generate IVDR-compliant technical documentation. In addition, many products will be classified higher. As an indication: approx. 80 % of the products do not require a notified body under the IVDD. The IVDR means that only about 20% of the products will not require a notified body. As a result, many manufacturers will experience an abundance of requirements and regulations that they were previously unaware of.