Original Research | Published: 8 June 2024
open-url

Evaluation of pre-exposure vaccine effectiveness against mpox during the 2022–2023 mpox outbreak in the Madrid region (Spain): a test-negative design

https://doi.org/10.1136/bmjph-2023-000620

Request reuse permissionopen-url
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) licenseopen-url

Abstract

Introduction An appropriate vaccination approach is critical to control the current outbreak of mpox but there is little research providing information on its effectiveness, particularly under circumstances of limited vaccine availability.

Methods Pre-exposure vaccination campaign in the Madrid region, with the modified vaccinia Ankara-Bavarian Nordic was indicated in the risk groups from 18 July 2022.

To evaluate the vaccine effectiveness (VE) of a dose of third-generation smallpox vaccine against mpox in the context of pre-exposure prophylaxis, a population-based indirect cohort study (Broome method) also known as test-negative design (TND) was conducted in the Madrid region (6 751 251 inhabitants). Logistic regression was used to obtain the adjusted VE in the TND with its time–response relationship and for the sensitivity analysis the conditional logistic regression for matched case–control groups.

Results By epidemiological surveillance, 1690 suspected episodes of mpox were detected of which 799 were cases and 891 controls.

The overall adjusted effectiveness of the pre-exposure vaccination against mpox considering an induction period of 14 days, was 86.4% (95% CI 62.2% to 95.1%). VE increases, with a statistically significant time–response effect, being greater than 77.4% with a 95% confidence level from week 7. The VE at eighth week of vaccination was 99% (95% CI 81.7% to 99.9%).

Conclusions The effectiveness of single-dose pre-exposure vaccination against mpox seems very high. Therefore, it appears as a reliable measure to minimise the spread of mpox.

The progressive increase in the effectiveness could justify the delay in the administration of the second dose in situations of shortage of vaccines or prioritisation in the vaccination of the maximum number of subjects at risk.

Further studies evaluating the long-term effectiveness of the full vaccination would be appropriate.

What is already known on this topic

  • Although several studies have researched the effectiveness of pre-exposure vaccination against mpox, they differ in the methodology, sample size and in some cases in the results.

What this study adds

  • Our study obtains a high vaccine effectiveness and similar to that found in most of the published studies that have used other designs and methodologies. This consistency, together with the obtaining of practically superimposable results through the analysis of paired cases and controls, would validate the application of test-negative design in the study of vaccine effectiveness.

  • Furthermore, this design allows the estimation of the vaccination time–response relationship to be easily evaluated.

How this study might affect research, practice or policy

  • High mpox prevention is obtained by vaccination with a single dose of smallpox vaccine (third generation).

  • The progressive increase in effectiveness of vaccination, from the second week that persists from the fourth to the eighth week, could justify the delay in the administration of the second vaccine dose (beyond the fourth week) due to a shortage of vaccines or prioritisation in the vaccination of the maximum number of subjects at risk against the administration of the booster dose.

Introduction

From 1 January 2022 to 15 January 2023, a total of 84 733 laboratory-confirmed mpox (formerly monkeypox) cases and 80 deaths were reported in 110 countries in the 6 WHO1 regions. This is the first time that mpox cases and sustained chains of transmission have been reported in countries without direct or immediate epidemiological links to endemic areas in West or Central Africa.2 On 23 July 2022, the WHO Director-General declared the escalating global mpox outbreak a Public Health Emergency of International Concern. Until the end of July, Europe remained the epicentre of this large, geographically widespread outbreak, with a steady increase in cases and affected countries.3 Spain is the European country that has reported the highest number of cases and the largest mortality since the beginning of the outbreak in Europe. The Community of Madrid (CM) was the region with the largest incidence in Spain.4 5

Both postexposure vaccination and pre-exposure vaccination (PrEP) are the main pharmaceutical measures available to contain the ongoing outbreak, and their eligibility criteria have been included in the guidelines of both national and international health organisations.2 6–9 For these interventions, a third generation of a live, attenuated, non-replicating strain of the vaccinia virus is being used, the modified vaccinia Ankara-Bavarian Nordic (MVA-BN), but more robust data on its effectiveness are needed.10 These recommendations involve the administration of two doses of vaccine spaced 4 weeks apart. However, in the early stages of the outbreak, due to the limited availability of vaccines, it was advisable to administer a single dose to reach as many people at risk as possible.

The primary objective of this study is to evaluate the pre-exposure vaccine effectiveness (VE) of a single dose through surveillance-based (real-world) evidence together with its time–response relationship. Secondary objectives include the identification of effect-modifying factors in the estimation of VE (previous vaccination against smallpox, HIV infection, sex, age and pre-exposure prophylaxis for HIV infection—HIV PrEP).

Materials and methods

Study design and study size

An observational population based—which includes the entire population of the region—indirect cohort (Broome method) also known as test-negative design (TND) was conducted, considering as study subjects those who had symptoms compatible with mpox and had undergone a genomic diagnostic test,11 12 either a specific or a generic PCR for Orthopoxvirus. PCR tests were requested at the discretion of the physician as part of differential diagnosis in line with their normal practice, mainly but not exclusively in settings that deal with sexually transmitted diseases, sexual and reproductive health clinics and in the diagnosis and control of HIV/AIDS, as well as by the general emergency services.13

In this study, the Strengthening the Reporting of Observational Studies in Epidemiology guidelines and recommendations have been followed.14

Setting and data sources

We used the epidemiological surveillance database of compulsory notifiable disease (CND) of the CM, which contains all the records of diagnostic tests performed by all clinical laboratories in the CM (linkage data) and their results together with the epidemiological survey. This database covers the population of the region of Madrid (6 751 251 inhabitants in 2021).

The study period was from 18 July 2022 (the start of the PrEP campaign in the CM) to 12 January 2023. Vaccination status was determined for those individuals with at least one dose of the IMVANEX or JYNNEOS (MVA-BN) vaccine in the CM vaccination registry.

Participants

As mpox is a reportable disease and subject to compulsory epidemiological surveillance, all requests for diagnostic tests are automatically recorded in our epidemiological surveillance information system. Such requests are made by both public and private healthcare providers in both primary and specialist healthcare settings.

Individuals suspected of having mpox were eligible as study subjects if they met the inclusion criteria and had no exclusion criteria (table 1). Those with a positive PCR were considered cases, and those with a negative result were considered controls (regardless of whether other pathogens were confirmed).

Table 1
|
Eligibility criteria

For the study of VE, the first pre-exposure dose was assessed as there were no cases or controls with a second dose.

Variables and measurement

The study factors were as follows: the time elapsed in weeks from vaccination to onset of symptoms of suspected mpox ( Inline Formula , continuous variable), with 8 or more weeks grouped at week 8 by considering that the increase in the immune response from that moment is negligible; and the vaccination status (V, binary variable) assuming that an individual had been vaccinated if an induction period of 2 weeks has elapsed prior to the suspected mpox.

As a response variable (C, binary), the classification of cases or controls was based on the result of the test.

The control variables were age (A, continuous variable), legal sex as recorded in the sanitary registry (S, dichotomous variable), history of HIV infection (H, dichotomous variable), HIV PrEP user (P, dichotomous variable), the number of sexually transmitted infections (STI) under epidemiological surveillance (syphilis, gonococcal infection, lymphogranuloma venereum and infection by Chlamydia trachomatis) during the year prior to suspicion excluding those in the last week ( Inline Formula , integer variable), continent of birth (B, categorical variable) and average personal net income (I, continuous variable) in the 2020 census tract of residence.15

Finally, for adjustment according to temporal and spatial trends during the evolution of the outbreak: the number of confirmed cases ( Inline Formula , integer variable) in the census tract of residence (excluding the case if it is a confirmed case with a positive test) and, as calendar date, the month (M, categorical variable) of suspected mpox.

Statistical methods

The surveillance database was scrubbed for inaccuracies to find, examine and correct any missing, unreliable or incorrect values. In addition, the missing information in the CND database on HIV PrEP was completed with the data obtained from the registry of the CM.

The χ2 test, Fisher’s exact test or the t-test according to the type of variable (categorical or continuous) was used to describe the characteristics of cases and controls.

The measure of VE was calculated using logistic regression with the  Inline Formula  as the measure of association, using the formula  Inline Formula  for the estimation of VE.

For crude effectiveness estimates, the OR estimated by simple logistic regression was used, with the study factor being vaccination status or the time elapsed from vaccination to suspected mpox and the response variable being the PCR result (which classifies suspicions as cases or controls).

To measure the adjusted VE, multivariable logistic regression analysis was used to determine the effect in the presence of confounding and interaction.

The variables and their role included in the model have been those previously described, and, as calendar date, the month of suspected mpox.

The time elapsed from vaccination to onset of symptoms of suspected mpox was introduced into the model together with an indicator variable (dichotomous) for the unvaccinated (zero level of exposure).16

In addition, as a proxy variable of previous immunisation history ( Inline Formula ), a dichotomous variable was included, considering vaccinated against smallpox those who were born prior to the elimination of mandatory vaccination17 in their respective countries of origin (84 countries).

For the model adjustment,18 19 we used a hierarchical model including the first-order interactions of the study factor with the variables previous smallpox vaccination ( Inline Formula ), sex ( Inline Formula ), HIV-PrEP ( Inline Formula ), HIV infection ( Inline Formula ), calendar date ( Inline Formula )—to evaluate whether the effect changes over the course of the outbreak—and the interaction of the number of cases in their census tract with the number of sexually transmitted infections in the previous year ( Inline Formula ). The coefficients of the model were estimated by maximum likelihood. In the adjustment process, interactions that were not statistically significant—p>0.05 in the likelihood ratio (LR) test—were eliminated. The model contains all of the potential confounders mentioned above.

After fitting the model, the model assumptions and goodness of fit were verified for compliance, as well as the assumption of the log-linear relationship of the continuous variables with the logit estimates.

The analysis of residuals included the Pearson residuals and their standardisation. For the assessment of possible influential values, leverage effect detection (hat), Cook’s distance ( Inline Formula ), residual deviance ( Inline Formula ) and the  Inline Formula  statistic (change in Pearson’s  Inline Formula ) were used.

To assess the influence of a possible selection bias (that the controls do not come from the same population as the cases and have different characteristics), a sensitivity analysis was performed by matching cases and controls on age, sex and calendar month and modelled by a conditional logistic regression with fixed effects for matched case‒control groups with adjustment for the rest of possible confounders. Furthermore, VE has been assessed only in the male population.

Construction and verification of the logistic regression model

The variables and interactions introduced in the hierarchical model are those indicated in the previous section. To achieve the log-linear relationship of age (for presenting an inverted-U shaped relationship with the logit), a fractional polynomial transformation of four dimensions was performed, since the best transformation model was significantly different from the linear relationship based on the deviance difference ( Inline Formula  =18.9, p=0.009).

The overall interactions (chunk test) were not statistically significant (LR test:  Inline Formula  =4.56, p=0.919).

The vaccination variable (the time elapsed in weeks from vaccination to onset of symptoms of suspected mpox) shows a statistically significant time–response relationship16 (LR test:  Inline Formula  = 24.52, p<0.0001), which does not differ from the linear trend (LR test:  Inline Formula  =6.05, p=0.418).

The goodness-of-fit test of the model shows its adequacy ( Inline Formula  = 1710.32, p=0.23), and the model coefficients ( Inline Formula ) are significantly different from zero ( Inline Formula  = 593.86, p<0.0001).

The analysis of the residuals shows that despite the extreme values in the Pearson residuals, there are no influential points: the maximum value of  Inline Formula is 0.55, and the maximum hat value is 0.12. Residual plots against the study factor and the control variables indicated no lack of linearity and no underlying trend against the omitted variables.

For the sensitivity analysis, matched case‒control groups were used. Grouping was performed by the variables sex, age groups ([17, 25), [25, 55) and [55, 69)—given the aforementioned inverted U-shaped relationship with the logit) and calendar month. After grouping, 33 groups (of the 36 possible combinations) were obtained, and 1607 subjects (799 cases and 808 controls) were matched within the same groups. 14 groups have been excluded because they contain only cases or controls. In addition, fixed effects for the rest of the confusion were adjusted.

For data scrubbing and manipulation, we used the Pandas, NumPy and DateTime libraries in Python V.3.9.13; for spatial data processing, we used QGIS V.3.4; and for statistical analysis, we used Stata/BE V.18.0 from StataCorp.

Patient and public involvement

Non-governmental associations and the community network of the affected groups participated in the establishment and dissemination of preventive measures together with their collaboration in the vaccination campaign.

Pre-exposure vaccination was recommended for the following risk groups:

  • People engaging in high-risk sexual practices, especially but not exclusively gay, bisexual and other men who have sex with men.

  • People at occupational risk, where individual protective equipment may not provide complete protection. Examples include healthcare workers in specialist STI/HIV clinics who provide care to people engaging in high-risk practices, laboratory staff who handle potentially mpox-contaminated samples and staff whose duties include disinfecting surfaces in locations where high-risk sexual practices are engaged in.

Vaccination was carried out at the Vaccination Centre for the Region of Madrid, using an app to make appointments with a wide selection of times that was extensively communicated among healthcare staff. The website of the CM had a specific page for mpox (Mpox %7C Comunidad de Madrid) that provided information about the disease (description, frequently asked questions, etc) and links to request an appointment for vaccination.

Results

Patient population

The population of the CM stood at 6 751 251 inhabitants in 2021, divided into 4417 census sections with a median population of 1400 inhabitants (IQR: 1089–1851) and a range of 55–5753 inhabitants.20 From the start of the vaccination campaign to 12 January 2023, a total of 7338 received at least one dose of vaccine, and 743 received a second dose of vaccine. The pre-exposure vaccinated people had a median age of 35 years (IQR: 30–40), with an age range of 17–68 years.

During the outbreak (from 26 April 2022, when the first retrospectively diagnosed case appeared in the CM, to 12 January 2023), there were 4093 suspected cases of mpox, 1981 (48.4%) of which occurred during the study period, and 1909 were within the age range of the vaccinated population during the study period.

The flow chart can be seen in figure 1, which shows 1690 suspected cases with all values for the response and study factor variables and for the control or adjustment variables.

Figure 1
Figure 1

Participant flow diagram

Of the 1690 suspected cases, 799 individuals were classified as cases because the diagnosis of mpox was confirmed, and 891 were classified as controls because the diagnostic test was negative. The study duration was 178 days (5.93 months).

Univariate analysis and crude VE

Table 2 describes the characteristics of the cases and controls.

Table 2
|
Description of case‒control characteristics

The age group distribution of cases and controls showed no differences, but there were differences by sex, with an OR 11 times higher for men than for women (95% CI 6.51 to 19.95). HIV PrEP users were 1.85 times more likely to be infected with mpox than non-users (95% CI 1.33 to 2.56) and persons with HIV were 3.43 times more likely to be infected with mpox than non-infected people (95% CI 2.72 to 4.34).

Regarding the history of STI in the year prior to suspicion, those who experienced an infection had a 1.55 times higher risk of being a case (95% CI 1.13 to 2.12) with a significant trend ( Inline Formula  = 9.74, p=0.0018) in the ORs.

The presence of at least one confirmed case in the census tract of residence is associated with a 1.42 times higher risk of being a case (95% CI 1.17 to 1.74), showing a trend ( Inline Formula  = 8.59, p=0.0034) for the different groupings of the number of cases in the census tract.

There were no significant differences according to income level ( Inline Formula  = 9.18, p=0.102), although there were lower probabilities of illness from the 70th percentile onwards, with the trend test being significant ( Inline Formula  = 4.09, p=0.043). On the other hand, there were significant differences by county/continent of birth ( Inline Formula  = 60.14, p<0.001), and Americans had a 2.15 times higher risk of being a case (95% CI 1.75 to 2.64).

Table 3 shows the crude VE reaching 99.3% in the eighth week after vaccination (95% CI 89.6 to 100). Considering a 2-week induction period, the VE was 84.0% (95% CI 59% to 93.8%). The VE by strata at 8 weeks after vaccination was calculated by simple logistic regression with the study factor being the time elapsed since vaccination in weeks. The crude effectiveness for the different strata is similar to that of the total.

Table 3
|
Crude and adjusted vaccine effectiveness

Multivariable analysis: measure of effect controlling by confounding factors

The adjusted VE with a 2-week induction period (table 3) was 86.4% (95% CI 62.2% to 95.1%).

The VE presented a statistically significant time–response effect as the time from vaccination to suspicion increased. The adjusted pre-exposure VE against mpox was 99.0% in the eighth week of vaccination (95% CI 81.7% to 99.9%). VE reached values above 90% in the fourth week and more than 80% with 95% confidence at week 8 postvaccination (see table 3 and figure 2).

Figure 2
Figure 2

Adjusted vaccine effectiveness with its 95% confidence intervals according to time (weeks) after administration, assessed in both test-negative and matched case-control (sensitivity analysis) design.

The VE was similar in all subgroups (age groups, sex, HIV PrEP users, HIV infection and natives vs non-natives) and did not differ from the previously stated values, except for the loss of statistical power.

The interaction of mpox vaccination with previous vaccination history was not statistically significant, showing no differences in the effect. This was consistent with the similarity of effectiveness estimates from week 8 for the 17–39 and 40–69 age groups (99.5% and 98%, respectively).

The adjusted VE using matched case‒control groups is superimposable to that obtained by the adjusted analysis without matching, as shown in table 3 and figure 2. The VE in the male population, considering an induction period of 14 days, is 86.8% (95% CI 63.1% to 95.3%).

The value of the VE estimate does not change over the course of the outbreak since the interaction with the calendar date has not been statistically significant.

Discussion

In our study, since we obtained both the cases and the controls in a surveillance setting, it implies that both have a population base and are, therefore, representative of the same.

The results allowed us to evaluate the ‘real-world’ effectiveness of a single pre-exposure dose of a third-generation smallpox vaccine in the prevention of mpox. A statistically significant time–response effect was observed between the time elapsed after vaccination and the protection obtained by vaccination, which made it possible to evaluate VE on a weekly basis. VE values above 90% were obtained from the fourth to the eighth weeks, with 95% confidence that VE is at least more than 80%. Similar figures were obtained in the sensitivity analysis using matched case‒control groups and for the male population.

The time–response relationship was consistent with data from the phase III immunogenicity study of MVA-BN, which showed an increase in response (antibody titre) over time.21 22 The overall adjusted pre-exposure VE of 86.4% (95% CI 62.2% to 95.1%) was also estimated considering the first 2 weeks after vaccine administration as the induction period and disregarding the time thereafter. This finding coincides with the results of the cohort study in men by Wolff Sagy et al,23 who obtained an adjusted HR for infection in the vaccinated population (one dose) compared with the non-vaccinated population of 0.14 (95% CI 0.05 to 0.41); therefore, a VE of 86% (95% CI 59% to 95%).

In our study, we considered the time elapsed in weeks from vaccination to onset of suspected mpox symptoms to be the most relevant variable, not only because of its statistical advantages but also because it reveals the evolution of the response to vaccination since its administration. Additionally, this variable prevents a possible classification bias by considering unvaccinated individuals who had not passed the induction period; therefore, it captures the ‘partial’ effect of the period and the possible involvement of the superinfection exclusion mechanism.24–27

The majority of published studies of pre-exposure VE with a single dose obtain a point estimate in the range of 68.1%–93% with different designs including cohort and case–control studies.28–32 However, Deputy et al33 obtained a low VE, of 35.8% (95% CI 22.1% to 47.1%), such a discrepancy would be conditioned by their selection of the controls.

The underestimation of VE by Deputy et al33 would be the result of three decisions when choosing their secondary-based controls. First, they include only incident HIV infections in their controls, but prevalent HIV infections predominate among their cases—the population that lives with HIV has a higher risk of getting sick from mpox.4 34 Second, when choosing HIV infections and PrEP-HIV users as controls, HIV infections or PrEP-HIV users with risk factors other than men who have sex with men (MSM), who are not the target population for vaccination, have been included (MSM have predominated as cases during the mpox outbreak). Finally, the exclusion of participants ‘who had no in-person medical encounters during the 3 years before their index event or who had only a telehealth visit to serve as the index event during the study period’, which removes from the study the healthiest population or those with the lowest use of healthcare (and therefore not vaccinated) has acted differentially between cases and controls: the difference in the exclusion percentage is 29% (95% CI 27.2% to 30.9%) between cases and HIV-infected controls and 6.5% (95% CI 4.9% to 8.1%) between cases and HIV-PrEP-using controls.

The use of TND to estimate the VE is well established, and its appropriateness has been demonstrated in various infectious diseases.35–42 This design, also called an indirect cohort, can also be thought of as a variant of a cohort design,43 and therefore, the OR can be interpreted as a risk OR. The TND has also been used to determine risk factors associated with susceptibility to infection.44–46

The representativeness of the target population in the TND studies is guaranteed since the controls come from the same population as the cases because the same selection process is used for controls as for tested positives, and the selection biases that case‒control studies often incur are prevented.44 Thus, for Franke et al, the TND may minimise bias due to differential health-seeking behaviour and recall.36

The mpox outbreak has predominantly affected MSM; thus, the control measures4 34 aimed at the rapid detection of cases in the risk group entail an association between the performance of the test and the characteristics of the tested persons. Given this potential association, the classic control group in case‒control studies may carry a bias given that tested persons might have different characteristics than the general population.44 45

The potential differential selection bias between cases and controls, which can occur in all case‒control studies, is not believed to have affected the results for three reasons. First, because of the exhaustive recording of all tests performed and the lack of differentiation between cases and controls, there was no prior knowledge of the vaccination history of any of them. Second, the suspected cases eliminated from the analysis for missing information represent 1.8% of the total finally analysed, thus minimising possible selection bias on this basis. Finally, the sensitivity analysis of the results using matched case‒control groups leads to the same results. Other authors36 38 have obtained similar results in VE using the TND and other study designs.

In addition, given the lack of vaccines, vaccination was prioritised for the groups at the highest risk of getting sick given their behavioural practices related to mpox exposure. Therefore, if the controls did not reflect this population at risk, there would not have been a higher proportion of vaccinated controls than vaccinated cases.

Potential disease or exposure misclassification was minimised by restricting the suspected cases to residents in the CM. The availability of electronic records (linkage data) of both diagnostic tests and vaccination status also prevented differential errors between cases and controls. Additionally, the extensive use of diagnostic tests with high sensitivity and specificity11 12 on samples with a high positivity rate ensured that false negatives were also reduced to a minimum.

There is a possibility that in vaccinated cases with suspicious symptoms, doctors tend to discard the diagnostic test because they consider the presence of mpox more unlikely than other differential diagnosis assessments. This would lead to a reduction in the number of vaccine recipients, especially among controls (there are 11% more controls than cases in our study) and would mean that our VE is underestimated. Narrowing down to severe cases in order to reduce this possibility would not be appropriate, as the majority of symptoms were mild: only 2.9% of cases (23/799) and 0.56% of controls (5/891) required hospital admission, mostly due to the occurrence of comorbidities.

The study accounted for possible differences or changes in exposure risk by controlling for proxy variables, although there may be some residual confounding, both in individual variables and factors derived from the evolution of the outbreak.

We do not carry out a sensitivity analysis with gender because it was not included in the sanitary registry.

Despite the above, among the possible limitations of the study is the generalisability of the results. The individuals studied may have had a lower intensity of exposure to risk factors than the individuals who became ill at the beginning of the outbreak. In other words, the individuals most likely to be exposed would have become ill at the beginning of the outbreak and therefore before the start of the vaccination campaign. These possible differences would not influence the VE estimate, provided that the likelihood of being vaccinated among the confirmed cases and the likelihood of being vaccinated among controls does not vary according to risk factors associated with higher exposure since higher exposure, dependent on the evolution of the outbreak, has been temporospatially controlled. If the presence of risk factors corresponded to a higher probability of immunisation among case subjects, our VE would be overestimated. However, if subjects with higher exposure to risk factors were less likely to be vaccinated, our VE would be underestimated.

In summary, appropriate vaccination strategies, measures aimed at reducing risk behaviours,47 the notification of risks and community engagement efforts, early diagnosis, isolation and effective contact traceability continues to be essential to control this outbreak.10

Conclusions

Single-dose pre-exposure vaccination with the third-generation non-replicating smallpox vaccine is an effective measure against mpox. The progressive increase in effectiveness from the second week that persists from the fourth to the eighth week could justify the delay in the administration of the second vaccine dose (beyond the fourth week) due to a shortage of vaccines or prioritisation in the vaccination of the maximum number of subjects at risk against the administration of the booster dose.

The TND is a fast execution design that is adapted to surveillance data and ongoing outbreaks where relevant information is needed as soon as possible and easily allows the analysis of the time–response relationship of vaccination.

Further studies evaluating the long-term effectiveness of the full vaccination would be appropriate.