Full Text Article

Introduction

Since the outbreak of coronavirus disease 2019 (COVID-19), it has posed a threat to the public health system worldwide and thus far, it, unfortunately, has not been well controlled. As of 15 February 2022, a total of 408 million COVID-19 cases and more than 5.8 million related deaths have been reported [1]. COVID-19 not only seriously affects people's health and daily life, but also puts intense pressure on the society and economy [2]. We expect to return to pre-epidemic normalcy with joint efforts from the world.

Vaccines are among the most important preventive measures against pathogens including viruses, as they not only stimulate the production of antibodies to enhance the immunity in humans but also effectively prevent the spread of pathogens and facilitate the control of outbreaks [3-5]. According to global statistics, as of 7 February 2022, there were 338 vaccine candidates in regular use, of which 121 were in clinical trials and 27 were in regular use, including nine inactivated vaccines, eleven protein subunit vaccines, two RNA vaccines, four non-replicating viral vector vaccines and one DNA vaccine [6]. Compared to vaccines targeting other pathogens, vaccines against SARS-CoV-2 have undergone a much shorter period of development. Since the data from clinical trials of SARS-CoV-2 vaccines continue to be published, there is a critical need to continuously update the evaluation of vaccine safety and efficacy. Therefore, this study aimed to assess the safety and efficacy of different vaccines and the safety of vaccines in different age groups.

Methods

This study was registered at PROSPERO (CRD42021290415). This study was conducted in strict accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [7]. We also followed the Cochrane Handbook for Systematic Reviews of Interventions

Inclusion and Exclusion Criteria

We included the eligible studies published before November 11, 2021, and the main outcomes reported were the safety and efficacy of SARS-CoV-2 vaccines in the general population, including people of all ages. We restricted the inclusion of the literature to randomized controlled trials (RCT) and included both English and Chinese. In addition, we also included studies that reported the safety or efficacy of any of the COVID-19 vaccines as a booster dose following two doses of COVID-19 vaccines.

We excluded studies that had not been peer-reviewed, where full texts were not available, where data were partially duplicated, or where the detailed data of adverse effects were not reported. Studies without a non-COVID-19 vaccinated control group was excluded from this study.

Literature Search Strategy

For the published articles, we systematically searched included MEDINE (via PubMed), Embase, Web of Science, Elsevier, China National Knowledge Infrastructure (CINK), and WanFang Data. Furthermore, relevant articles from the first 10 pages of the Google Scholar search engine were selected. In this study the following combinations were used as search items: COVID-19, SARS-CoV-2, coronavirus, vaccine, safety, efficacy, side effects, effectiveness and randomized controlled trial. For all the databases, two researchers MG and JY independently performed the literature search. The supplementary file contains the complete search method for this study.

Literature Screening

The researchers YL and YJ conducted the post-search literature screening independently, and then discussed disagreements and resolved them with senior researcher JC. The process of selecting literature was carried out in three steps: 1) removing duplicates from search results; 2) screening titles and abstracts of studies to remove studies that do not fit the topic or do not report on the safety and efficacy; 3) reading the full texts to exclude studies with duplicate data samples, and those outcome indicators do not include the safety and efficacy.

Data Extraction

The following study data were extracted independently by two researchers MG and YJ: (1) basic information of the studies, including first author, publication date, code of trial registration and clinical trial phase; (2) study population and vaccines, including age group, sample sizes, country, types and dosage of SARS-CoV-2 vaccines; (3) results for the safety and effectiveness: incidence, type and number of adverse reactions associated with SARS-CoV-2 vaccines, and the incidence and amount of COVID-19 infections after vaccination.

Risk of Bias Assessment

We used the Cochrane Collaboration tool to assess the risk of bias in randomized trials when evaluating the original studies [8]. This assessment tool was designed to evaluate the RCT study design and implementation for selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases.

Data Synthesis and Analysis

We mainly aim to assess the safety and efficacy of different vaccines. For assessing the solicited adverse events following immunization (AEFI) in different vaccine groups and different age groups. Additionally, we also assessed the risk of bias for included studies. We used a Mantel-Haenszel model of random effects to evaluate the pooled effect sizes of three or more RCTs. Risk ratio (RR) and 95% confidence intervals (95% CI) were used to compare the effectiveness and safety of SARS-CoV-2 vaccines with those of the placebo group. The magnitude of I²was used to compare the heterogeneity between different vaccines for subgroup analysis (I² <25%, low heterogeneity; 25.0-75.0%, moderate heterogeneity; and I² >75.0%, considerable heterogeneity). Engage digitizer11.1 extracted data from the figures of the studies which did not have access detailed data. Review manager 5.3 and GraphPad Prism 8.0 were used for data collection, statistical analysis and diagram production.

Results

Literature Search

We identified 193 potential studies from Embase, 105 studies from Web of Science, 72 studies from Pubmed, 72 studies from Elsevier, 200 studies from CNKI and 36 studies from WAN FANG Data, respectively (Figure 1). From Google scholar, 35 potentially eligible studies were included. For the total of 606 records, 165 duplicates were excluded. After screening the titles and abstracts, we excluded another 386 studies which failed to meet our inclusion criteria. Among the 65 studies under the full-text review, 35 studies were excluded. Finally, this meta-analysis consisted of 30 eligible studies that reported the safety and effectiveness of various SARS-CoV-2 vaccines.

Studies Characteristics

Among the 30 included studies, 21 kinds of SARS-CoV-2 vaccines could be classified into four vaccine platforms (mRNA vaccines, Adenovirus vector vaccines, inactivated vaccines and Protein Subunit vaccines). Of all these studies about vaccine safety, there were four reports about mRNA vaccines [9-13], five about adenovirus vector-based vaccines [14-19], nine about inactivated vaccines [20-28], eight reports about protein subunit vaccines [29-36], and one study about different vaccines [37], respectively. Moreover, ten studies examined the effectiveness of SARS-CoV-2 vaccines[12,13,15,19,20,24,25,36,38,39]. Additionally four reported studies were about phase Ⅰ trial,[11,13,33,35] ten about Ⅰ/Ⅱ trial [14,17,21-23,26,27,31,32,40], six about phase Ⅱ trial[9,16,29,30,33,34], two about Ⅱ/Ⅲ tria [18,39] and seven about phase Ⅲ trial[12,15,19,20,24,25,36]. In the above eligible studies, a total of 93568 participants received at least one dose of SARS-CoV-2 vaccines and 55886 participants in the placebo group who received non-SARS-CoV-2 vaccines. The basic characteristics of the included RCT were described in table 1 (Table 1).

Quality of Included Studies

In the 30 peer-reviewed RCT studies, 23 studies adopted a double-blinded methodology [9,11,12,14-17,20-27,29-35,40], while five studies were single-blinded [13,18,36,37,39] and one study were unblended [19]. Additionally, one study used both single blind and double blind methods at different study stages [38]. All studies clearly explained the applied randomized assignment strategy. The summary of the analysis with the Cochrane Collaboration’s tool was: that the methodological bias in nine papers was low, the bias in twelve papers was moderate and the bias was high in the rest of the studies. The details of the methodological quality of all studies were presented in Appendix A.

Safety of COVID-19 Vaccines

The pooled AEFI proportion of all kinds of vaccines was 41.15% (RR=1.42, 95%CI: 1.23-1.64, I²=96%). Similarly, the incidences of mRNA vaccines, adenovirus vector-based vaccines, inactivated vaccines and protein subunit vaccines were 93.58% (RR=1.85, 95%CI: 1.34-2.55,I²=91%), 52.13% (RR=1.81, 95% CI: 1.56-2.10,I²=0%), 40.39% (RR=1.05,95% CI: 0.94-1.18,I²=69%) and 36.13% (RR=1.69, 95% CI: 1.13-2.53,I²=96%,), respectively (Figure 2).

We compared the local and systemic adverse effects of different vaccines between vaccinated and placebo groups. The pooled proportion of local adverse reactions to inactivated vaccines (18.94%) was significantly lower than mRNA vaccines (85.01%), adenovirus vector-based vaccines (60.42%) and protein subunit vaccines (56.99%), respectively. For the incidence of systemic adverse reactions, the highest was mRNA vaccines (80.51%). Pooled RRs of local and systemic adverse reactions of all types of vaccines were 2.77 (95%CI: 1.68-4.55) and 1.27 (95%CI: 1.07-1.51), respectively. Furthermore, we found the heterogeneity was considerable in the meta-analysis (I²=98% for local reaction and 95% for systemic reaction) (Table 2).

In the subgroup analysis of different adverse reactions, we primarily evaluated in detail the local symptoms including pain, redness and swelling, and the systemic reactions including fever, headache, fatigue, myalgia, arthralgia, and chills and vomiting. For all kinds of vaccines, the most common adverse effect was pain at the injection site. In this meta-analysis, we found the RRs of the mRNA-based vaccines were greatly higher than other vaccines, in the aspects of pain (RR=5.57, 95%CI 2.65-11.70), redness (RR=5.03, 95%CI 2.14-11.80), swelling (RR=7.56, 95%CI 4.65-12.29) and fever (RR=10.22, 95%CI 6.40-16.30), respectively. However, the heterogeneity of the inactivated vaccines on redness (I²= 0%), swelling (I²= 0%), fever (I²= 0%), fatigue (I²= 0%) and vomiting (I²= 33%) were lower than other vaccines (Table 2).

Ten studies reported adverse reactions in the different age groups [9,11,12,14,18,22,27,30,31,34]. We divided general population into three age groups (<18y, 18-55y and >55y). Two studies were grouped in <18 years old. Figure 3 was shown that the total incidence of adverse reactions was 72.74% in the <18y group, 63.27% in the 18-55y group and 42.02% in the>55y group, respectively. It is obvious that the proportion of adverse effects in the <18 years old group was much higher, regardless of local or systemic reactions (Figure. 3). Furthermore, the incidence of grade 3 and above AEFI was significantly higher in children and adolescents (8.23%) than in adults (1.57%) and the elderly (1.01%). Adverse event classification standards of COVID-19 vaccines were evaluated according to Guidelines for Classification Standards of Adverse Events in Clinical Trials of Prophylactic Vaccines which was issued by the National Medical Products Administration [41].

The analysis of reported studies about SARS-CoV-2 vaccines revealed that the incidence of grade 3 AEFI observed for mRNA vaccines (21.55%) and adenovirus vector-based vaccines (5.07%) were higher than those of inactivated vaccines (0.71%) or protein subunit vaccine (2.91%) (Table 2). Furthermore, the heterogeneity of adverse reactions caused by mRNA vaccines was considerable (I²=88%), while this heterogeneity was not observed for the inactivated vaccines (I²=0%). Additionally, eight studies reported hypersensitivity reaction, liver injury, cerebrovascular accident, myocardial infarction and et al (Table 2).

Efficacy of COVID-19 vaccines

Ten studies reported the efficacy of SARS-CoV-2 vaccines. Two studies evaluated the mRNA vaccine (BNT162b2) and the protein subunit vaccines (NVX-CoV2373), respectively. Three reports examined the efficacy of inactivated vaccines (CoronaVac, WIV04 and HB02). Five studies evaluated the adenovirus vector vaccines (three about ChAdOx1 Nov-19, one about CVnCoV SARS-CoV-2 and one about rAd26/rAd5) (Figure 4). The pooled efficacy of the random effects for all types of vaccines was 68% (95%CI: 57.0-76.0). Of all the studies, mRNA vaccines conferred the best effectiveness, which is 97% (95%CI: 56.0-100.0) in the 12-15-year-old teenage group. The second most effective vaccine after the mRNA vaccine is the protein subunit vaccine (90%, 95%CI: 79.0-95.0). The efficacy of all inactivated vaccines was 60% (95%CI: 41.0-73.0) and the adenovirus vector vaccines were 65% (95%CI: 49.0-75.0) (Figure 4).

Discussion

This study is one of the comprehensive systematic reviews of high-level evidence on the efficacy and safety of various COVID-19 vaccines. The statistically significant differences in safety were observed among four different platform-based vaccines. But serious adverse reactions to AEFI vaccinations including mRNA vaccines, inactivated vaccines, and other vaccines, are unusual, and the most common adverse reactions are mild. Additionally, the pooled efficacy of all vaccines based on four platforms showed that the most effective vaccine after at least 7 days post-immunization is the mRNA vaccine. In terms of vaccine safety and efficacy, COVID-19 vaccination remains a proven strategy to control the epidemic.

It is important to clarify the severity and incidence of adverse reactions in response to vaccinations. The safety of vaccines would impact the willingness of the general population to receive vaccines, and there continues to be a high rate of hesitancy and reluctance to receive vaccines [42, 43]. In this meta-analysis, the most common local symptoms of anti-COVID-19 vaccination were pain and redness at the injection site, while systemic symptoms are mainly mild fever and headache. It’s similar to the flu vaccination, in which most of the symptoms of adverse reactions to the flu vaccines are mild, with few serious adverse reactions occurring [44, 45]. These symptoms usually resolve spontaneously with time. To further clarify the safety of the different vaccines we performed detailed subgroup analysis.

For the analysis of the different vaccine subgroups, we found the incidence of AEFI to mRNA vaccines is significantly higher, both for local and systemic symptoms. Especially for the solicited grade 3 AEFI, mRNA (20.59%) SARS-CoV-2 vaccines exhibited higher rates of adverse reactions compared to inactivated, adenovirus vector-based, and protein subunit vaccines (Table 2). In a multicenter RCT study that included ten countries, Kremsner et al found that the incidence of grade3 events of mRNA vaccine (CVnCoV) was 27.1% and the median duration of adverse reactions was 1-2 days after mRNA vaccination in most patients but did not have long-term effects.13 However, due to the possibility of acute diseases or accidental injuries, unsolicited serious adverse reactions (SAEs) are rare but cannot be ignored. In this meta-analysis, eight studies reported unsolicited SAEs which included the local and systemic adverse reactions reported in the clinical trials, as well as the SAEs observed only in rare subjects such as hypersensitivity reactions, cardiovascular diseases, allergic reaction, visual organ disturbances and anaphylaxis [13,15,17,24,25,37,38]. For any vaccination, allergic reactions are an important event that requires the attention. Previous studies revealed that vaccination-associated anaphylaxis was uncommon, occurring around once per 1 million immunizations for most of the known vaccines [46]. Although several mRNA vaccines of SARS-CoV-2 have been approved for clinical application, the mechanisms of allergic reactions remain unclear. Currently, vaccinations are only recommended for the majority of the population who neither has a history of allergy associated with vaccines nor allergic reactions to mRNA vaccine components [47]. Furthermore, in a pooled analysis of four RCT studies, Voysey. et al. found that the incidence of cardiac disorders with ChAdOx1 to-19 vaccination is 0.04% (5/12021). There have also been case reports of possible myocarditis or pericarditis with the second dose of the mRNA-1273 vaccine developed by Pfizer [48,49].

For the analysis of the age subgroup, there may be differences in the safety of SARS-CoV-2 vaccines for different age groups. Frenck13 et al. assessed the safety of an mRNA vaccine (BNT162b2) in adolescents with a high proportion of AEFI [12]. The pooled incidence of AEFI in our studies shows that the vaccines induced adverse reactions more strongly in adolescents aged 12-15 years than in adults (Figure 3). This observation is consistent with that proposed by Cai and Andrew et al, who found that young people appeared to be more susceptible to higher-level AEFI and speculated that younger people have stronger immune systems, leading to a higher frequency of ADRs and better vaccination outcomes [50, 51]. But there was no relationship between vaccination and any of the SAEs documented in our studies. Additionally, we found that the solicited AEFI in the elderly group (>55y) was lower than that of adolescents and adults (18-55y), such to injection site pain, headache, chill and grade3 AEFI (Figure 3). Overall, data from RCT studies have given reassuring safety profiles but recruited few frailties older and younger participants. With very sparse data reported, especially for young people, only two RCT studies have been published and separately confirmed that CoronaVac and BNT162b2 are safe for the younger(<18y). We still need to wait for more evidence from more regions and countries to confirm the safety of the vaccine in young and elderly people to better protect them from COVID-19.

The effectiveness profiles must be considered when evaluating the safety of SARS-CoV-2 vaccines. The mRNA and the protein subunit vaccines against SARS-CoV-2 reported the efficacy of 97% and 90%, respectively, and vaccines based on these two platforms are significantly more effective than inactivated and adenovirus vector vaccines (Figure 4). Here we precisely found that although mRNA vaccination is more effective, the higher incidence of observed adverse reactions poses an important challenge in promoting its application. The effectiveness of protein subunit vaccines is similar to that of mRNA vaccines, but the overall incidence of adverse reactions is only 36.13% and the level of grade 3 or higher AEFI is 2.91% (Table 2). Thus, the overall frequency of adverse reactions for adenovirus vector-based vaccines is considerably lower than that of the mRNA vaccine. Therefore, the protein subunit vaccines may be a superior option, when it comes to the combined evaluation of safety and efficacy. In summary, regardless of the platforms on which SARS-CoV-2 vaccines are generated or the population to which it is administered, vaccination effectively protects the immunized people from COVID-19 symptoms in most cases and reduces the hospitalization rates, serious disease and death, while the protein subunit vaccines may be a better option.

The main limitation of this study is the failure to evaluate the efficacy of the protein subunit vaccine due to the lack of clinical studies in terms of its effectiveness. Because some of the studies were not RCT, they were excluded from this meta-analysis. Secondly, some studies biased reporting of outcomes, such as not reporting the overall incidence of adverse reactions, local or systemic adverse reactions, and serious adverse events. Finally, we did not summarize those symptoms mentioned in the supplemental files besides the adverse reactions routinely reported in clinical trials.

Conclusions

Our study sheds new light on the current status of the SARS-CoV-2 vaccine. Adverse reactions to the vaccine are generally mild, but ongoing attention is needed for the rare unsolicited vaccination-related symptoms. Vaccination could effectively reduce infection and hospitalization rates. In particular, we expect that more data will be reported on the monitoring of vaccine safety in infants, adolescents, and the elderly. Despite the ongoing mutation of the SARS-CoV-2 variant, we still believe from the data that vaccination is a necessary and important tool in the fight against the COVID-19 pandemic.

Acknowledgment

A special thanks to the study participants for their contribution to the research.

Youli Jiang(Y.J.), Jingfang Chen(J.C.), Yalong Chen(Y.C.), Ya Li(Y.L.), Meng Lv(M.L.), Xufei Luo(X.L.), Min Gui(M.G.), Qing He(Q.H.)

Author Contributions

Conceptualization: Q.H., and Y.C.; Study design: Y.C. and M.L.; literature search: Y.J., J.C. and Y.L.; figures and tables: Y.L., M.G.; data collection: Y.L. and J.C.; data analysis: Q.H., J.C. and Y.J.; data interpretation: Q.H., J.C. and Y.J.; writing: Y.J., M.L and X.L.supervision: Y.C. and Q.H.; All authors provided input regarding the direction of the study and the content of the paper. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the Team of Wang Fusheng, academician of Shenzhen “Three Medical and Health Project” (SZSM201612014). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the paper. No payment was received by any of the co-authors for the preparation of this article.

Institutional Review Board Statement: Not applicable.

Informed Consent Statement: Not applicable.

Data Availability Statement: The datasets used during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflicts of interest.