Cost-effectiveness of two-dose human papillomavirus vaccination in Singapore



1Department of Obstetrics and Gynaecology, Singapore General Hospital, Singapore


2Duke-NUS Medical School, Singapore


3Department of Paediatrics, NUS Yong Loo Lin School of Medicine, National University of Singapore, Singapore


4Medical Affairs, GSK, Singapore


5Health Economics, GSK, Singapore


6Health Economics, GSK, Wavre, Belgium


Correspondence: Mr Georges Van Kriekinge, Health Economics, GSK, 20 Avenue Fleming, 1300 Wavre, Belgium. [email protected]


INTRODUCTION

Cervical cancer is the tenth most common cancer and the eighth most frequent cause of death among women in Singapore. As human papillomavirus (HPV) infection is the necessary cause of cervical cancer, the risk of cervical cancer can be substantially reduced through vaccination. This study was conducted to evaluate the cost-effectiveness of two-dose HPV vaccination as part of a national vaccination programme for 12-year-old girls in Singapore, from the perspective of the healthcare payer.


METHODS

A lifetime Markov cohort model was used to evaluate the cost-effectiveness of introducing the AS04-adjuvanted HPV-16/18 vaccine (AS04-HPV-16/18v) to the current cervical screening programme in Singapore. Furthermore, the cost-effectiveness of the AS04-HPV-16/18v was compared with the HPV-6/11/16/18 vaccine (4vHPV). Model inputs were derived from local data, where possible, and validated by clinical experts in Singapore.


RESULTS

Introduction of the AS04-HPV-16/18v in Singapore was shown to prevent 137 cervical cancer cases and 48 cervical cancer deaths when compared with screening alone. This resulted in an incremental cost-effectiveness ratio of SGD 12,645 per quality-adjusted life year (QALY) gained, which is cost-effective according to the World Health Organization threshold for Singapore. When discounted at 3%, AS04-HPV-16/18v was dominant over 4vHPV, with cost savings of SGD 80,559 and 28 additional QALYs gained. In the one-way sensitivity analysis, AS04-HPV-16/18v remained cost-effective compared with screening alone and dominant compared with 4vHPV.


CONCLUSION

AS04-HPV-16/18v is the most cost-effective choice for reducing the burden of cervical cancer through universal mass vaccination for 12-year-old girls in Singapore.



INTRODUCTION

Cervical cancer is the tenth most common cancer and the eighth most frequent cause of death among women in Singapore.(1) Between 2010 and 2014, there were 1,005 patients with cervical cancer and 357 deaths attributed to cervical cancer.(1) The economic burden associated with the treatment of cervical cancer in Singapore is also substantial, with total costs estimated to be SGD 57.6 million over a period of 25 years starting from 2008.(2) Over the same period, an additional SGD 25.5 million was associated with the treatment and management of pre-malignant stages and genital warts (GWs) caused by human papillomavirus (HPV) infection.(2)

There are multiple types of HPV and at least 13 of them are considered oncogenic due to their association with cervical cancer.(3) Although the majority of HPV infections are self-limiting and often asymptomatic, persistent infection with oncogenic HPV types may lead to the development of pre-malignant low-grade cervical intraepithelial neoplasia (CIN1onc), followed by moderate-grade cervical intraepithelial neoplasia (CIN2), high-grade cervical intraepithelial neoplasia (CIN3) and subsequently cervical cancer.(4) The low-risk HPV types do not cause cervical cancer, but are instead associated with GWs and low-grade cervical intraepithelial neoplasia (CIN1).(4) As HPV infection is the necessary cause of cervical cancer,(5,6) the risk of cervical cancer can be substantially reduced through vaccination.(7,8) In addition, cervical screening to detect pre-malignant cervical intraepithelial neoplasia (CIN) lesions may allow early treatment by ablation or excision before they enter the malignant state.(9,10) It may therefore be possible to alleviate the majority of the cervical cancer burden through a combination of vaccination and screening programmes.(79)

In a national cervical cancer screening programme that has been available in Singapore since 2004, eligible women are recommended to undergo Pap smear tests once every three years.(9,10) The screening cost (SGD 15–24) is paid out of pocket, except for low-income citizens aged above 40 years, who have been fully subsidised since 2014.(11) Opportunistic screening is sometimes provided for free through charitable organisations.(12) However, the uptake of cervical screening, which is estimated to be in the range of 42%–50%,(1315) has been below the Singapore Ministry of Health’s (MOH Singapore) target of 80%.(15)

Since 2014, two-dose schedules of both the AS04-adjuvanted HPV-16/18 vaccine (AS04-HPV-16/18v) (Cervarix; GSK, Rixensart, Belgium) and the HPV-6/11/16/18 vaccine (4vHPV) (Gardasil; Merck and Co. Inc, NJ, USA) have been approved in Singapore for girls aged 9–14 years and 9–13 years, respectively.(16) HPV vaccination is available upon request from public and private clinics, and is reimbursed through an individual’s medical savings account (Medisave).(17) So far, the uptake of HPV vaccination has also been low; only 13.6% of women aged 18–26 years have been immunised.(18) The reasons contributing to the low uptake are unclear, but inconvenience, low awareness of HPV and the benefits of HPV vaccination, along with misconceptions about the safety of HPV vaccination have been raised as potential barriers to uptake.(14,19)

A recent meta-analysis concluded that AS04-HPV-16/18v was more efficacious compared with 4vHPV in terms of protection against the pre-malignant stages CIN2 (65% vs. 43%, respectively) and CIN3 (93% vs. 43%, respectively), irrespective of the causative HPV type.(20) On the other hand, 4vHPV provides protection against HPV-6 and HPV-11 infections, thereby reducing the risk of GWs.(4) Recent studies have suggested that AS04-HPV-16/18v also provides moderate cross-protection against persistent infection by HPV-6 and HPV-11.(2124)

The cost-effectiveness of both AS04-HPV-16/18v and 4vHPV has been evaluated in a number of countries in order to establish the value of introducing nationwide immunisation programmes,(2529) in accordance with the World Health Organization’s (WHO) recommendation to first establish the cost-effectiveness of a new vaccine in the country before introduction.(30) While the effectiveness and cost-effectiveness of the three-dose vaccine regimens have been evaluated for Singapore,(31,32) there are no such cost-effectiveness studies on the two-dose vaccine regimens to our knowledge. A national HPV vaccination programme is currently under evaluation by MOH Singapore. Given the previous success of school-based vaccination programmes in Singapore – including the Tdap programme (tetanus toxoid, reduced diphtheria toxoid and acellular pertussis vaccination) involving the vaccination of 11-year-old children by School Health Services nurses, which achieved a coverage of 91.4% in 2013(33) – it was of interest to evaluate the cost-effectiveness of the two-dose HPV vaccination as part of a school-based vaccination programme.


METHODS

A previously published lifetime Markov cohort model was adapted to the Singapore setting.(26,34) The model was based on three fundamental assumptions: (a) HPV infection is associated with cervical cancer; (b) screening and early detection of cervical lesions impact the natural history of cervical cancer; and (c) vaccination alters the natural history of disease at infection. 12-year-old girls were chosen as the cohort of study for the model due to the presence of an established Tdap vaccination programme for this age group in Singapore. HPV vaccination of the same cohort would avoid additional costs associated with establishing a separate vaccination programme for a different age cohort. The model consisted of 95 cycles of one year each, which covered the lifetime of the cohort and captured the total benefit associated with prevention of cancer.

We evaluated the cost-effectiveness of introducing AS04-HPV-16/18v in addition to the current cervical screening programme in Singapore. Furthermore, we examined the cost-effectiveness of AS04-HPV-16/18v compared with 4vHPV, both of which are available in Singapore. The analyses were carried out from the perspective of the healthcare payer, which is MOH Singapore. Thus, only direct medical costs, such as costs of hospitalisation, screening tests and procedures, and vaccine costs were included. In accordance with previous health economic evaluations in Singapore, a discount rate of 3% was applied to both costs and benefits.(31,35,36) A discount rate of 1.5% was also explored, as described in Scenario Analysis I later in this article.

Prof Tay SK, Singapore General Hospital and Duke-NUS Medical School, Singapore, and Prof Lee BW, National University of Singapore, Singapore, validated the model data input and assumptions during a roundtable discussion held on 29 January 2014 and throughout the study period. In addition, the incidence of cervical cancer, cervical cancer mortality rate and incidence of GWs per age group, as estimated by the model, were compared with external sources for validation.

The model structure has been described in previous publications.(26,34) In short, participants enter the model in the non-infected state (NoHPV), and with each cycle of the model, there is a probability of remaining uninfected or of becoming infected and transitioning to the oncogenic infection state (HPVonc) and/or the low-risk infection state (HPVlr). Participants in HPVonc may remain in this state, return to NoHPV or move through the pre-malignant states (CIN1, CIN2/3 and persistent CIN2/3) before reaching the cervical cancer state and cervical cancer death. If the pre-malignant lesion is detected through screening, participants move to the corresponding detected CIN states, which are associated with a higher probability of returning to NoHPV, reflecting the impact of medical follow-up. Participants in HPVlr may remain in the same disease state, return to NoHPV, or experience GWs or low-risk cervical intraepithelial neoplasia 1 (CIN1lr). CIN1lr lesions may be detected, hence enabling medical intervention that is associated with a higher probability of returning to NoHPV.

Markov models are suitable for modelling cervical cancer, as they are able to model the long natural history of HPV infection, as well as the effect of early detection through screening and immunisation through vaccination.(37) Age-specific mortality rates from the Singapore general population in 2013 were used in the model.(38)

No national statistics could be identified for GWs in Singapore. The incidence of female GWs was estimated from the Communicable Diseases Surveillance in Singapore in 2013 report, by MOH Singapore.(33) The report quoted an incidence of 33.5 cases in males and 6.6 cases in females per 100,000 population. The discrepancy in incidence is likely to be a reflection of different treatment providers for GWs in men and women in Singapore. It was therefore agreed that the higher rate of 33.5 cases per 100,000 population should be used as a conservative estimate of the female GW incidence in Singapore. The overall GW incidence was further stratified by age group according to the female GW distribution in Japan, as a distribution from Singapore could not be found.(33,39)

The cohort of 12-year-old girls in 2013 was estimated to be 22,000.(40) Transition probabilities between the health states, representing the natural history of the disease, are shown in Table I. Cervical cancer screening variables were based on guidelines in Singapore, data from the literature and local clinical expert opinion (Table II).

Table I

Table II

The overall vaccine effectiveness used in the model was calculated based on data from pivotal trials and the literature,(8,5357) according to the formula:

where %HPV represented the HPV type prevalence and VE represented the efficacy of the vaccine against the specific HPV type. Table III further details the effectiveness of each vaccine for CIN1, GWs, CIN2/3 and cervical cancer. As reflected by the differences in vaccine efficacy, the AS04-HPV-16/18v is formulated with the AS04 adjuvant, which stimulates a greater immune response compared with the traditional aluminium salt adjuvants used for the 4vHPV.(5860) It has been suggested that differences in the magnitude of immune response between vaccines determine the duration of protection, although such differences were not accounted for in the current model.(59) Vaccination coverage was assumed to be 90%, based on local experience with Tdap vaccination.(33)

Table III

Due to the lack of published HPV-related disutility weights for the Singaporean population, disutility weights for pre-malignant stages and cervical cancer stages from other countries were applied. Disutilities were assumed to be constant over time and they were subtracted from a utility value of 1 for all ages. The disutility value per disease stage was adjusted for the duration of each lesion type to represent the disutility over a single year, to fit the cycle length of the model. For disease stages that lasted for more than one year (i.e. cervical cancer), the disutility was applied to all consecutive cycles during which the disease stage was active (Table IV).

Table IV

Treatment costs for CIN lesions and for cervical cancer (Table V) were taken from a previously published study on the economic burden of HPV in Singapore.(2) The published lifetime cost of cervical cancer was adjusted to the cost per year by dividing the lifetime cost by the average duration of a cervical cancer case in the model (i.e. 3.34 years). Adjustment of costs to the cost year, 2014, was not considered necessary, as the costs were validated by external experts to be still relevant at the time of study. A vaccine cost of SGD 90 per dose was assumed (i.e. SGD 180 per vaccinated girl under the two-dose schedule). Price parity between the two vaccines was assumed to ensure that results would be due to clinical differentiators and not vaccine price differentials. The main health benefits of interest, quality-adjusted life years (QALYs) and life years (LYs) gained were computed in the model alongside the number of CIN1 and CIN2/3 cases (as detected by screening), GW cases, cervical cancer cases and cervical cancer deaths averted, as well as associated costs over the cohort’s lifetime. The incremental cost-effectiveness ratio (ICER) threshold of 1–3 times the gross domestic product (GDP) per capita was used, as recommended by the WHO.(65)

Table V

Two analyses were conducted: (a) vaccination with AS04-HPV-16/18v added to the current screening programme compared with screening alone; and (b) vaccination with 4vHPV compared with AS04-HPV-16/18v. One-way deterministic sensitivity analyses were performed in order to evaluate the robustness of the results to changes in the model inputs. All base-case variables were varied ± 20%, with the exception of vaccine efficacy related to cross-protection, which was varied to the upper and lower 95% confidence intervals reported.

In addition, two scenario analyses were conducted. Scenario Analysis I was conducted to investigate the impact of applying an alternative discount rate of 1.5%, as recommended by the recently updated National Institute for Health and Care Excellence (NICE) United Kingdom (UK), guidelines for interventions with long-term benefits exceeding 30 years.(66,67) Scenario Analysis II aimed to investigate the impact of applying alternative GW incidence, vaccine and GW treatment costs, and disutility weights, as applied by Lee et al(31) in 2011 (Table VI).

Table VI

Alternative input variables (adapted from Lee et al)(31) for Scenario Analysis II.


RESULTS

The model adequately reproduced age-dependent cervical cancer incidence, cervical cancer mortality and female GW incidence in Singapore when compared with published data (Figs. 1–3).

Fig. 1

Chart shows observed and modelled age-stratified incidence of cervical cancer among women in Singapore. (Data from Singapore Cancer Trends Reports, 2009–2013.)(68)

Fig. 2

Chart shows observed and modelled age-stratified mortality of cervical cancer among women in Singapore. (Data from International Agency for Research on Cancer, GLOBOCAN, Singapore, 2012.)(47)

Fig. 3

Chart shows observed and modelled age-stratified incidence of genital warts among women in Singapore. (Data from Communicable Diseases Surveillance in Singapore 2013;(33) Kumamoto et al 2004.)(39)

The introduction of AS04-HPV-16/18v, in addition to the current cervical screening programme, compared with screening alone, resulted in a gain of 1,314 QALYs (undiscounted) over the lifetime of the cohort (Table VII). These QALY gains were attributed to fewer cases of CIN1 and CIN2/3 (as detected by screening), as well as to the avoidance of cervical cancer cases (137 avoided) and cervical cancer deaths (48 avoided). With the introduction of AS04-HPV-16/18v in addition to screening, an additional cost of SGD 3,564,000 was estimated for vaccine acquisition. However, there were important cost savings associated with CIN and cervical cancer treatment that amounted to SGD 2,163,130 (undiscounted), resulting in a net cost of SGD 1,769,338 (undiscounted). Discounted at 3% annually, a total of 243 QALYs were gained for a total net cost of SGD 3,072,752. The ICER of AS04-HPV-16/18v in addition to screening, compared with screening alone, was therefore SGD 12,645 per QALY gained. Thus, AS04-HPV-16/18v is considered highly cost-effective according to the WHO threshold (< 1 times GDP/capita [or SGD 70,967 in 2014]).(65,69)

Table VII

When comparing the effectiveness of AS04-HPV-16/18v with that of 4vHPV, in addition to screening in both cases, AS04-HPV-16/18v was found to be more effective for avoiding CIN, cervical cancer cases and cervical cancer deaths, whereas 4vHPV was more effective for avoiding GW cases. Before discounting, AS04-HPV-16/18v generated 174 additional QALYs compared with 4vHPV at a lower cost (cost savings of SGD 379,709). The additional costs associated with treatment of GW cases with AS04-HPV-16/18v (SGD 112,115) were offset by cost savings due to the lower treatment costs for CIN and cervical cancer cases (SGD 372,271). After discounting at 3%, vaccination with AS04-HPV-16/18v generated 28 more QALYs and led to a total cost savings of SGD 80,559 compared with 4vHPV. This makes AS04-HPV-16/18v the dominant choice (more effective and less costly when compared with 4vHPV) for the introduction of universal mass vaccination for 12-year-old girls in Singapore.

Results of the one-way sensitivity analyses showed the discount rate to be the most influential factor on the ICER for the comparison of AS04-HPV-16/18v versus screening alone (Fig. 4). Similarly, the discount rate was the factor that had the greatest influence on the difference in total QALYs for the comparison between AS04-HPV-16/18v and 4vHPV (Fig. 5). The most influential factor on the difference in total costs for this comparison was cervical screening frequency, which was closely followed by the discount rate (Fig. 6). Despite variations in the model inputs, AS04-HPV-16/18v remained cost-effective when compared with screening alone and AS04-HPV-16/18v still provided more cost savings and was more effective than 4vHPV in all sensitivity analyses.

Fig. 4

Diagram shows one-way sensitivity analysis of the ten most influential factors for AS04-HPV-16/18v versus screening alone. AS04-HPV-16/18v: AS04-adjuvanted HPV-16/18 vaccine; HPV: human papillomavirus; ICER: incremental cost-effectiveness ratio

Fig. 5

Diagram shows one-way sensitivity analysis of the ten most influential factors on QALYs for AS04-HPV-16/18v vs. 4vHPV. 4vHPV: HPV-6/11/16/18 vaccine; AS04-HPV-16/18v: AS04-adjuvanted HPV-16/18 vaccine; HPV: human papillomavirus; QALY: quality-adjusted life year

Fig. 6

Diagram shows one-way sensitivity analysis of the ten most influential factors on costs for AS04-HPV-16/18v vs. 4vHPV. 4vHPV: HPV-6/11/16/18 vaccine; AS04-HPV-16/18v: AS04-adjuvanted HPV-16/18 vaccine; CIN1: low-grade cervical intraepithelial neoplasia; HPV: human papillomavirus

When a 1.5% discount rate was applied in Scenario Analysis I, total costs as well as total QALY gains increased for all three options (i.e. screening alone, AS04-HPV-16/18v and 4vHPV) (Table VIII). Results for AS04-HPV-16/18v were even more favourable at the discount rate used in this scenario analysis when compared with the base case. When input variables from Lee et al(31) were applied in Scenario Analysis II, total discounted costs increased for all options, while total discounted QALY gains slightly decreased. In this scenario, AS04-HPV-16/18v remained cost-effective compared with screening alone (ICER per QALY: SGD 17,226 < 1 times GDP/capita [or SGD 70,967 in 2014]).(65,69) However, in contrast to the base-case analysis, when input variables from Lee et al(31) were used, 4vHPV dominated AS04-HPV-16/18v due to both overall cost savings and overall QALY gains. On the other hand, when the cost-effectiveness results were evaluated in terms of cost per LY gained, AS04-HPV-16/18v was cost-effective when compared with 4vHPV (26 LYs gained; ICER: SGD 97,208 per LY gained, < 3 times GDP per capita or SGD 212,901 in 2014).(65,69)

Table VIII


DISCUSSION

This study showed that introducing a national AS04-HPV-16/18v vaccination programme for 12-year-old girls in Singapore, in addition to the existing cervical cancer screening programme, could be highly cost-effective compared with the current practice of screening and treatment alone. Furthermore, this study also showed that AS04-HPV-16/18v dominates 4vHPV, suggesting that AS04-HPV-16/18v would deliver greater value for investment if a reimbursed universal mass HPV vaccination became available for 12-year-old girls in Singapore. The results of one-way sensitivity analyses confirmed that the model was robust to uncertainty in the input variables. The most influential input variable was the discount rate, which was due to the long time horizon of the model combined with the time lag between vaccination and protection against cervical cancer.

To ensure that the model input was applicable to the healthcare setting in Singapore, the treatment pathways, costs and distribution of HPV types, as well as other data input, were derived from local data wherever possible. The data, along with input derived from other countries when local data was not available, was validated by local clinical experts with extensive clinical experience in treating cervical cancer in Singapore in order to ensure that the input was accurate at the time of study. In the absence of local data, the utilities applied in this model were derived from Europe and the United States, introducing potential generalisability concerns. As with all data input, local data is preferred, but in the absence of such data, guidance from the International Society for Pharmacoeconomics and Outcomes Research recommends the use of published utility weights from other countries.(70)

HPV type prevalence varies across the world and the effectiveness of vaccines may be variable depending on the distribution of HPV types in cervical lesions. It is therefore important to consider the local or regional HPV type prevalence when estimating overall vaccine effectiveness. As the model used the HPV distribution for South Eastern Asia and Singapore, it was able to estimate the vaccine effectiveness specific to the Singapore setting.

Markov cohort models are transparent and have the added advantage of requiring a limited number of parameter inputs.(37) This is important in settings where input-rich, more complex models may be difficult to populate and interpret. The static cohort design leads to conservative estimates of benefits from vaccination, as the model does not account for dynamic effects, such as herd immunity and changes in population behaviour.(37) As such, it is likely that the current model underestimated the true benefits of vaccination. Such underestimation may have been counter-balanced by potential overestimation in the model due to a lack of waning and the assumption of lifelong protection once vaccinated. Although the phenomenon of waning is uncertain,(29) it is included in many modelling studies in which the estimated benefit of vaccination is reduced as a result.

Discounting is applied to health economic evaluations in order to account for an intrinsic preference among the general population for immediate benefits compared with benefits in the distant future.(71) To avoid undervaluing long-term benefits, the updated guidelines by NICE, UK, recommend applying a discount rate of 1.5%, instead of the usually applied 3.5% in the UK, when health benefits are likely to be achieved over a duration of more than 30 years.(66,67) The results from our sensitivity analysis showed that the discount rate had a major influence on both the total costs and total QALYs, and thus the ICER. By comparing the base-case analysis (3% discount rate) with the 1.5% discount rate scenario, an additional 32% of the discounted QALY gains and costs were retained for both AS04-HPV-16/18v and 4vHPV.

Cervical cancer can take many years to develop. In Singapore, its peak incidence occurs in women in the 45–54 years age group.(1) In contrast, GWs can occur much sooner after HPV infection, with a peak incidence of GWs among women aged 20–39 years in Singapore.(2) Benefits from avoiding cervical cancer (in the long term) and GWs (in the short term) are weighted differently depending on the discount rate chosen, as observed in our analyses (3% in base case, 1.5% in scenario analysis) – a higher proportion of GW-associated QALY gains (base case: 60%, scenario analysis: 77%) were retained after discounting when compared with cervical cancer-associated QALY gains (base case: 18%, scenario analysis: 42%) (Fig. 7). A high discount rate diminishes long-term benefits and overvalues the importance of short-term benefits, while a lower discount rate partially restores the balance. In Singapore, there is no official discount rate recommendation; the most appropriate discount rate depends on the policy objective of the vaccination programme and the relative value the society places on the prevention of cervical cancer versus GWs.

Fig. 7

Graph shows scenario analysis of the impact of applying discount rates of 0%, 1.5% and 3.0% on QALY gains associated with genital warts (for 4vHPV) and cervical cancer (for AS04-HPV-16/18v). 4vHPV: HPV-6/11/16/18 vaccine; AS04-HPV-16/18v: AS04-adjuvanted HPV-16/18 vaccine; HPV: human papillomavirus; QALY: quality-adjusted life year

While the results from our evaluation, which concluded that AS04-HPV-16/18v is cost-effective when compared with screening alone, concurred with those reported by Lee et al(31) for a three-dose vaccination schedule, Lee et al differed from our analysis in concluding that 4vHPV was dominant over AS04-HPV-16/18v in terms of cost per QALY gained. However, when the ICER was calculated in terms of cost per LY gained, Lee et al found AS04-HPV-16/18v to be more cost-effective when compared with 4vHPV, due to the focus on LYs and the exclusion of GW-associated QALY changes.(31)

Some of the key differences between the present model and the model described by Lee et al(31) relate to GW-associated input variables (e.g. incidence, disutility and cost of treatment). When applying the input variables from Lee et al to our model, substantial differences were observed in the total number of GW cases (AS04-HPV-16/18v: n = 3,773; 4vHPV: n = 804) when compared with our base case (AS04-HPV-16/18v: n = 424; 4vHPV: n = 90). The higher incidence of GWs was likely to have favoured 4vHPV due to its efficacy against GW-associated HPV-6 and HPV-11 infections. The GW incidence in Lee et al,(31) derived from international studies with a low proportion of women living in Asia,(56,72) was several-fold higher compared with the incidence in our model, and was likely to be one of the major contributors to the contradictory results. Furthermore, the greater disutility and greater cost of GW treatment is likely to have further amplified the economic impact of GWs. Data associated with GWs is difficult to source in Asia. The incidence of GWs in our model was derived from an MOH Singapore report in 2013,(33) and we chose to apply the higher incidence rate for males (33.5 cases per 100,000 population) rather than that for females or the total incidence (6.6 or 20.7 cases per 100,000 population, respectively) in order to take a conservative approach with our analysis. The application of the female rate (6.6 cases per 100,000 women), as an alternative, was shown to lead to minor differences in the results (data not shown).

Unsurprisingly, when the input variables from Lee et al(31) were applied to our model, the results mirrored those of the earlier study – AS04-HPV-16/18v was cost-effective when compared with screening alone, and 4vHPV was dominant over AS04-HPV-16/18v in terms of cost per QALY gained, while AS04-HPV-16/18v was cost-effective when compared with 4vHPV in terms of cost per LY gained.

A further difference between both models was the rate of cross-protection against oncogenic HPV types, other than HPV-16 and HPV-18, applied to AS04-HPV-16/18v. Lee et al(31) applied a non-vaccine-specific rate of cross-protection (23.4%) to both vaccines, whereas vaccine-specific rates reflecting the difference in the rate of cross-protection by AS04-HPV-16/18v (48%) and 4vHPV (23%) were used in the current model.

Observational data and a post-hoc analysis of a Phase III clinical trial of AS04-HPV-16/18v suggest that AS04-HPV-16/18v also provides moderate protection against the low-risk HPV-6 and HPV-11 infections.(21,22,73) This additional cross-protection against these low-risk HPV types was not included in our model. Therefore, the current results are likely to be a conservative estimate of the relative effectiveness of AS04-HPV-16/18v versus 4vHPV. Similarly, in other HPV vaccination studies in Asia, the cross-protection of AS04-HPV-16/18v against HPV-6 and HPV-11 has not been taken into account, and this may be important to note when interpreting the findings from these studies.(31,7477)

This study demonstrated that the two-dose AS04-HPV-16/18v would be cost-effective as part of a school-based programme for 12-year-old girls in Singapore from the perspective of the healthcare payer (MOH Singapore). AS04-HPV-16/18v was also shown to dominate 4vHPV by generating more QALYs at a lower cost. The vaccination is currently covered under the Medisave programme but relies on voluntary initiation or parental initiation, which may explain the current low uptake rate (13.6%).(18) Official recommendation from relevant medical societies, endorsement of the AS04-HPV-16/18v from MOH Singapore and the Health Promotion Board, and dissemination of these recommendations through public awareness campaigns could contribute to higher uptake rates. Coverage could be further increased by adding AS04-HPV-16/18v to the school-based vaccination programme for 12-year-old girls and by including AS04-HPV-16/18v in the National Immunisation Registry in Singapore, as this would deliver immunisation in an organised manner and provide reminders to parents if a child misses a vaccine dose.(78) Catch-up vaccination policies could also be considered in order to further increase the coverage in the overall population.(79) By building public awareness about HPV immunisation and improving the HPV vaccination coverage, the burden and suffering associated with cervical pre-cancer, cervical cancer and cancer death could be alleviated in the future.


ACKNOWLEDGEMENTS

The authors acknowledge Emily Lloyd (GSK, Singapore) for her contribution to this study; Gengshi Chen, Costello Medical Singapore, Singapore, on behalf of GSK, for writing and editorial assistance in preparing this manuscript for publication based on the authors’ input and direction. The authors would also like to thank Business and Decision Life Sciences platform for editorial assistance and manuscript coordination, on behalf of GSK. Nathalie Arts coordinated manuscript development and editorial support. GlaxoSmithKline Biologicals SA was the funding source and was involved in all stages of the study conduct and analysis. GlaxoSmithKline Biologicals SA also funded all costs associated with the development and publishing of the present manuscript. Cervarix is a trademark of the GSK group of companies. Gardasil is a trademark of Merck and Co Inc.

REFERENCES:
1. National Registry of Diseases Office (NRDO), Health Promotion Board,Singapore. Singapore Cancer Registry. Interim Annual Report. Trends in cancer incidence in Singapore 2010-2014. Available at: https://www.nrdo.gov.sg/docs/librariesprovider3/default-document-library/cancer-trends-2010-2014_interim-annual-report_final-(public).pdf?sfvrsn=0. Accessed January 21, 2016.
 
2. Low JJ, Ko Y, Ilancheran A, et al. Health and economic burden of HPV-related diseases in Singapore. Asian Pac J Cancer Prev 2012; 13:305-8.https://doi.org/10.7314/APJCP.2012.13.1.305
PMid:22502690
 
3. World Health Organization. Human papillomavirus (HPV) and cervical cancer. Fact sheet updated June 2016. Available at: http://www.who.int/mediacentre/factsheets/fs380/en/#. Accessed June 9, 2016.
 
4. Dochez C, Bogers JJ, Verhelst R, Rees H. HPV vaccines to prevent cervical cancer and genital warts: an update. Vaccine 2014; 32:1595-601.https://doi.org/10.1016/j.vaccine.2013.10.081
PMid:24606637
 
5. Walboomers JM, Jacobs MV, Manos MM, et al. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J Pathol 1999; 189:12-9.https://doi.org/10.1002/(SICI)1096-9896(199909)189:1<12::AID-PATH431>3.0.CO;2-F
 
6. Cogliano V, Baan R, Straif K, et al; WHO International Agency for Research on Cancer. Carcinogenicity of human papillomaviruses. Lancet Oncol 2005; 6:204.https://doi.org/10.1016/S1470-2045(05)70086-3
 
7. FUTURE II Study Group. Quadrivalent vaccine against human papillomavirus to prevent high-grade cervical lesions. N Engl J Med 2007; 356:1915-27.https://doi.org/10.1056/NEJMoa061741
PMid:17494925
 
8. Paavonen J, Naud P, Salmerón J, et al; HPV PATRICIA Study Group. Efficacy of human papillomavirus (HPV)-16/18 AS04-adjuvanted vaccine against cervical infection and precancer caused by oncogenic HPV types (PATRICIA): final analysis of a double-blind, randomised study in young women. Lancet 2009; 374:301-14.https://doi.org/10.1016/S0140-6736(09)61248-4
 
9. Health Promotion Board, Singapore. Management Guidelines for Abnormal Pap Smear & Preinvasive Disease of the Cervix. Available at: http://www.hpb.gov.sg/HOPPortal/content/conn/HOPUCM/path/Contribution%20…. Accessed January 8, 2016.
 
10. Jin AZ, Louange EC, Chow KY, Fock CW. Evaluation of the National Cervical Cancer Screening Programme in Singapore. Singapore Med J 2013; 54:96-101.https://doi.org/10.11622/smedj.2013032
PMid:23462834
 
11. Health Promotion Board, Singapore. Subsidies for Health Screening and Follow-up FAQs. Available at: http://www.hpb.gov.sg/HOPPortal/health-article/HPB054805. Accessed December 8, 2015.
 
12. Singapore Cancer Society. Public Education. Women’s Gynaecological Cancer Awareness Campaign. Available at: https://web.archive.org/web/20160102132353/http:/www.singaporecancersoci…. Accessed August 4, 2016.
 
13. Tay K, Tay SK, Tesalona KC, et al. Factors affecting the uptake of cervical cancer screening among nurses in Singapore. Int J Gynaecol Obstet 2015; 130:230-4.https://doi.org/10.1016/j.ijgo.2015.03.037
PMid:26032624
 
14. Tay SK, Tesalona KC, Rashid NM, Tai EY, Najib SM. Vaccine misconceptions and low HPV vaccination take-up rates in Singapore. Asian Pac J Cancer Prev 2014; 16:5119-24.https://doi.org/10.7314/APJCP.2015.16.12.5119
 
15. Yeoh KG, Chew L, Wang SC. Cancer screening in Singapore, with particular reference to breast, cervical and colorectal cancer screening. J Med Screen 2006; 13 Suppl 1:S14-9.
 
16. Health Sciences Authority, Singapore. PZ4970 Infosearch – register of therapeutic products. Available at: http://eservice.hsa.gov.sg/prism/common/enquirepublic/SearchDRBProduct.d…. Accessed January 11, 2016.
 
17. Ministry of Health, Singapore. Summary of Medisave Withdrawal Limits. Available at: https://www.moh.gov.sg/content/moh_web/home/costs_and_financing/schemes_…. Accessed June 9, 2016.
 
18. Chirayil EI, Thompson CL, Burney S. Predicting human papilloma virus vaccination and pap smear screening intentions among young Singaporean women using the theory of planned behavior. Sage Open 2014; 4:1-10.https://doi.org/10.1177/2158244014554961
 
19. Pitts M, Smith A, Croy S, et al. Singaporean women’s knowledge of human papillomavirus (HPV) and attitudes toward HPV vaccination. Women Health 2009; 49:334-51.https://doi.org/10.1080/03630240903158420
PMid:19753508
 
20. Di Mario S, Basevi V, Lopalco PL, et al. Are the two human papillomavirus vaccines really similar? A systematic review of available evidence: efficacy of the two vaccines against HPV. J Immunol Res 2015; 2015:435141.https://doi.org/10.1155/2015/435141
PMid:26380321 PMCid:PMC4562171
 
21. Howell-Jones R, Soldan K, Wetten S, et al. Declining genital warts in young women in England associated with HPV 16/18 vaccination: an ecological study. J Infect Dis 2013; 208:1397-403.https://doi.org/10.1093/infdis/jit361
PMid:24092908 PMCid:PMC3789575
 
22. Szarewski A, Skinner SR, Garland SM, et al. Efficacy of the HPV-16/18 AS04-adjuvanted vaccine against low-risk HPV types (PATRICIA randomized trial): an unexpected observation. J Infect Dis 2013; 208:1391-6.https://doi.org/10.1093/infdis/jit360
PMid:24092907 PMCid:PMC3789574
 
23. Public Health England. HPR volume 10 issue 22: news (8 July). 3. Continuing trend of declining genital warts diagnoses in young women in England: update to end 2015. Available at: https://www.gov.uk/government/publications/health-protection-report-volu…. Accessed August 4, 2016.
 
24. Wheeler CM, Castellsagué X, Garland SM, et al; HPV PATRICIA Study Group. Cross-protective efficacy of HPV-16/18 AS04-adjuvanted vaccine against cervical infection and precancer caused by non-vaccine oncogenic HPV types: 4-year end-of-study analysis of the randomised, double-blind PATRICIA trial. Lancet Oncol 2012; 13:100-10.https://doi.org/10.1016/S1470-2045(11)70287-X
 
25. Brisson M, Van de Velde N, De Wals P, Boily MC. The potential cost-effectiveness of prophylactic human papillomavirus vaccines in Canada. Vaccine 2007; 25:5399-408.https://doi.org/10.1016/j.vaccine.2007.04.086
PMid:17561316
 
26. Demarteau N, Standaert B. Modelling the economic value of cross- and sustained-protection in vaccines against cervical cancer. J Med Econ 2010; 13:324-38.https://doi.org/10.3111/13696998.2010.490481
PMid:20504110
 
27. Lee H, Kim BG, Hur S, et al. Cost-effectiveness analysis of AS04-adjuvanted human papillomavirus 16/18 vaccine compared with human papillomavirus 6/11/16/18 vaccine in adolescent girls in Korea, with the new 2-dose schedule. 9th Vaccines & ISV Congress, Seoul, Korea 2015.
 
28. Seto K, Marra F, Raymakers A, Marra CA. The cost effectiveness of human papillomavirus vaccines: a systematic review. Drugs 2012; 72:715-43.https://doi.org/10.2165/11599470-000000000-00000
PMid:22413761
 
29. Suárez E, Smith JS, Bosch FX, et al. Cost-effectiveness of vaccination against cervical cancer: a multi-regional analysis assessing the impact of vaccine characteristics and alternative vaccination scenarios. Vaccine 2008; 26 Suppl 5:F29-45.
 
30. Jit M, Demarteau N, Elbasha E, et al. Human papillomavirus vaccine introduction in low-income and middle-income countries: guidance on the use of cost-effectiveness models. BMC Med 2011; 9:54.https://doi.org/10.1186/1741-7015-9-54
PMid:21569406 PMCid:PMC3123559
 
31. Lee VJ, Tay SK, Teoh YL, Tok MY. Cost-effectiveness of different human papillomavirus vaccines in Singapore. BMC Public Health 2011; 11:203.https://doi.org/10.1186/1471-2458-11-203
PMid:21453537 PMCid:PMC3082243
 
32. Tay K, Tay SK. The impact of cytology screening and HPV vaccination on the burden of cervical cancer. Asia Pac J Clin Oncol 2011; 7:154-9.https://doi.org/10.1111/j.1743-7563.2011.01396.x
PMid:21585695
 
33. Ministry of Health, Singapore. Communicable Diseases Surveilance in Singapore 2013. Available at: https://www.moh.gov.sg/content/dam/moh_web/Publications/Reports/2014/Com…. Accessed January 21, 2016.
 
34. Starkie Camejo H, Li X, Van Kriekinge G. Does it matter? Discounting and its role in the cost-effectiveness of preventative interventions. The case of HPV vaccination. Public Health 2015; 129:989-92.https://doi.org/10.1016/j.puhe.2015.02.028
PMid:25834927
 
35. Goh SY, Tan SC, Lim LC, Chua B, Hunt B. Cost-effectiveness of switching from biphasic human insulin (BHI) to biphasic insulin aspart 30 (BIAsp-30) in type 2 diabetes patients with suboptimal glycaemic control in Singapore. J Diabetol 2015; 1:2.
 
36. Lin L, Teng M, Zhao Y, et al. Long-term cost-effectiveness of statin treatment for primary prevention of cardiovascular disease in the elderly. Cardiovasc Drugs Ther 2015; 29:187-97.https://doi.org/10.1007/s10557-015-6584-7
PMid:25860556
 
37. Ferko N, Postma M, Gallivan S, Kruzikas D, Drummond M. Evolution of the health economics of cervical cancer vaccination. Vaccine 2008; 26 Suppl 5:F3-15.
 
38. Department of Statistics Singapore. Complete Life Tables 2013-2014 for Singapore resident population. Available at: https://web.archive.org/web/20151113164324/http:/www.singstat.gov.sg/doc…. Accessed January 11, 2015.
 
39. Kumamoto Y, Tsukamoto T, Sugiyama T, et al. National surveillance of sexually transmitted diseases of Japan in 2002. Jpn Arch Sex Transm Dis 2004; 15:17-45.
 
40. Department of Statistics Singapore. Yearbook of Statistics Singapore 2015. Available at: http://www.singstat.gov.sg/docs/default-source/default-document-library/…. Accessed January 11, 2016.
 
41. Goldie SJ, Kohli M, Grima D, et al. Projected clinical benefits and cost-effectiveness of a human papillomavirus 16/18 vaccine. J Natl Cancer Inst 2004; 96:604-15.https://doi.org/10.1093/jnci/djh104
PMid:15100338
 
42. Melnikow J, Nuovo J, Willan AR, Chan BK, Howell LP. Natural history of cervical squamous intraepithelial lesions: a meta-analysis. Obstet Gynecol 1998; 92(4 Pt 2):727-35.
 
43. Moscicki AB, Hills N, Shiboski S, et al. Risks for incident human papillomavirus infection and low-grade squamous intraepithelial lesion development in young females. JAMA 2001; 285:2995-3002.https://doi.org/10.1001/jama.285.23.2995
PMid:11410098
 
44. Schlecht NF, Platt RW, Duarte-Franco E, et al. Human papillomavirus infection and time to progression and regression of cervical intraepithelial neoplasia. J Natl Cancer Inst 2003; 95:1336-43.https://doi.org/10.1093/jnci/djg037
PMid:12953088
 
45. Sanders GD, Taira AV. Cost effectiveness of a potential vaccine for human papillomavirus. Emerg Infect Dis 2003; 9:37-48.https://doi.org/10.3201/eid0901.020168
PMid:12533280 PMCid:PMC2873748
 
46. Van de Velde N, Brisson M, Boily MC. Modeling human papillomavirus vaccine effectiveness: quantifying the impact of parameter uncertainty. Am J Epidemiol 2007;165:762-75.https://doi.org/10.1093/aje/kwk059
PMid:17276976
 
47. International Agency for Research on Cancer. GLOBOCAN 2012: Estimated Cancer Incidence, Mortality and Prevalence Worldwide in 2012. The GLOBOCAN Project. Available at: http://globocan.iarc.fr/Default.aspx. Accessed January 8, 2016.
 
48. Richardson H, Kelsall G, Tellier P, et al. The natural history of type-specific human papillomavirus infections in female university students. Cancer Epidemiol Biomarkers Prev 2003; 12:485-90.
 
49. Woodhall SC, Jit M, Soldan K, et al; QOLIGEN study group. The impact of genital warts: loss of quality of life and cost of treatment in eight sexual health clinics in the UK. Sex Transm Infect 2011; 87:458-63.https://doi.org/10.1136/sextrans-2011-050073
PMid:21636616 PMCid:PMC3253069
 
50. Fahey MT, Irwig L, Macaskill P. Meta-analysis of Pap test accuracy. Am J Epidemiol 1995; 141:680-9.https://doi.org/10.1093/oxfordjournals.aje.a117485
PMid:7702044
 
51. Bergeron C, Cartier I, Guldner L, et al. [Lésions précancéreuses et cancers du col de l’utérus diagnostiqués par le frottis cervical, Ile-de-France, enquête Crisap, 2002]. Bull Epidemiol Hebd (Paris) 2005; 2:5-6. French.
 
52. Fender M, Schott J, Baldauf JJ, et al. [EVE, a regional campaign for the screening of cervical cancer. Organization, 7-years results and perspectives]. Presse Med 2003; 32:1545-51. French.
 
53. Brown DR, Kjaer SK, Sigurdsson K, et al. The impact of quadrivalent human papillomavirus (HPV; types 6, 11, 16, and 18) L1 virus-like particle vaccine on infection and disease due to oncogenic nonvaccine HPV types in generally HPV-naive women aged 16-26 years. J Infect Dis 2009; 199:926-35.https://doi.org/10.1086/597307
PMid:19236279
 
54. Skinner R, Apter D, Chow S, Wheeler C, Dubin G. Cross-protection efficacy of Cervarix(tm) against oncogenic HPV types beyond HPV-16/18. Abstract n° O-29.01. 25th International Papillomavirus Conference, Malmö, Sweden, May 8-14, 2009.
 
55. Tjalma W, Paavonen J, Naud P, et al. Efficacy of the HPV-16/18 AS04-adjuvanted vaccine against abnormal cytology and low-grade histopathological lesions in an oncogenic HPV-naïve population. Abstract no. A-171-0004-01446. 16th International Meeting of the European Society of Gynaecological Oncology (ESGO), 11-14 Oct, Belgrade, Serbia. Int J Gynecol Cancer 2009; 19(Suppl 2):1008.
 
56. FUTURE I/II Study Group, Dillner J, Kjaer SK, et al. Four year efficacy of prophylactic human papillomavirus quadrivalent vaccine against low grade cervical, vulvar, and vaginal intraepithelial neoplasia and anogenital warts: randomised controlled trial. BMJ 2010; 341:c3493.
 
57. Institut Català d’Oncologia (ICO). Information Centre on HPV and Cancer. Available at: http://www.hpvcentre.net/index.php. Accessed August 4, 2016.
 
58. Didierlaurent AM, Morel S, Lockman L, et al. AS04, an aluminum salt- and TLR4 agonist-based adjuvant system, induces a transient localized innate immune response leading to enhanced adaptive immunity. J Immunol 2009; 183:6186-97.https://doi.org/10.4049/jimmunol.0901474
PMid:19864596
 
59. Einstein MH, Baron M, Levin MJ, et al; HPV-010 Study Group. Comparative immunogenicity and safety of human papillomavirus (HPV)-16/18 vaccine and HPV-6/11/16/18 vaccine: follow-up from months 12-24 in a Phase III randomized study of healthy women aged 18-45 years. Hum Vaccin 2011; 7:1343-58.https://doi.org/10.4161/hv.7.12.18281
PMid:22048173 PMCid:PMC3338932
 
60. Giannini SL, Hanon E, Moris P, et al. Enhanced humoral and memory B cellular immunity using HPV16/18 L1 VLP vaccine formulated with the MPL/aluminium salt combination (AS04) compared to aluminium salt only. Vaccine 2006; 24:5937-49.https://doi.org/10.1016/j.vaccine.2006.06.005
PMid:16828940
 
61. Appendix 11: Human Papillomavirus. In: Stratton K, Durch J, Lawrence S, eds. Vaccines for the 21st Century: A Tool for Decisionmaking. Washington DC: National Academy Press, 2000: 213-21.
 
62. Gold MR, Franks P, McCoy KI, Fryback DG. Toward consistency in cost-utility analyses: using national measures to create condition-specific values. Med Care 1998; 36:778-92.https://doi.org/10.1097/00005650-199806000-00002
PMid:9630120
 
63. Insinga R, Glass A, Rush B, eds. Health state transitions following an abnormal pap smear: implications for health utility assessment in cost-effectiveness analyses. Abstract W-02. 22nd International Papillomavirus Conference & Clinical Workshop 2005.
 
64. Myers E, Green S, Lipkus I, eds. Patient preferences for health states related to HPV infection: visual analogue scales versus time trade-off elicitation. Proceedings of the 21st International Papillomavirus Conference, Mexico City; 2004.
 
65. World Health Organization. WHO guide for standardization of economic evaluations of immunization programmes. Available at: http://apps.who.int/iris/bitstream/10665/69981/1/WHO_IVB_08.14_eng.pdf. Accessed June 9, 2016.
 
66. National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. Available at: https://www.nice.org.uk/process/pmg9/resources/guide-to-the-methods-of-t…. Accessed January 11, 2016.
 
67. National Institute for Health and Care Excellence. Methods for the development of NICE public health guidance (third edition). Available at: https://www.nice.org.uk/process/pmg4/resources/methods-for-the-developme…. Accessed January 11, 2016.
 
68. National Registry of Diseases Office, Singapore. Singapore Cancer Registry. Interim Annual Registry Report. Trends in Cancer Incidence in Singapore 2009-2013. Available at: https://www.nrdo.gov.sg/docs/librariesprovider3/Publications-Cancer/tren…. Accessed June 9, 2016.
 
69. Department of Statistics Singapore. Annual GDP at current market prices 2015. Available at: http://www.singstat.gov.sg/statistics/browse-by-theme/national-accounts…. January 11, 2016.
 
70. Mullins CD, Onwudiwe NC, de Araújo GT, et al. Guidance document: global pharmacoeconomic model adaption strategies. Value Health Reg Issues 2014; 5:7-13.https://doi.org/10.1016/j.vhri.2014.02.006
PMid:29702790
71. Jit M, Mibei W. Discounting in the evaluation of the cost-effectiveness of a vaccination programme: a critical review. Vaccine 2015; 33:3788-94.https://doi.org/10.1016/j.vaccine.2015.06.084
PMid:26144897
 
72. Villa LL, Costa RL, Petta CA, et al. High sustained efficacy of a prophylactic quadrivalent human papillomavirus types 6/11/16/18 L1 virus-like particle vaccine through 5 years of follow-up. Br J Cancer 2006; 95:1459-66.https://doi.org/10.1038/sj.bjc.6603469
PMid:17117182 PMCid:PMC2360730
 
73. Woestenberg P, King A, Bogaards J, van der Sande M, van Benthem B. Women vaccinated against HPV-16/18 (bivalent vaccine) have less genital warts than unvaccinated women: a cross-sectional study among female STI-clinic visitors. 30th International Papillomavirus Conference & Clinical and Public Health Workshops 2015.
 
74. Levin CE, Sharma M, Olson Z, et al. An extended cost-effectiveness analysis of publicly financed HPV vaccination to prevent cervical cancer in China. Vaccine 2015; 33:2830-41.https://doi.org/10.1016/j.vaccine.2015.02.052
PMid:25770785
 
75. Yamamoto N, Mori R, Jacklin P, et al. Introducing HPV vaccine and scaling up screening procedures to prevent deaths from cervical cancer in Japan: a cost-effectiveness analysis. BJOG 2012; 119:177-86.https://doi.org/10.1111/j.1471-0528.2011.03036.x
PMid:21794070
 
76. Praditsitthikorn N, Teerawattananon Y, Tantivess S, et al. Economic evaluation of policy options for prevention and control of cervical cancer in Thailand. Pharmacoeconomics 2011; 29:781-806.https://doi.org/10.2165/11586560-000000000-00000
PMid:21838332
 
77. Ezat SW, Aljunid S. Comparative cost-effectiveness of HPV vaccines in the prevention of cervical cancer in Malaysia. Asian Pac J Cancer Prev 2010; 11:943-51.
PMid:21133606
 
78. Hopkins TG, Wood N. Female human papillomavirus (HPV) vaccination: global uptake and the impact of attitudes. Vaccine 2013; 31:1673-9.https://doi.org/10.1016/j.vaccine.2013.01.028
PMid:23375978
 
79. Rehn M, Uhnoo I, Kuhlmann-Berenzon S, et al. Highest vaccine uptake after school-based delivery – a county-level evaluation of the implementation strategies for HPV catch-up vaccination in Sweden. PLoS One 2016; 11:e0149857.https://doi.org/10.1371/journal.pone.0149857
PMid:26974977 PMCid:PMC4790890