author: Duncan T Odom
date: 2013
institute: 1University of Cambridge, Cancer Research UK - Cambridge Institute, Cambridge, CB2 0RE, UK
title: How to evaluate a graduate studentship, or choosing the right doctoral advisor

A potential graduate student may spend more time in deciding what trendy digital device to buy than the environment and conditions of a studentship that guide their future career - I know this is true, because I got a really good Walkman, back in the day. This brief essay summarizes the core advice that I give folks who are about to take the first steps on the (potentially perilous) path towards an academic or industrial research career in biological sciences.

# The golden rules

This article is written for someone who is passionate about science, and who knows that having a doctorate is the first magical step toward a principal investigator position in a research institute or a university. You have made all the right moves so far in your undergraduate studies; so you have (a) solid to excellent grades, and (b) done substantial amounts of research with a faculty member who can vouch for your competence (this means by the way, ideally more than a year). Now, famous people in storied places as diverse as MIT, Oxford and Singapore have invited you to interview for PhD studentships, which means you have been favorably evaluated by these programs. Congratulations!

But...how do *you* evaluate *them*: that is, the group leaders whose studentships are offered to you? In other words, how do you maximize the chances of surviving the next five (all too often brutal) years of your professional life?

My advice is based on my own experiences as a graduate student in chemistry (at Caltech, 5 years), a postdoctoral fellow in genomics (at the Whitehead and MIT, 6 years), and a faculty member in comparative and functional genomics (at University of Cambridge, 6 years). I currently run a medium-sized research group with a steady state of two to four postdocs, two to three graduate students, and about four staff scientists of various flavors. Here, I emphasize points relevant to biological sciences, but some of the ideas are applicable more widely.

To begin, let's ask a quick, basic, rarely-asked question. What do you need to accomplish as a graduate student in order to get a great postdoctoral position, from which you could launch a high-caliber faculty job? Although many accomplishments might be desirable, the most basic is this:

You must obtain your doctoral degree in a reasonable amount of time (think: ideally less than five years) with (1) an intact, enthusiastic spirit of curiosity, unbroken by terrible experiences, and (2) at least one solid, first-author publication that has been peer-reviewed.

This clear, unambiguous goal for survival should really help guide your choice of graduate advisor with open and critical eyes. Working in a Nobel-winning laboratory and producing publications in the highest impact journals might be ideal, but might not be a realistic (or even desirable) strategy for surviving your first years as a young scientist. Since the biggest variable you can control for is the scientific identity and character of your PhD advisor, you should evaluate this *before* you join a laboratory.

# Established laboratories

The easiest situation to evaluate is the established laboratory, where you should start by asking hard-nosed questions about their prior record, both in general, and specifically with graduate students' training.

## History with students

How many people have accepted PhD studentships in the past?

Did they all publish first-author papers somewhere?

If not, who didn't publish, and what fraction of accepted students does that represent?

How many students have left the group, successfully and otherwise?

Where did they go?

Why?

All of these questions assume that your experience will mirror that of a typical person in that lab.

## The lab's scientific leadership

How is the advisor as a leader and scientific role model?

An interview is your chance to ask pointed questions, probably in private, with as many folks as possible. Consider soliciting gossip, either from your social network or from other labs that neighbor your prospective host lab. This advice may be particularly uncomfortable to follow, since we all want to assume the best of everyone.

But think about it this way: if you were about to adventure-drive across the Sahara desert, would you just buy any random vehicle on the word of the owner, or would you have an independent car mechanic in to evaluate its mechanical condition, and thus to maximize your survival? The first four to five years as a scientist in large part shapes one's future identity irrevocably. Again, the single biggest influence will be your advisor, so choose wisely and be as informed as possible.

To state the obvious, certain character traits are undesirable in a mentor (well, at least when present in excess). These include, but are not limited to: conflict avoidance, being a bully, having an unstable temper, overt narcissism, passive-aggressive patterns, un-professionalism (such as outright intoxication at an institute beer hour or multiple paper retractions), crippling insecurity, excessive perfectionism (never submitting papers, for example), a damaging history of playing favorites.

Use your common sense, and always remember that you are trying to avoid only serious, Titanic-sinking phenotypes. You should not be demanding perfection - as everyone is human and we all have limitations and make mistakes - just look for survivability.

How much does your candidate lab's group leader travel?

Ask the laboratory if the group leader's travel to conferences and talks interferes with the lab's smooth function and publication rate. When my travel gets heavy, papers and grants turn around more slowly - and so my lab suffers. During trips, work time is more fragmented and less gets done, even when I set aside time. And, even in this wired world, I am simply much more effective in my own office, where my folks can just drop in.

## Lab dynamics

How big is the prospective lab?

Having now run a group for a few years, I know from hard experience that I can only manage about five to six researchers, who are themselves actively running research projects. My realization of this limitation has meant that my lab naturally equilibrates at about ten people, counting the four staff scientists who do not normally run larger projects. If my lab got bigger than it currently is, then I would be forced to neglect people's hard-fought discoveries. My caveat here is that some faculty are much better at managing folks than I am, which leads to the next point.

Are the lab's publications spread (somewhat) evenly across all the lab members, or do publications concentrate for excessively long stretches with one or two people as first authors?

For your own sake, you should be ruthless in your evaluations, but, to be fair, do consider lab size carefully. If someone has five graduate students, and they publish three solid papers a year with different lead authors, that sounds like a potentially promising environment (averaging one paper per person, every year and a half or so). Instead, though, what if there are a total of fifteen postdocs and grad students in a single lab, and two solid papers a year get published? That situation would mean one paper published per person, every seven and a half years or so. Not quite so good.

# Fresh-faced group leaders

With a newly minted junior group leader, you probably have less actual history to go on. However, you can certainly ask them questions about what kind of lab they *intend* to build.

How many people do they want/intend to have in the lab in five years? In what roles? Why?

Most senior postdocs do not think strongly about these issues, as most of their efforts are spent in trying to get a faculty position. However, the starting group leaders who have thought these issues through would be expected to be more prepared to train promising graduate students like you. There are powerful advantages of starting work for a very junior group, not least that your interests and theirs are exactly aligned in ways that senior group leaders cannot be. In particular, untenured group leaders will be wanting your first stories published at least as much as you do, possibly more.

An aside: I believe that the first four or five people who join a nascent laboratory have the best chances of hitting a scientific gold mine. It does, however, require courage to take a chance on an unknown group leader.

# Medics

With a medically trained group leader, you have a huge advantage in easier access to patient materials, which facilitates looking at questions directly relating to human disease biology. But there are certain drawbacks.

What is the allocation of the clinician-scientist's time between clinic and lab?

Many clinician scientists stay engaged with patient care, which is often an excellent strategy, but the cost of this dedication can at times be how much attention they can give their laboratory. Furthermore, many areas of translational medicine are scientifically over-subscribed and extraordinarily competitive. Remember that as long as you, as a grad student, get your solid publication or two, you are fine. However, if the clinician-scientist group leader is aiming for a big splash in a top-tier journal, then lower-impact stories that make up the bedrock of a successful graduate studentship might get neglected along the way.

# And finally

Good luck!


abstract: **Objectives** To investigate the association between opium use and subsequent risk of death.
  .
  **Design** Prospective cohort study.
  .
  **Setting** The Golestan Cohort Study in north-eastern Iran collected detailed validated data on opium use and other exposures at baseline. Participants were enrolled between January 2004 and June 2008 and were followed to May 2011, with a follow-up success rate of over 99%.
  .
  **Participants** 50 045 participants aged 40-75 at baseline.
  .
  **Main outcomes** Mortality, all cause and major subcategories.
  .
  **Results** 17% (n=8487) of the participants reported opium use, with a mean duration of 12.7 years. During the follow-up period 2145 deaths were reported. The adjusted hazard ratio for all cause mortality associated with ever use of opium was 1.86 (95% confidence interval 1.68 to 2.06). Opium consumption was significantly associated with increased risks of deaths from several causes including circulatory diseases (hazard ratio 1.81) and cancer (1.61). The strongest associations were seen with deaths from asthma, tuberculosis, and chronic obstructive pulmonary disease (11.0, 6.22, and 5.44, respectively). After exclusion of people who self prescribed opium after the onset of major chronic illnesses, the associations remained strong with a dose-response relation.
  .
  **Conclusion** Opium users have an increased risk of death from multiple causes compared with non-users. Increased risks were also seen in people who used low amounts of opium for a long period and those who had no major illness before use.
author: Hooman Khademi; Reza Malekzadeh; Akram Pourshams; Elham Jafari; Rasool Salahi; Shahryar Semnani; Behrooz Abaie; Farhad Islami; Siavosh Nasseri-Moghaddam; Arash Etemadi; Graham Byrnes; Christian C Abnet; Sanford M Dawsey; Nicholas E Day; Paul D Pharoah; Paolo Boffetta; Paul Brennan; Farin KamangarCorrespondence to: R Malekzadeh email@example.com, P Brennan <firstname.lastname@example.com>
date: 2012
references:
title: Opium use and mortality in Golestan Cohort Study: prospective cohort study of 50 000 adults in Iran

# Extra material supplied by the author

Appendix: Supplementary tables A-E

Click here for additional data file.

# Introduction

Use of opium (lachryma papaveris) for medicinal or recreational purposes has a long history in many parts of the world1 and continues to be common. In 2008 an estimated 13-22 million people worldwide used opium or its derivatives as illicit drugs: 6.5-12.5 million in Asia; 3.3-3.8 million in Europe; 2.3-2.4 million in the Americas; and the rest in Africa and Oceania.2

A small number of retrospective case-control studies or case series have described the possible role of opium in oesophageal cancer,3 4 bladder cancer,5 6 7 coronary heart disease,8 9 10 11 and a few other select health conditions.12 13 14 There is little information from well designed prospective epidemiological studies on the effect of the use of opium on overall or cause specific mortality, partly because of relatively low opium use in most established cohort studies and the difficulty of obtaining validated data. There is also scant information on the effect of different types of opium with respect to health outcomes.

The Golestan Cohort Study (GCS), a prospective study established in north eastern Iran,15 where opium consumption is exceptionally common,16 has collected detailed validated data on opium use from over 50 000 people.17 This study has collected data on typical amounts used, duration of use, and the four major types of opiates used locally—that is, teriak, shireh, sukhteh, and heroin. The first two can be smoked or ingested. Sukhteh can only be ingested, and heroin can be injected, sniffed, or smoked.4 Using data on over 2000 deaths identified in the initial four years of follow-up, we report the effects of opium use, including by subtypes,4 18 on overall and cause specific mortality.

# Methods

## Study population

The Golestan Cohort Study was launched in January 2004 in north eastern Iran after completion of a pilot study in 2003.16 In brief, a total of 50 045 people aged 40-75 were recruited from January 2004 to June 2008. The cohort study was designed to investigate the causes of oesophageal squamous cell carcinoma in Golestan province, where the risk of this cancer is extremely high. Study participants were enrolled from those who lived in Gonbad City and 326 villages in Golestan province, north eastern Iran. Further details have been discussed elsewhere.15

## Assessment of exposure

Trained interviewers used a general questionnaire and a food frequency questionnaire to collect data. The general questionnaire included detailed queries on opium use (see below), tobacco use, and alcohol drinking and questions on demographic characteristics, residential history, occupation, education, and medical history. Height, weight, and waist and hip circumferences were also measured. Blood pressure for each participant was measured twice from both arms, with 10 minute intervals, while the person was seated and relaxed. Nutritional data were collected with a food frequency questionnaire that was specifically designed and validated for this area.19 Biospecimens included 10 ml of blood, 4.5 ml of midstream urine, and hair and nail samples from each individual.

Detailed data on opium use were collected as opium use is common in Golestan and could be a risk factor for oesophageal cancer. Data were obtained on age of start and end of opium use, typical amount of use in nokhod (the local unit for opium use, equivalent to about 0.2 g), frequency of use (how many days a week if weekly or more), and routes of administration for the different types of opiates commonly used—namely, teriak, sukhteh, shireh, and heroin. If participants used multiple types of opium, or if they used opium intermittently, data were recorded separately for each type and period of use.

Teriak, or raw opium, is directly obtained from the seed capsule of the opium poppy, *Papaver somniferum*.20 The ripening poppy capsules are incised to extract a milky juice, which turns into a dark sticky or crumbly mass, known locally as teriak. Teriak can be taken orally or smoked after direct heating with burning charcoal in specialised devices such as an opium pipe. Sukhteh is the opium dross remaining after the opium is smoked and is taken orally. This form of opium, which is generally cheaper than teriak, has been shown to have high mutagenic activity.18 Shireh is a refined product of opium that is mainly obtained from sukhteh, with or without adding teriak, boiled in water and filtered several times to remove the insoluble materials.18 Shireh can be ingested or smoked, by indirect heating, in a special type of opium pipe. As indirect heating is used, the vapour of opiate is inhaled, not its smoke. In this region users who smoke opium generally do not mix it with any other substances such as tobacco.

## Ascertainment of cause of death

As nearly all study participants have telephones, they are actively followed by annual telephone calls. Detailed questions are asked regarding their health status and any admission to hospital or outpatient procedures. All data are recorded on hard copies as well as on a computerised database. If a study participant is inaccessible through telephone calls, after seven attempts during a two week period, researchers contacted friends or local health workers. Using this method, we successfully obtained follow-up information for over 99% of all participants.15 21 On learning of a reported death through the phone calls or through the local health workers or monthly provincial reports of death registration, a general practitioner who is a member of the follow-up team visits the home of the dead person and completes a validated verbal autopsy questionnaire21 by interviewing the closest relative of the dead person. At the same time other members of the team gather all available and relevant medical documents including medical charts, radiographs, pathology reports, hospital discharge documents, etc, from all hospitals or pathology centres, either within the province or neighbouring provinces.

For 65% of all deaths, in addition to the verbal autopsy, extensive medical documents—including physician notes, test results, findings on electrocardiography and radiology, endoscopy results, and pathology reports—were retrieved to determine cause of death.21 For the remaining deaths, cause was determined only by verbal autopsy. Our previous studies show high accuracy (all measures of accuracy above 81%) and reliability (κ \>0.75), particularly for major groups of cause of death, for this verbal autopsy.21 Two separate internists independently reviewed all collected documents, including the verbal autopsy information and medical charts, and the cause of death was documented with ICD-10 codes (international classification of diseases, 10th revision). If the results were concordant, a diagnosis was made. In case of discordance a third, more experienced internist reviewed all documents and the two initial diagnoses and made the final diagnosis. If a final diagnosis could not be made for any reason, the cause of death was coded as "unknown."

Though cause of death should ideally be assessed by someone blinded to the exposure data, the physician who completed the verbal autopsy by interviewing the next of kin of the dead person also collected some information on medical history and habits (such as smoking and alcohol and opium use). Nevertheless, total deaths, the main outcome of the study, could not be biased by knowledge of the exposure. Furthermore, as the internists who determined the cause of death had no prior belief regarding the association between opium use and any specific cause of death, except perhaps for oesophageal cancer, we believe such information should have minimal impact on classification of cause of death.

The major categories of death evaluated in this analysis were the most prevalent causes of death among the participants, including circulatory, cancer, respiratory, digestive, infectious, external (mainly from motor vehicle crashes or other unintentional injury and suicide), unknown, or other.

## Statistical analysis

After examining the proportional hazard assumption using Schoenfeld residuals, we fitted a Cox proportional hazards model, with age as the timescale, to estimate unadjusted and adjusted hazard ratios and 95% confidence intervals for total and cause specific mortality in relation to the exposures of interest, including any opium use, type of opium used, duration of use, cumulative use, and the method of opium consumption. Follow-up continued until loss to follow-up, death, or 20 May 2011, whichever came first. While opium use is a potentially important factor in all cause mortality, age is still the most important risk factor for death and is biologically more important than the time of recruitment into the cohort. Hence we chose age as the Cox regression time variable, implying that all risk comparisons are made between individuals of the same age. As deaths before recruitment were not observed, participants were left censored at the age of enrolment. We also used accumulated exposure at the time of enrolment as a fixed variable, rather than allowing it to increment with time in the study as might seem more natural. This was done because we had no data on continued exposure after enrolment; we do not know how the risk from accumulated exposure changes after cessation of consumption; and the time within the study is typically short compared with the accumulated years of exposure. We adjusted for potential confounders—including sex, place of residence (urban or rural), marital status, highest educational level, ethnicity (Turkmen or others), and cigarette smoking—by including indicator variables in the Cox regression models. Further inclusion of alcohol consumption, body mass index (BMI, either categorical or continuous), height (as a measure of health during early adulthood), and consumption of fruit and vegetables did not materially change the hazard ratios. As adjustment for a composite wealth score (including information on ownership of car or motorbike, black and white TV, colour TV, refrigerator, freezer, vacuum cleaner, and washing machine, as well as having bath in the home) that has been previously used to estimate socioeconomic status in Golestan Province22 did not change the results, we have presented the main results without adjustment for these variables. Interactions were examined by including appropriate interaction terms in the Cox regression models.

For the purpose of this analysis, we defined opium users as those who consumed opium at least once a week for at least six months. Opium users were further classified as former users and current users. Opium types analysed in this study were teriak, sukhteh, shireh, and heroin. Among current users, duration of use, which was calculated by summing the years using any type of opium, was categorised into fifths. We then calculated cumulative use by multiplying duration of use by amount used. If study participants used opium intermittently during several periods of their life, we obtained overall cumulative use by summing cumulative use during each period of use. The methods of opium consumption were classified into three categories: smoking, ingestion, or both. As the number of heroin users was small, we did not consider injection as one of the categories of use.

People of this area may use opium to alleviate pain from chronic illnesses, which could lead to associations from reverse causality. To assess this possibility, we used data collected at enrolment to exclude from analysis any participants who started opium use after experiencing ischaemic heart disease, cerebrovascular event, hypertension, or diabetes mellitus. To further study reverse causality, we also conducted a sensitivity analysis by excluding deaths that occurred in the first 6, 12, 18, and 24 months of follow-up.

We calculated both age and age-sex standardised mortality rates based on World Population 2000. We also calculated the fraction of deaths attributed to opium use in this population using the hazard ratios and prevalence of opium use. All statistical analyses were conducted with Stata statistical software, version 11 (StataCorp, College Station, TX).

# Results

Data from all 50 045 cohort participants were used for this study, with a total of 234 928 person years of follow-up (median 4.7 years per participant). The mean age of participants at baseline was 52.1 (SD 8.9) years. Most participants were women (58%), 74% were Turkmen, 80% lived in rural areas, 88% were married, 83% were non-smokers, and 70% had no formal education.

Of the 17% (n=8487) of cohort members who reported ever using opium, the mean age at enrolment was 53.4 (SD 9.2), the mean age at which they started using opium was 39.7 (SD 9.2), and the mean reported duration of use was 12.7 (SD 11.1) years. The median daily amount of opium used was 0.6 g (25th-75th centile 0.2-1.2g). Opium users were more likely than the rest of the cohort to be men, rural dwellers, and cigarette smokers. The proportion of Turkmen people, married individuals, and those with no formal education was similar to the entire cohort.

During the follow-up period until 20 May 2011, 293 (0.6%) of the study participants, including 226 (0.5%) of those who had never used opium and 67 (0.8%) of the opium users, were lost to follow-up, and 2145 deaths were reported, resulting in a crude death rate of 913 per 100 000 person years. As shown in table 1, an overall increased risk of death was observed for being male, being illiterate, not being married (single, widowed, divorced), and medical history of myocardial infarction, cerebrovascular events, hypertension, and diabetes, whereas ethnicity and place of residence were not associated with an increased risk.

 Demographic and anthropometric characteristics, medical history, and habits in relation to death in participants of Golestan Cohort Study

| Variables | No of participants (person years) | No of deaths (crude rate\*) | SMR† | Relative rate |
|----|----|----|----|----|
| Sex: |  |  |  |  |
|  Female | 28 811 (136 279) | 893 (655) | 727 | Reference |
|  Male | 21 234 (98 649) | 1252 (1269) | 1207 | 1.66 |
| Ethnicity: |  |  |  |  |
|  Non-Turkmen | 12 792 (57 038) | 520 (912) | 972 | Reference |
|  Turkmen | 37 253 (177 890) | 1625 (914) | 959 | 0.99 |
| Education (highest level): |  |  |  |  |
|  No formal | 35 118 (163 647) | 1708 (1044) | 1050 | Reference |
|  Up to eight years | 10 708 (49 753) | 316 (635) | 704 | 0.67 |
|  High school | 3155 (15 883) | 89 (560) | 460 | 0.44 |
|  University | 1064 (5644) | 32 (567) | 505 | 0.48 |
| Marital status: |  |  |  |  |
|  Married | 43 890 (205 921) | 1732 (841) | 924 | Reference |
|  Single | 302 (1449) | 22 (1519) | 1738 | 1.88 |
|  Widow/er | 5603 (26 422) | 376 (1423) | 1703 | 1.84 |
|  Divorced/other | 135 (635) | 10 (1575) | 1700 | 1.84 |
| Place of residence: |  |  |  |  |
|  Rural | 40 012 (179 745) | 1625 (904) | 979 | Reference |
|  Urban | 10 033 (55 183) | 520 (942) | 942 | 0.96 |
| History of ischaemic heart disease: |  |  |  |  |
|  No | 46 994 (221 407) | 1797 (812) | 886 | Reference |
|  Yes | 3051 (13 521) | 348 (2574) | 2032 | 2.29 |
| History of cerebrovascular event: |  |  |  |  |
|  No | 49 616 (233 040) | 2078 (892) | 948 | Reference |
|  Yes | 429 (1,888) | 67 (3548) | 2646 | 2.79 |
| Hypertension: |  |  |  |  |
|  No | 40 170 (189 871) | 1388 (731) | 827 | Reference |
|  Yes | 9875 (45 057) | 757 (1680) | 1627 | 1.97 |
| History of diabetes mellitus: |  |  |  |  |
|  No | 46 591 (219 541) | 1836 (836) | 897 | Reference |
|  Yes | 3454 (15 387) | 309 (2008) | 1997 | 2.23 |
| Cigarette smoking: |  |  |  |  |
|  Never smoker | 41 385 (194 823) | 1537 (789) | 878 | Reference |
|  Ever smoker | 8660 (40 105) | 608 (1516) | 1429 | 1.63 |
| Opium use: |  |  |  |  |
|  Never user | 41 558 (196 825) | 1440 (732) | 818 | Reference |
|  Ever user | 8487 (38 103) | 705 (1850) | 1650 | 2.02 |

\*Crude rates calculated in 10^5^ person years.

†Age and sex standardised mortality rate based on 2000 world standard population.

In all, 8660 (17%) participants reported ever smoking cigarettes, with an adjusted hazard ratio of 1.53 (95% confidence interval of 1.37 to 1.70) for all causes of death. The hazard ratio for death associated with opium use was 1.98 (1.80 to 2.17), before adjustment for cigarette smoking and 1.86 (1.68 to 2.06) after adjustment for cigarette smoking.

Table 2 compares hazard ratios for overall mortality among opium users stratified by sex, ethnicity, education, marital status, place of residence, history of chronic diseases, BMI, and categories of cigarette smoking, with tests for interaction across each group. With the exception of people with history of cerebrovascular events, a relatively small group, opium use was associated with an increased risk of mortality among all strata. Significantly higher increased risks of mortality were observed among women than men (P\<0.001), urban than rural dwellers (P=0.008), obese than non-obese people (P=0.020), and those who had never smoked than those who did (P=0.001).

 Hazard ratios for association between opium use and mortality by subgroups in participants in Golestan Cohort Study

| Variable | No (%) who never used opium | No (%) who ever used opium | Cox hazard ratio\* (95% CI) |  | P for interaction |
|----|----|----|----|----|----|
|  |  |  | Unadjusted | Adjusted† |  |
| All cohort | 41 558 | 8487 | 2.26 (2.06 to 2.47) | 1.86 (1.68 to 2.06) | — |
| Sex: |  |  |  |  |  |
|  Women | 26 456 (63.7) | 2355 (27.7) | 2.54 (2.16 to 2.98) | 2.43 (2.05 to 2.88) | \<0.001 |
|  Men | 15 102 (36.3) | 6132 (72.3) | 1.82 (1.62 to 2.03) | 1.63 (1.44 to 1.84) |  |
| Ethnicity: |  |  |  |  |  |
|  Turkmen | 30 701 (73.9) | 6552 (77.2) | 2.24 (2.02 to 2.48) | 1.82 (1.62 to 2.05) | 0.578 |
|  Non-Turkmen | 10 857 (26.1) | 1935 (22.8) | 2.28 (1.89 to 2.76) | 1.99 (1.62 to 2.45) |  |
| Education: |  |  |  |  |  |
|  Illiterate | 29 490 (71.0) | 5628 (66.3) | 2.24 (2.02 to 2.48) | 1.89 (1.68 to 2.11) | 0.558 |
|  Up to 8 years | 8480 (20.4) | 2228 (26.3) | 2.23 (1.77 to 2.82) | 1.76 (1.35 to 2.29) |  |
|  High school | 2601 (6.30) | 554 (6.50) | 2.68 (1.71 to 4.18) | 1.61 (0.98 to 2.67) |  |
|  University | 987 (2.30) | 77 (0.90) | 3.90 (1.55 to 9.81) | 3.65 (1.28 to 10.4) |  |
| Marital status: |  |  |  |  |  |
|  Married | 36 345 (87.7) | 7545 (89.0) | 2.22 (2.00 to 2.45) | 1.80 (1.61 to 2.02) | 0.629 |
|  Single | 240 (0.60) | 62 (0.70) | 3.22 (1.32 to 7.81) | 2.36 (0.81 to 6.90) |  |
|  Widow/er | 4760 (11.5) | 843 (9.90) | 2.40 (1.92 to 2.99) | 2.16 (1.69 to 2.75) |  |
|  Divorced/other | 108 (0.20) | 27 (0.40) | 2.22 (0.60 to 8.26) | 2.10 (0.41 to 10.9) |  |
| Place of residence: |  |  |  |  |  |
|  Rural | 32 527 (78.3) | 7485 (88.2) | 2.12 (1.91 to 2.35) | 1.77 (1.58 to 1.99) | 0.008 |
|  Urban | 9031 (21.7) | 1002 (11.8) | 2.97 (2.43 to 3.61) | 2.32 (1.85 to 2.92) |  |
| History of ischaemic heart disease: |  |  |  |  |  |
|  No | 39 210 (94.3) | 7784 (91.7) | 2.30 (2.09 to 2.54) | 1.87 (1.67 to 2.09) | 0.053 |
|  Yes | 2348 (5.70) | 703 (8.30) | 1.82 (1.46 to 2.28) | 1.58 (1.23 to 2.04) |  |
| History of cerebrovascular event: |  |  |  |  |  |
|  No | 41 237 (99.2) | 8379 (98.7) | 2.28 (2.08 to 2.50) | 1.88 (1.69 to 2.08) | 0.164 |
|  Yes | 321 (0.80) | 108 (1.30) | 1.34 (0.78 to 2.33) | 1.12 (0.59 to 2.13) |  |
| Hypertension: |  |  |  |  |  |
|  No | 33 148 (79.8) | 7022 (82.7) | 2.30 (2.06 to 2.57) | 1.81 (1.59 to 2.05) | 0.685 |
|  Yes | 8410 (20.2) | 1465 (17.3) | 2.39 (2.04 to 2.79) | 1.94 (1.63 to 2.32) |  |
| History of diabetes mellitus: |  |  |  |  |  |
|  No | 38 647 (93.0) | 7944 (93.6) | 2.33 (2.12 to 2.57) | 1.87 (1.67 to 2.08) | 0.320 |
|  Yes | 2911 (7.00) | 543 (6.40) | 2.00 (1.55 to 2.57) | 1.67 (1.26 to 2.22) |  |
| Body mass index (BMI): |  |  |  |  |  |
|  Underweight | 1328 (3.20) | 1080 (12.7) | 1.76 (1.32 to 2.35) | 1.54 (1.12 to 2.10) | 0.020 |
|  Normal weight | 13 585 (32.7) | 4344 (51.2) | 1.99 (1.74 to 2.28) | 1.77 (1.53 to 2.06) |  |
|  Overweight | 14 884 (35.8) | 2088 (24.6) | 2.30 (1.92 to 2.75) | 1.83 (1.50 to 2.23) |  |
|  Obese | 11 753 (28.3) | 971 (11.5) | 2.93 (2.30 to 3.72) | 2.25 (1.72 to 2.93) |  |
| Cigarette smoking: |  |  |  |  |  |
|  Never smoked | 37 373 (89.9) | 4012 (47.3) | 2.25 (1.99 to 2.53) | 2.07 (1.83 to 2.35) | 0.001 |
|  Ever smoked | 4185 (10.1) | 4475 (52.7) | 1.53 (1.30 to 1.81) | 1.50 (1.27 to 1.77) |  |

\*Hazard ratio of dying in opium users in each stratum.

‡Adjusted for variables including sex, ethnicity, education level, marital status, residential place, and cigarette smoking, except if each of them is considered as stratifying variable.

Table 3 shows the hazard ratios (95% confidence intervals) for major causes of death in relation to opium use. Opium use was associated with an increased risk of death in all major categories, except the category classified as "external," which mainly included unintentional injuries and trauma. The strongest associations were found for infections (adjusted hazard ratio 5.47), respiratory diseases (3.78), and digestive diseases (3.12). Consistent with the overall association, the hazard ratios were higher in women than in men for most causes of deaths. Among deaths from circulatory causes, opium use was associated with an increased risk of ischaemic heart disease and cerebrovascular events (table 3). Among deaths from cancer, opium use was associated with an increased risk of three common cancers that have been shown to be related to tobacco smoking—that is, oesophageal, gastric, and lung cancers—though this was significant only for lung cancer. Additional examination of the association between opium use and deaths from respiratory conditions showed that the adjusted hazard ratios were 11.0 (95% confidence interval 3.97 to 30.6) for asthma, 6.22 (2.36 to 16.4) for tuberculosis, and 5.44 (2.03 to 14.5) for chronic obstructive pulmonary disease.

 Hazard ratios for major causes of death in relation to opium use in Golestan Cohort Study

| Cause of death | Total (n=50 045) | No (%) of opium users (n=8487, 17%) | Cox hazard ratio\* (95% CI) |  |  |
|----|----|----|----|----|----|
|  |  |  | Total | Men | Women |
| Any | 2145 | 705 (33) | 1.86 (1.68 to 2.06) | 1.63 (1.44 to 1.84) | 2.43 (2.05 to 2.88) |
| Circulatory: |  |  |  |  |  |
|  All | 1073 | 335 (31) | 1.81 (1.56 to 2.09) | 1.53 (1.28 to 1.82) | 2.52 (1.98 to 3.20) |
|  Ischaemic heart disease | 624 | 208 (33) | 1.90 (1.57 to 2.29) | 1.58 (1.26 to 1.98) | 2.90 (2.10 to 3.99) |
|  Cerebrovascular event | 352 | 98 (28) | 1.68 (1.29 to 2.18) | 1.49 (1.07 to 2.08) | 1.97 (1.30 to 2.97) |
|  Other circulatory | 97 | 29 (30) | 1.73 (1.06 to 2.82) | 1.30 (0.71 to 2.36) | 2.98 (1.35 to 6.58) |
| Cancer: |  |  |  |  |  |
|  All | 449 | 137 (31) | 1.61 (1.28 to 2.03) | 1.70 (1.30 to 2.24) | 1.42 (0.92 to 2.20) |
|  Oesophagus | 100 | 30 (30) | 1.51 (0.93 to 2.43) | 1.12 (0.62 to 2.04) | 2.40 (1.13 to 5.10) |
|  Stomach | 75 | 25 (33) | 1.32 (0.77 to 2.26) | 1.33 (0.74 to 2.40) | 1.23 (0.33 to 4.65) |
|  Lung | 37 | 18 (49) | 2.27 (1.07 to 4.80) | 2.65 (1.13 to 6.21) | 1.19 (0.15 to 9.72) |
|  Other cancer | 237 | 64 (27) | 1.67 (1.21 to 2.31) | 2.09 (1.40 to 3.13) | 1.10 (0.59 to 2.07) |
| Respiratory: |  |  |  |  |  |
|  All | 95 | 52 (55) | 3.78 (2.36 to 6.04) | 2.64 (1.50 to 4.73) | 6.64 (3.30 to 13.3) |
|  Asthma | 23 | 17 (74) | 11.0 (3.97 to 30.6) | 21.5 (2.60 to 177) | 8.04 (2.26 to 28.6) |
|  COPD | 24 | 17 (71) | 5.44 (2.03 to 14.5) | 3.18 (1.11 to 9.14) | 34.1 (3.29 to 353) |
|  Other respiratory | 48 | 18 (38) | 1.87 (0.94 to 3.71) | 1.15 (0.49 to 2.70) | 4.08 (1.51 to 11.0) |
| Digestive: |  |  |  |  |  |
|  All | 69 | 32 (46) | 3.12 (1.82 to 5.37) | 3.05 (1.60 to 5.79) | 3.32 (1.24 to 8.84) |
|  Liver cirrhosis | 36 | 14 (39) | 2.28 (1.05 to 4.97) | 2.90 (1.14 to 7.34) | 1.17 (0.21 to 6.40) |
|  Other digestive | 33 | 18 (55) | 4.35 (2.01 to 9.39) | 3.25 (1.34 to 7.90) | 8.89 (2.20 to 35.9) |
| Infectious: |  |  |  |  |  |
|  All | 51 | 30 (59) | 5.47 (2.87 to 10.4) | 3.44 (1.54 to 7.69) | 8.85 (3.51 to 22.3) |
|  Tuberculosis | 24 | 16 (67) | 6.22 (2.36 to 16.4) | 4.12 (1.33 to 12.7) | 10.6 (2.33 to 48.7) |
|  Other infectious | 27 | 14 (52) | 5.07 (2.11 to 12.2) | 2.90 (0.91 to 9.23) | 8.11 (2.48 to 26.5) |
| External | 135 | 30 (22) | 0.86 (0.54 to 1.35) | 0.73 (0.44 to 1.21) | 1.45 (0.58 to 3.64) |
| Other | 150 | 43 (29) | 1.66 (1.11 to 2.48) | 1.21 (0.73 to 2.02) | 2.78 (1.52 to 5.06) |
| Unknown | 123 | 46 (37) | 2.42 (1.59 to 3.67) | 2.66 (1.55 to 4.55) | 2.03 (1.02 to 4.03) |

COPD=chronic obstructive pulmonary disease.

\*Cox hazard ratios for opium use, adjusted for sex (except when stratified by sex), ethnicity, education level, marital status, residential place, and cigarette smoking

Amongst 8487 opium users, most used teriak only (7308), shireh only (721), or a combination of the two. Other forms of opium use were far less common. Increased mortality was seen for each subtype of opium (see table A in appendix), with the strongest risk with heroin, albeit with wide confidence intervals. Use of shireh (adjusted hazard ratio 2.19) was associated with a slightly higher hazard of death than teriak (1.83), but the difference was not significant.

To assess the dose-response relation, we classified opium users as former users and current users. Current opium users had a higher hazard of death than former users (table 4). Among current users, longer duration of use was not associated with a higher mortality hazard (P=0.497 for trend). When we excluded 542 people who started opium use after receiving a diagnosis of a major illness (6.4% of all opium users), however, we found a dose-response association with duration of opium use (P=0.001 for trend). This pattern was also seen for several major causes of death. Assessment of the dose-response relation with cumulative use yielded similar results (see table B in appendix). Further sensitivity analysis with exclusion of deaths that occurred in the first 6, 12, 18, and 24 months of follow up did not have a material effect on the point estimates (see table C in appendix). Further adjustment did not change the results meaningfully (see table D in appendix), reducing the possibility of residual and unmeasured confounding. When we created a "healthy group" by excluding participants with any history of hypertension, ischemic heart disease, diabetes mellitus, or cerebrovascular events and underweight participants and those who had ever smoked cigarettes, the adjusted Cox hazard ratio for opium use in relation to overall mortality was 1.90 (95% confidence interval of 1.55 to 2.33).

 Cox hazard ratios\* (95% confidence intervals) by duration of opium use in relation to all mortality and cause specific mortality overall and after exclusion of participants who started using opium after diagnosis of serious disease (ischaemic heart disease, cerebrovascular event, hypertension, or diabetes mellitus) in Golestan Cohort Study

|  | Never users | Former users | Current users | Fifth of duration (years) of opium use in current opium users |  |  |  |  | P for trend† |
|----|----|----|----|----|----|----|----|----|----|
|  |  |  |  | 1 (≤3 years) | 2 (4-7 years) | 3 (8-12 years) | 4 (13-20 years) | 5 (≥21 years) |  |
| **All causes** |  |  |  |  |  |  |  |  |  |
| No of deaths | 1440 | 63 | 642 | 131 | 107 | 99 | 122 | 183 | — |
| Overall | Reference | 1.29 (1.00 to 1.67) | 1.94 (1.75 to 2.16) | 2.14 (1.79 to 2.57) | 1.85 (1.51 to 2.26) | 1.97 (1.59 to 2.42) | 1.81 (1.49 to 2.19) | 1.94 (1.64 to 2.29) | 0.497 |
| Before diagnosis‡ | Reference | 1.27 (0.98 to 1.65) | 1.70 (1.52 to 1.90) | 1.62 (1.31 to 1.99) | 1.39 (1.09 to 1.78) | 2.00 (1.62 to 2.48) | 1.79 (1.46 to 2.18) | 1.90 (1.60 to 2.25) | 0.001 |
| **Circulatory causes** |  |  |  |  |  |  |  |  |  |
| No of deaths | 738 | 38 | 297 | 63 | 53 | 50 | 59 | 72 | — |
| Overall | Reference | 1.57 (1.12 to 2.20) | 1.84 (1.58 to 2.14) | 2.09 (1.61 to 2.72) | 1.87 (1.41 to 2.49) | 2.05 (1.52 to 2.75) | 1.79 (1.36 to 2.35) | 1.55 (1.20 to 2.01) | 0.490 |
| Before diagnosis‡ | Reference | 1.40 (0.98 to 1.99) | 1.53 (1.30 to 1.80) | 1.21 (0.86 to 1.69) | 1.36 (0.95 to 1.94) | 1.99 (1.47 to 2.70) | 1.76 (1.32 to 2.34) | 1.48 (1.13 to 1.93) | 0.039 |
| **Cancer causes** |  |  |  |  |  |  |  |  |  |
| No of deaths | 312 | 8 | 129 | 13 | 22 | 17 | 30 | 47 | — |
| Overall | Reference | 0.77 (0.38 to 1.57) | 1.73 (1.37 to 2.18) | 0.96 (0.55 to 1.68) | 1.71 (1.10 to 2.66) | 1.52 (0.92 to 2.51) | 2.01 (1.36 to 2.98) | 2.26 (1.61 to 3.17) | 0.003 |
| Before diagnosis‡ | Reference | 0.81 (0.39 to 1.64) | 1.79 (1.41 to 2.26) | 1.57 (1.00 to 2.47) | 1.19 (0.68 to 2.10) | 1.70 (1.05 to 2.78) | 1.87 (1.23 to 2.86) | 2.32 (1.65 to 3.26) | 0.016 |
| **Respiratory causes** |  |  |  |  |  |  |  |  |  |
| No of deaths | 43 | 4 | 48 | 9 | 5 | 6 | 8 | 20 | — |
| Overall | Reference | 2.02 (0.70 to 5.84) | 4.05 (2.52 to 6.52) | 4.25 (2.02 to 8.93) | 2.64 (1.02 to 6.83) | 3.52 (1.45 to 8.56) | 3.36 (1.52 to 7.46) | 5.75 (3.11 to 10.6) | 0.117 |
| Before diagnosis‡ | Reference | 2.01 (0.69 to 5.82) | 3.85 (2.36 to 6.29) | 3.75 (1.71 to 8.23) | 2.42 (0.84 to 6.92) | 2.98 (1.14 to 7.77) | 3.56 (1.60 to 7.92) | 5.51 (2.94 to 10.3) | 0.078 |
| **Digestive causes** |  |  |  |  |  |  |  |  |  |
| No of deaths | 37 | 4 | 28 | 5 | 6 | 8 | 5 | 4 | — |
| All | Reference | 3.01 (1.02 to 8.85) | 3.14 (1.80 to 5.47) | 2.98 (1.14 to 7.75) | 3.93 (1.61 to 9.60) | 5.60 (2.49 to 12.6) | 2.60 (0.98 to 6.88) | 1.58 (0.53 to 4.67) | 0.168 |
| Before diagnosis‡ | Reference | 3.06 (1.04 to 9.02) | 3.16 (1.78 to 5.60) | 3.01 (1.15 to 7.88) | 3.15 (1.09 to 9.12) | 6.22 (2.76 to 14.0) | 2.87 (1.08 to 7.63) | 1.66 (0.56 to 4.93) | 0.219 |
| **Infectious causes** |  |  |  |  |  |  |  |  |  |
| No of deaths | 21 | 1 | 29 | 11 | 2 | 4 | 5 | 7 | — |
| Overall | Reference | 1.37 (0.18 to 10.5) | 6.05 (3.17 to 11.6) | 12.3 (5.69 to 26.7) | 2.23 (0.50 to 9.80) | 5.54 (1.81 to 16.9) | 5.07 (1.81 to 14.2) | 4.70 (1.80 to 12.3) | 0.270 |
| Before diagnosis‡ | Reference | 1.25 (0.16 to 9.70) | 5.23 (2.66 to 10.3) | 8.62 (3.63 to 20.5) | 2.54 (0.57 to 11.2) | 7.06 (2.52 to 19.8) | 4.22 (1.37 to 13.0) | 3.80 (1.38 to 10.5) | 0.424 |

\*Adjusted for sex, ethnicity, education level, marital status, residential place, and cigarette smoking.

†P for trend calculated by assigning consecutive integers to fifths of duration of opium use in current opium users.

‡Excludes those who started opium use after diagnosis with serious disease.

Table E in the appendix shows the results by broad categories of opium use—that is, ingestion and smoking. After exclusion of rare methods of opium use—for instance, the four and six participants who exclusively used heroin and sukhteh, respectively—of the remaining individuals, 5804 only smoked opium, 2176 exclusively ingested opium, and 497 reported both smoking and ingestion. Both methods were associated with increased risk of mortality: adjusted hazard ratio 2.08 for ingestion; 1.68 for smoking; and 2.34 for both. There was a similar pattern for most major causes of death, both before and after adjustment for either duration of opium use or its cumulative consumption.

The figure presents Cox hazard ratios in former users by years after cessation of opium use until they were enrolled into the study. The longer duration of cessation was associated with a decreasing hazard ratio (P=0.003 for trend).

Based on the overall adjusted hazard ratio for opium use of 1.86 (1.68 to 2.06), and assuming this represents a causal association, we calculate the fraction of deaths attributable to opium in this population as 14.9% (12.3% to 17.5%).

# Discussion

## Major findings

Opium use is common in this population in Iran and is associated with an estimated 86% increased risk of death overall; in women the increase was 143%. This increased risk was present for several major causes of death, including death from circulatory disorders and cancer, for different subtypes of opium (teriak, shireh, heroin, or combinations), for various routes of use (smoking and ingestion), and in several study subgroups (sex, ethnicity, residence, and others).

## Evidence for causality

This large prospective study looked at opium use in relation to overall mortality and cause specific mortality. Although causal inferences from this study could be premature, the findings are consistent with those from retrospective case-control studies that have shown increased risk of oesophageal cancer,3 4 bladder cancer,5 6 7 coronary heart disease,8 10 11 and other diseases12 13 14 in relation to opium use. Also, several of our findings point toward a causal role for opium in increasing mortality. A nearly twofold risk of mortality in opium users is substantial and is stronger than the estimated risk posed by tobacco use in this population. After adjustment for several factors, including tobacco use, the associations diminished only slightly but were still strong and significant.

People in this region might use opium to alleviate pain from chronic conditions, which could result in associations from reverse causality. To avoid misleading conclusions, we carried out a sensitivity analysis by excluding participants who started using self prescribed opium after the onset of four major chronic diseases. This resulted in some attenuation of risk that was restricted to recent users, as expected, but little change in risks for longer term users of eight years or more. Further sensitivity analysis excluding deaths occurring in the first 24 months of follow-up did not make a material difference either. Finally, longer opium cessation was associated with lower risk of death.

Opium alkaloid (such as morphine) and non-alkaloid (such as meconic acid) constituents, and other chemicals added during processing or generated during smoking, have a wide range of effects and can potentially explain the diverse increased risk of morbidity and mortality with opium use. Activation of µ-opioid receptors in pre-Bötzinger complex, located in the ventrolateral medulla, can reduce breathing frequency, tidal volume, chest wall compliance, and respiratory response to increased arterial carbon dioxide concentration.23 Opium use can also cause upper airway constriction through inhibition of acetylcholine release23 and increased bronchial tone through provocation of histamine release from mast cells.11 24 Smoking opium can also cause histological damage to the bronchi and lungs.20 Prolonged administration of exogenous opioids can have inhibitory effects on both humoral and cell mediated immune response.25 All of these mechanisms could underlie the strong associations seen with deaths from asthma, tuberculosis, and chronic obstructive pulmonary disease. Smoking opium can expose the body to carcinogenic polycyclic aromatic hydrocarbons, heterocyclic aromatic hydrocarbons, and primary aromatic amines.26 There is a growing body of evidence indicating that opiates could act as cancer promoters through impairing immune function, activating angiogenesis and tumour neovascularisation,25 increasing the concentrations of N-nitrosamines through modifying their pharmacokinetics,27 increasing the release of nitric oxide,28 or inhibiting production and release of hydrogen peroxide.29 Although opium use might have cardioprotective effects in short term use,30 it can have deleterious effects in the long term through causing hypotension, bradycardia, and respiratory depression that could decrease myocardial oxygenation and hence increase the infarct size and mortality.31 Morphine, like other opiate alkaloids, is metabolised principally in the liver via conjugation with glucuronic acid.32 Similar to other substances metabolised by the liver, morphine can result in hepatotoxicity.33 Extensive animal research has shown that morphine and morphine-like agents can increase liver enzyme activity and lead to perivenular and focal hepatocellular necrosis.34 35 36 37 Chronic administration of morphine in rats is thought to increase inflammatory infiltration and focal parenchymal necrosis in the liver, in addition to an increase in total lipid content of hepatocytes, all leading to lipid fibrosis and hepatic damage.38

Opium users in our cohort reported relatively low amounts of use (median 0.6 g/day), considerably lower than ranges of 1-24 g reported elsewhere.20 Although study participants might under-report the amount used, average daily use is likely to be low in this group as these were people who were able and willing to participate in the study, had low rates of loss to follow-up (\<1%), and whose education, ethnic background, and marital status was similar to non-users. Therefore, the opium users perhaps had a lifestyle that was not substantially different from others. These findings are in favour of a causal connection and suggest that even relatively small amounts of opium use over a long period can have a substantial effect on mortality.

Both opium ingestion and opium smoking were associated with an increased risk of death. Compared with ingestion, smoking exposes individuals to more polycyclic and heterocyclic aromatic hydrocarbons and primary aromatic amines26 but also exposes users to nearly 10 times less morphine. Almost 50% of opium that is smoked adheres to the opium pipe and of the other half that is vaporised in the pipe, a large proportion is lost in the smoke that escapes into the surrounding air.20

We found that opium increased the risk of death in nearly all subgroups of the study participants, including men and women, Turkmen and non-Turkmen people, people who lived in urban and rural areas, people with and without major chronic diseases, and smokers and non-smokers. The strength of association, however, differed in some subgroup analyses. Most importantly, the hazards ratios were higher in women than in men and in non-smokers than tobacco smokers.

Opium use is probably accurately reported in this population. A previous pilot study nested in this population showed that there was a strong correspondence between self reported opium use and opium metabolites in urine, with a sensitivity and specificity of 93% and 89%, respectively.17 Unlike alcohol, opium is not considered to be strictly religiously forbidden in this area, and recreational low dose consumption is culturally acceptable. Although there is evidence that the participants' report of any opium use was valid, we do not know whether their reported amount of use was also correct. In our analyses, cumulative use of opium, estimated by multiplying duration and amount of use, did not result in stronger associations than duration of use alone. Therefore, responses to amount of use could be subject to misclassification.

We used several methods to control for the effect of known and unknown confounders. Statistical adjustment for important potential confounders such as age, sex, place of residence, levels of education, marital status, BMI, height, intake of fruit and vegetables, a composite wealth score, alcohol consumption, and cigarette smoking status had little effect on the risk estimates. Compared with those who never used opium, stratified analyses showed a consistently significant increased risk of mortality among people who had ever used opium in almost all strata, including in those who had ever or never smoked cigarettes. Dose-response analyses showed a significant increasing trend of overall and cause specific risks of mortality for increasing doses of opium use. Finally, sensitivity analyses, through both classic and customised methods in our study, consistently showed the same results for the effect of opium use on risks of mortality. Although there could still be some unmeasured confounders or residual confounding from modelling or measurement errors, we believe their presence and effect would not be considerable because of extensive adjustment for several known confounders.39 40 Confounding factors should be strongly associated with both the exposure and outcomes to have a material effect,41 and we have adequately adjusted for age, smoking, and other confounders strongly related to both opium use and death; other unknown factors are unlikely to be significantly associated with both opium use and death. Whereas large scale randomised controlled clinical trials are the best design to eliminate or limit residual confounding, conducting randomised studies of opium and mortality is not ethically feasible; therefore prospective observational studies remain the design of choice in this setting.

Assuming a causal relation, the impact of opium use on mortality is substantial. About 20 million people in the world use opium or its derivatives, and these individuals are at a substantially increased risk of death.

## Strengths and limitations of the study

The strengths of this study include its prospective design, large sample size, availability of data to adjust for confounders, prior validation of exposure and outcome measurement, and minimal loss to follow-up. This study also has limitations, which include its observational design and therefore the potential for residual confounding and reverse causality. There is also a possibility of outcome misclassification as the cause of death was determined solely based on verbal autopsy in 35% of the cases. However, we used several methods to increase the validity of our outcome determination and to reduce the possibility of confounding and reverse causality.

## Conclusions and implications

In summary, in this large scale prospective study we found strong increased risks of death from multiple causes in opium users compared with non-users, even among those who used low doses of opium. These results are consistent with results of previous case-control studies in humans and animal studies suggesting deleterious health consequences of exogenous opioids such as morphine. Further epidemiological studies on opium use in relation to morality are needed. Also, follow-up studies of patients taking opioid analgesics long term for treatment of pain could shed further light on this issue.

### What is already known on this topic

1.  A small number of retrospective case-control studies or case series have described the possible role of opium in causing oesophageal cancer, bladder cancer, coronary heart disease, and a few other select health conditions

2.  There is little information from well designed prospective epidemiological studies on the effect of opium use, particularly low doses over a long period, on overall or cause specific mortality

### What this study adds

1.  Long term opium use, even in relatively low doses, is associated with an estimated 86% increased risk of death overall

2.  Users have an increased risk for several major causes of death, including circulatory disorders and cancer

Cite this as: *BMJ* 2012;344:e2502


abstract: # Background
  .
  This study examines whether young never smokers in Scotland, UK, who have tried an e-cigarette are more likely than those who have not, to try a cigarette during the following year.
  .
  # Methods
  .
  Prospective cohort survey conducted in four high schools in Scotland, UK during February/March 2015 (n=3807) with follow-up 1 year later. All pupils (age 11–18) were surveyed. Response rates were high in both years (87% in 2015) and 2680/3807 (70.4%) of the original cohort completed the follow-up survey. Analysis was restricted to baseline 'never smokers' (n=3001/3807), 2125 of whom were available to follow-up (70.8%).
  .
  # Results
  .
  At baseline, 183 of 2125 (8.6%) never smokers had tried an e-cigarette and 1942 had not. Of the young people who had not tried an e-cigarette at baseline, 249 (12.8%) went on to try smoking a cigarette by follow-up. This compares with 74 (40.4%) of those who had tried an e-cigarette at baseline. This effect remained significant in a logistic regression model adjusted for smoking susceptibility, having friends who smoke, family members' smoking status, age, sex, family affluence score, ethnic group and school (adjusted OR 2.42 (95% CI 1.63 to 3.60)). There was a significant interaction between e-cigarette use and smoking susceptibility and between e-cigarette use and smoking within the friendship group.
  .
  # Conclusions
  .
  Young never smokers are more likely to experiment with cigarettes if they have tried an e-cigarette. Causality cannot be inferred, but continued close monitoring of e-cigarette use in young people is warranted.
author: Catherine Best; Farhana Haseen; Dorothy Currie; Gozde Ozakinci; Anne Marie MacKintosh; Martine Stead; Douglas Eadie; Andy MacGregor; Jamie Pearce; Amanda Amos; John Frank; Sally HawCorrespondence to Dr Catherine Best, Faculty of Health Sciences and Sport, University of Stirling, Stirling, FK9 4LA, UK; <email@example.com>
date: 2018-07
institute: 1 Faculty of Health Sciences and Sport, University of Stirling, Stirling, UK; 2 Child and Adolescent Health Research Unit, School of Medicine, University of St Andrews, St Andrews, UK; 3 School of Medicine, University of St Andrews, St Andrews, UK; 4 Institute for Social Marketing, Faculty of Health Sciences and Sport, University of Stirling, Stirling, UK; 5 ScotCen Social Research, Edinburgh, UK; 6 Centre for Research on Environment Society and Health, School of GeoSciences, University of Edinburgh, Edinburgh, UK; 7 The Usher Institute of Population Health Sciences and Informatics, College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, UK
references:
title: Relationship between trying an electronic cigarette and subsequent cigarette experimentation in Scottish adolescents: a cohort study

# Introduction

In the UK and many other countries, e-cigarette use among young people is largely confined to those who have already tried tobacco and is mostly experimental in nature.1 2 That is, most young people who have never tried tobacco smoking, hereon referred to as never-smokers, do not engage in regular e-cigarette use that is sustained over time. Nevertheless, there remains concern that trying an e-cigarette could ease the pathway to experimentation with tobacco smoking for young never-smokers.

Eight longitudinal studies, all conducted in the USA with follow-up after 63 4 and/or 12 months,5–10 have explored the relationship between e-cigarette use and smoking initiation in young never-smokers. They found that young people who had ever used an e-cigarette at baseline were more likely to have tried a cigarette by follow-up.

Most of the evidence from prospective cohort studies of young never smokers, e-cigarette use and smoking initiation has come from the USA. It is important this evidence can be compared with studies from different countries because varied national contexts, such as different tobacco control regulations, historical and cultural factors around tobacco use, availability and supply of products, ethnic composition of the population and investment in advertising of products, make it difficult to generalise findings across national boundaries. For example, in Poland 27.4% of adolescents report using an e-cigarette in the past month.11 Poland is a major European tobacco and e-cigarette producer. Recently smoking rates have increased among Polish female adolescents although they are stable in men12–14 and by late adolescence most Polish e-cigarette users are dual users (tobacco and e-cigarette use). A recent study found 21.8% of students (16–18 years) were dual users and this was not associated with reduced cigarette consumption compared with tobacco-only users.15 The case of Poland highlights the potential role of national factors such as tobacco production and industry involvement in affecting levels of use in young people.

In Scotland, the prevalence of cigarette smoking among young people has steadily fallen over the last two decades. In 2015, only 2% of 13 year olds and 7% of 15 year olds were regular smokers.16 However, current smoking among young people aged 16–24 years in Scotland is significantly higher at 21%.17 This disparity suggests that smoking initiation may now be delayed until early adulthood. Therefore early risk factors for later smoking initiation require further investigation.

Previous cross-sectional research has shown a positive association between e-cigarette use and weakened intentions not to smoke in children aged 10–11 years in Wales.1 Recently the ever use of e-cigarettes among young non-smokers has increased in Scotland with 10% of non-smoking 15 year olds having tried them in 2013 and 24% in 2015.16 Levels of regular e-cigarette use among young people in Wales have also increased with 2.7% of young people aged 11–18 years reporting using them at least once a week in 2015.18 These increases were preceded by a marked growth in the retail availability of e-cigarettes with the proportion of retailers with displays of e-cigarettes doubling between 2013 and 2014.19 When this study was conducted, within store advertising and promotion of e-cigarettes was not regulated and there was no age restriction on the legal purchase of e-cigarettes in the UK. This study is one of the first to examine e-cigarette use and cigarette experimentation in a UK longitudinal sample.

# Methods

The data presented here are drawn from the Determining the Impact of Smoking Point-of-Sale Legislation Among Youth (DISPLAY) study.20 The DISPLAY study is a 5-year multimodal study designed to measure the impact of UK legislation to ban point-of-sale displays of tobacco products on the smoking attitudes and behaviours of young people. One element of the DISPLAY study is an annual school survey conducted in four Scottish secondary schools located in communities that differ in terms of their socioeconomic and urban–rural profiles. The data presented here are from the 2015 and 2016 surveys which included all pupils (aged 11–18) in the four schools. All four schools had pupils across the age range 11–18 years and a breakdown of participant numbers by school and by year group is given in table 1. The survey was administered by class teachers under exam conditions and took on average 40 min to complete. Pupils who were absent on the day of the survey were given opportunity during the following 2 weeks to complete the survey.

Number of 'never smoking' respondents by school and year group

Year 1
Year 2
Year 3
Year 4
Year 5
Year 6
Total

School 1

Accessible small town/medium–low deprivation
207

24.1%
184

21.5%
193

22.5%
129

15.0%
93

10.8%
52

6.1%
858

100%

School 2Urban/medium–low deprivation
147

19.9%
175

23.7%
136

18.4%
134

18.2%
85

11.5%
61

8.3%
738

100%

School 3

Other urban/high deprivation
177

26.3%
160

23.8%
106

15.8%
125

18.6%
62

9.2%
42

6.3%
672

100%

School 4

Urban/high deprivation
151

20.6%
197

26.9%
122

16.6%
126

17.2%
94

12.8%
43

5.9%
733

100%

Total
682
716
557
513
333
197
3001

Mean age (SD)
12.5 (0.34)
13.5

(0.34)
14.6

(0.34)
15.6

(0.35)
16.6

(0.36)
17.6

(0.32)
14.4

(1.58)

Ethical approval was obtained from the University of St Andrews, University Teaching and Research Ethics Committee (UTREC). Parental opt-out consent was obtained prior to pupils completing the survey. Pupils also provided active consent by completing the survey.

## Derivation of variables

### Smoking status

Respondents were asked "Have you ever smoked cigarettes or hand-rolled cigarettes (roll-ups), even if it is just one or two puffs?" to which they could respond 'yes' or 'no'. Young people who responded 'no' were deemed to be never-smokers at that point.

### E-cigarette use

Respondents were asked whether or not they had heard of e-cigarettes. Pupils who answered that they had not heard of e-cigarettes were routed past further questions on e-cigarettes. Pupils that had heard of e-cigarettes were then asked "Which ONE of the following is closest to describing your experience of e-cigarettes/vapourisers/shisha pens?" with response options of 'I have never used them', 'I have tried them once or twice', 'I use them sometimes (more than once a month)' or 'I use them often (more than once a week)'. Young people who responded that they had never heard of e-cigarettes were coded as having 'never used them'.

For the logistic regression analysis, due to low frequencies in the categories reflecting regular use, participants were divided into those who had never tried e-cigarettes versus those who had tried e-cigarettes.

### Susceptibility to smoking

Susceptibility to smoking was assessed through two questions "If one of your friends offered you a cigarette or hand-rolled cigarettes (roll-ups), would you smoke it?" and "Do you think you will smoke a cigarette or hand-rolled cigarettes (roll-ups) at any time during the next year?". The response option for these questions was 'definitely yes', 'probably yes', 'probably not' and 'definitely not'. If respondents answered anything other than 'definitely not' to either of these questions then they were coded as being susceptible to smoking. These measures of smoking susceptibility have been used in related studies6 and are based on validated measures.21

### Number of friends and family who smoke

Respondents were asked "How many of your friends smoke cigarettes or hand-rolled cigarettes (roll-ups)?" and could respond 'most of them', 'about half of them', 'some of them', 'none of them' or 'don't know'. 'Don't know' responses were coded as missing and then a binary variable was generated distinguishing those who responded 'none of them' versus any other response.

Respondents were asked "which if any of the following people smoke cigarettes or hand rolled cigarettes (roll-ups)?". Options included their mother or female carer, father or male carer, brother (eldest if more than one) and sister (eldest if more than one). A binary variable was created splitting participants who had responded that any of these family members smoked versus those that reported no smokers in their immediate family.

### Demographic variables

Respondents were asked their gender, ethnic group and date of birth. Individual family material well-being was assessed through the Family Affluence Scale (FAS).22 The FAS consists of four questions (own bedroom, number of family cars, number of computers and number of family holidays abroad per year). The FAS raw scores were transformed though categorical principal component analysis into single-dimensional scores that were then divided into tertiles of high, medium and low FAS.

## Analysis

Analysis was conducted in Stata V.14 (StataCorp).

Never smokers were divided into those who had tried an e-cigarette at baseline and those who had not and these groups were compared in terms of the proportion of participants that reported having experimented with cigarettes by follow-up. Tobacco experimentation in this study was defined as any cigarette use, even just one or two puffs.

Multivariate logistic regression was used to control for potential confounding factors—sex, age, ethnicity, family affluence, smoking within the family, smoking by friends and susceptibility to smoking. The model was built in three blocks, first with only e-cigarette use and smoking-related variables as independent variables and in the second block demographic variables were added and an indicator for school was included in the model. Including school as a covariate makes explicit the effect of school as school-level smoking norms are an important influence on smoking behaviour.23 In the third block interactions between e-cigarette use, smoking susceptibility and smoking within friendship group were included. The risk ratio (RR) for the unadjusted model was obtained from a binomial log-linear regression and for the adjusted models a Poisson regression model with a robust variance estimator.24

To test the effect of missing data on the parameter estimates, we used multiple imputation by chained equations (Stata V.14: mi impute chained). Further information on the imputation procedure is given in the online supplementary materials.

10.1136/tobaccocontrol-2017-053691.supp1

# Results

## Sample characteristics

In 2015, there were 3001 never smokers in our sample, of these 9.4% had tried an e-cigarette. Twenty-six per cent were coded as susceptible to smoking, 32.8% had a family member who smoked and 23.8% reported having at least one friend who smoked.

Our final sample included 2125 young people for whom we had data on e-cigarette use and smoking status at baseline and follow-up. Of these, 183 (8.6%) had tried an e-cigarette at baseline and 1942 (91.4%) had not. Table 1 shows the year group distribution of the sample by school.

## Relationship between baseline e-cigarette use and smoking status at follow-up in baseline never smokers

Of the young people who had tried an e-cigarette at baseline (n=183), 74 (40.4%) went on to initiate smoking cigarettes by follow-up. This compares with 249 (12.8%) of those who reported never having used an e-cigarette at baseline (n=1942) and went on to initiate smoking cigarettes by follow-up. Table 2 shows the bivariate relationship between e-cigarette use in 2015 and smoking status in 2016.

Baseline e-cigarette use in 2015 and follow-up smoking status in 2016

|  |  | Have you ever smoked cigarettes or roll-ups, even if it is just one or two puffs? (2016) |  | Total |
|----|----|----|----|----|
|  |  | No | Yes |  |
| E-cigarette use (2015) | I have never used an e-cigarette | 1693 | 249 | 1942 |
|  |  | 87.2% | 12.8% | 100% |
|  | I have only used them once or twice | 104 | 65 | 169 |
|  |  | 61.5% | 38.5% | 100% |
|  | I use them sometimes (monthly) | 3 | 5 | 8 |
|  |  | 37.5% | 62.5% | 100% |
|  | I use them often (weekly) | 2 | 4 | 6 |
|  |  | 33.3% | 66.7% | 100% |
| Total |  | 1802 | 323 | 2125 |
|  |  | 84.9% | 15.2% | 100% |

## Logistic regression on 'experimented with cigarettes by follow-up'

Baseline e-cigarette use is a significant predictor of experimentation with cigarettes. In an unadjusted model, the OR for ever-smoking at follow-up in ever e-cigarette users versus never e-cigarette users was 4.62 (95% CI 3.34 to 6.38), giving a RR of 3.15 (95% CI 2.55 to 3.89). Table 3 below shows the ORs, p values and 95% CIs for the OR for each of the models. All the models below were adjusted for sex, age centred on the mean (ie, individual age minus the mean age of the sample) FAS, ethnic group and school.

Multivariate logistic regressions on 'ever smoked a cigarette' in 2016

| Variable | Model 1—adjusted main effects model (n=1806) |  |  | Model 2—adjusted model including interactions (n=1806) |  |  | Model 3—imputed model with interactions (n=2520) |  |  |
|----|----|----|----|----|----|----|----|----|----|
|  | OR | CI | p | OR | CI | p | OR | CI | p |
| E-cigarette ever use 2015 | 2.42 | 1.63 to 3.60 | \<0.001 | 5.97 | 3.12 to 11.40 | \<0.001 | 6.64 | 3.60 to 12.26 | \<0.001 |
| Susceptibility to smoking 2015 | 3.65 | 2.70 to 4.94 | \<0.001 | 4.13 | 2.98 to 5.72 | \<0.001 | 5.19 | 3.74 to 7.21 | \<0.001 |
| Any family member smokes 2015 | 1.89 | 1.40 to 2.56 | \<0.001 | 1.93 | 1.43 to 2.61 | \<0.001 | 1.83 | 1\. 37 to 2.44 | \<0.001 |
| 'At least some' friends smoke 2015 | 1.33 | 0.95 to 1.85 | 0.094 | 1.56 | 1.09 to 2.25 | 0.016 | 1.51 | 1.07 to 2.14 | 0.020 |
| Interaction between e-cigarette and susceptibility |  |  |  | 0.42 | 0.19 to 0.94 | 0.036 | 0.42 | 0.20 to 0.88 | 0.021 |
| Interaction between e-cigarette and friends smoking |  |  |  | 0.49 | 0.23 to 1.07 | 0.072 | 0.52 | 0.25 to 1.09 | 0.082 |

Model 1 RR for e-cigarette use is 1.72 (95% CI 1.31 to 2.26), model 2 RR for e-cigarette use is 4.09 (95% CI 2.57 to 6.52), RR for e-cigarette\*susceptibility interaction is 0.43 (95% CI 0.25 to 0.72), RR for e-cigarette\*friend smokes interaction 0.62 (95% CI 0.39 to 0.99), model 3 RR for e-cigarette use is 4.22 (95% CI 2.83 to 6.36), RR for e-cigarette\*susceptibility interaction is 0.41 (95% CI 0.26 to 0.64) and RR for e-cigarette\*friend smokes interaction 0.65 (95% CI 0.44 to 0.97).

Figure 1 shows that the impact of having tried an e-cigarette at baseline on probability of tobacco experimentation at follow-up is much greater for young people who were non-susceptible to smoking at baseline. The contrast of predicted probabilities is significant (χ^2^=53.93, p\<0.001).

Figure 2 shows that the impact of having tried an e-cigarette at baseline on probability of tobacco experimentation at follow-up is much greater for young people who have no friends who smoke. The contrast of predicted probabilities is significant (χ^2^=4.91, p=0.042).

Further information on characteristics of missing cases is given in online supplementary materials. To test the effect of missing data on our parameter estimates we used multiple imputation by chained equations. Model 3 shows the estimates from an imputed model (m=100). The model estimates are stable under complete case analysis and imputation.

# Discussion

This study found that young 'never-smokers' who had tried an e-cigarette were more likely to try a cigarette during the following year than young never-smokers who had not tried an e-cigarette. This is consistent with the results of all previous published longitudinal studies of the relationship between e-cigarettes and tobacco experimentation in young people from the USA,3–9 providing further confirmation in a non-US context.

It is possible that the relationship between e-cigarettes and tobacco experimentation may not be causal if young never-smokers who try an e-cigarette would have gone on to initiate smoking anyway due to being already favourably disposed towards tobacco use. In other words, it is possible that e-cigarette use and tobacco experimentation have common liability25 and the former is incidental to tobacco experimentation. To address this possibility, we controlled for factors associated with transition to smoking such as smoking susceptibility26 and smoking among friends and family27 in the analysis. However, even when these items were included in the model e-cigarette use remained a significant predictor of cigarette experimentation. Importantly, there was also an interaction between smoking susceptibility and e-cigarette use and between e-cigarette use and having friends who smoked. These data indicate that e-cigarette use had a greater effect on the odds of cigarette experimentation in young people not traditionally thought to be high risk, that is, those with a firm intention not to smoke and/or those with no smokers in their friendship group.

There is some evidence from other studies that young people who try e-cigarettes before tobacco have different characteristics to those who go straight to smoking. Wills and colleagues28 found that those who used an e-cigarette first were less rebellious and more likely to receive social support from their parents. Miech and colleagues10 found that young never-smokers who had tried e-cigarettes were more likely to move away from the perception that cigarettes were a 'great risk' over the following year. Wills and colleagues29 also found that young never smokers who used e-cigarettes were also more likely to increase their positive smoking expectancies (such as beliefs that smoking would make them more confident, help them relax and reduce boredom) and were more likely to become friends with smokers and subsequently try smoking. However, with only 1-year follow-up these studies were not able to determine whether changes in expectancies or affiliations preceded smoking. Further research on this topic is required over longer follow-up periods.

Schneider and Diehl have outlined a 'catalyst model' of e-cigarette influence on smoking uptake in adolescence.30 This is intended as an alternative to 'gateway theory'31 32 as an explanation of the relationship between e-cigarette and tobacco use. They break the process down into two stages: factors influencing transition from 'no use' to 'e-cigarette use' and then the factors influencing the second stage of transition from 'e-cigarette use' to 'tobacco use'. The first-stage mechanisms include easing the process of initial trial, for example, with sweet flavours. The second-stage mechanisms include increased accessibility and learning of smoking rituals. Thus, there are a number of paths within the catalyst model whereby e-cigarette use, even single trial, might facilitate smoking uptake. There are also pathways by which e-cigarettes could mitigate against a transition to regular smoking. For those young people who are curious to try the performative aspects of smoking (the hand to mouth action and inhalation process), the act of trying e-cigarettes may result in lower motivation to try tobacco smoking.

Levy and colleagues have modelled the public health impacts of e-cigarettes and estimate that under a range of conditions, e-cigarettes may have a positive net impact on public health at a population level because of the greater benefits conferred on smokers relative to the potential harm to young people.33 Further studies could usefully examine e-cigarette use, smoking and smoking-related attitudes over longer time periods to determine the conditions under which e-cigarettes enhance adult quit rates without facilitating uptake in young people.

The importance of research findings about the relationship between e-cigarette use and smoking initiation has been debated on the basis that most e-cigarette use among young people is occasional and therefore unlikely to be directly harmful or be sufficient to influence other behaviours. However, some argue that the influence of e-cigarette experimentation may be psychosocial rather than chemical; it has been suggested that e-cigarettes '(convey) to young apprehensive would-be smokers that nicotine is a benign drug and potentially weaken the established message that smoking kills'.34 E-cigarette advertising has emphasised the commonalities between the products with the message that e-cigarettes can give the psychological and social benefits of smoking without the health or social costs.35 There are some signs that these messages confuse young people about the harms of smoking. For example, a recent study found that after viewing an e-cigarette advert young people were more likely to rate occasional cigarette smoking as less harmful.36

At the time this research was conducted there were no legal restrictions on sales or advertisement of e-cigarettes. However, in the UK e-cigarettes are now banned from sale to people under 1837 38 and advertising on television, print media and radio is prohibited under the Tobacco Products Directive and associated UK regulations,39 40 although at present point-of-sale marketing is still permitted. It will be important to ascertain if this legislation is sufficient to prevent or reduce the numbers of young people trying e-cigarettes.

## Strengths and limitations

The strengths of this study are its prospective design, large sample and high response and follow-up rates. Importantly, the multiple imputation models indicate that model estimates are not biased by missing data. However, there are a number of limitations. First, most of the young people whom we categorised as having initiated smoking may have only taken one or two puffs of a cigarette during the follow-up period. Therefore, we do not know whether any of these young people will transition to regular smoking. Transition from never-smoker to smoker is often conceptualised as a multistep pathway.41–43 Recent research suggests that any experimentation with cigarettes is a strong predictor of transition to regular smoking, with experimentation at baseline identifying two-thirds of regular smokers at 2-year follow-up with a false positive rate of only 8%.44

Second, participants were drawn from only four schools in Scotland and therefore may not be representative of the Scottish school population. However, comparison of the demographic characteristics of our sample with a nationally representative one does not indicate any significant deviation.45 Third, the study is based on self-reports and we do not yet know the reliability of young people's self-reported use of e-cigarettes.

The age range of the sample (11–18 years) is broader than in some other research in this area. Therefore, we split our sample in half by age and repeated the analysis on the split samples. The results we obtained were the same and are presented in the online supplementary materials.

Finally, although we have used validated measures of smoking susceptibility, they were developed more than 20 years ago and there may be other aspects of common liability to tobacco and e-cigarette use that are not assessed by existing measures of susceptibility.

# Conclusions

This UK longitudinal study found that young never-smokers who try e-cigarettes are at elevated risk of initiating smoking compared with young never-smokers who do not try e-cigarettes. Further research with longer follow-up is required to discover how many of the full sample of young people, if any, transition to regular smoking and to explore the longitudinal relationship between use of e-cigarettes and changes in attitudes to smoking. Careful and regular monitoring of smoking rates and e-cigarette use among young people is necessary over the coming years. This needs to be set within the context of the rapidly changing landscape of tobacco and nicotine product availability, recent changes in the regulation of advertising and strategies used by industry, particularly the tobacco industry, to promote these products.

###### What this paper adds

-   Eight prospective studies in the USA have reported a temporal relationship between trying an e-cigarette and subsequent experimentation with cigarettes.

-   Consistent with the US studies, this study indicates a positive relationship between e-cigarette use in never smokers and their subsequent first experimentation with cigarettes by follow-up 1 year later.

-   This UK study found that e-cigarette use had a greater impact on the odds of cigarette experimentation in young never smokers not traditionally thought to be high risk, that is, those with a firm intention not to smoke and/or no smokers in their friendship group.

# References