We thank Daniele Fallin and Linda Kao,1 George Davey Smith,2 and Shalom Wacholder3 for their insightful commentaries. Fallin and Kao1 doubt whether X-WAS is the future of all epidemiology and whether the rest of epidemiology beyond genomics is as biased as we implied.4 Epidemiology, and observational research in general, comprises a vast cluster of diverse disciplines; some areas may be less biased than human genome epidemiology in the candidate gene era, but others are probably even more biased. There are hundreds of possible biases.5 No one denies that traditional epidemiology and observational studies have been successful at times; we can add numerous examples of successful identification of risk factors6 besides the 2 mentioned by Fallin and Kao.1 However, as indicated by fellow commentator, Davey Smith,2 the average track record of observational epidemiology to date has been poor, almost a systemic failure.7 The successes are needles in a haystack of associations reported in thousands and possibly millions of studies. A quick PubMed search with [risk factor OR (case-control OR cohort OR cross-sectional) OR association] yields 1,489,912 articles. Perusing but a small sample, we find 90% of these retrievals are observational epidemiologic studies, while many other observational studies are probably not captured by this preliminary search. Hundreds of thousands of scientists have been involved in observational studies across diverse domains. Some of them have not even realized that this is the approach they have been using, and that they may have employed spurious inferential methods. Some prolific cohorts have published well over 1000 papers, often each examining 1 or 2 exposures for one disease outcome. We do not think that such fragmentation serves our science well. Fallin and Kao indicate that findings from GWAS studies hold translational potential but are concerned that stringent criteria will limit the pursuit of important paths to translation. Successful translation occurs sparingly in biomedical science.8 This difficulty should be acknowledged and communicated to the wider public. We do not expect massive, agnostic approaches to make the translation process much easier. However, a rigorous approach with appropriate FP:FN stringency, large sample sizes, transparent design, and nonselective reporting will ensure a starting point with fewer false leads. False-positives continue to haunt the literature, even after being roundly refuted.9 Moreover, putative discoveries from observational studies continue to have a high refutation rate even when findings have reached the point of being close to translation.10 The current emphasis on translation by funders may exacerbate the false positive problem, as it presupposes validity of research findings. Funding incentives to rush to the bedside with presumed treatments will likely encourage bad science. We agree with the principle of including additional lines of evidence, such as animal models,11 in vitro work, and other sources of biologic plausibility. However, biologic plausibility also needs an equally systematic and transparent approach to the evidence—otherwise scientists may selectively invoke those pieces of the evidence that best match their results or prejudices.12 Biases related to statistical testing and selective reporting are also highly prevalent in these additional lines of biologic evidence.13–16 Wacholder3 nicely summarizes some differences between human genome epidemiology and other disciplines. He points out that, with more stringent FP:FN criteria, the power of current studies would be eroded. However, if one was able to put together all the fragmented single-study efforts in traditional epidemiology, the cumulative sample sizes of the resulting consortia would often be as large as or larger than the typical samples amassed in current genetic consortia. Future efforts may be even more far-reaching. If 50 biobanks are launched around the world, with an average of 500,000 participants each, the total sample size is 25,000,000 participants. Some nationwide studies can already accommodate information from many millions of participants each. Unless criteria of discovery and validation are stringent, these studies may show nominally statistically significant results for almost any tested association. Wacholder also highlights differences in traditional versus genome epidemiology in cost, free availability of measurements, and use of combined (pooled) analyses. We do not see why traditional epidemiology cannot follow the paradigm of genetics in these aspects. The cost of genetic measurements has decreased a million-fold in the last 20 years. One can also envision a sharp decrease in the cost of nongenetic measurements and data gathering. Cost is high when technologies are home-made, idiosyncratic, and nongeneralizable. Conversely, cost routinely decreases when a technology becomes standardized and then applied by many scientists using common analytic plans and converging purposes. Several envirome measurement platforms are already available and continuously being improved—including tools to capture the epigenome, metabolome, and microbiome; administrative databases and national registries; and electronic epidemiology, to name a few.17 We also see no rationale why traditional epidemiologic data should not be freely available,18 and why prospectively designed combined analyses in overarching consortia should not be the norm. Once these deficits have been amended, one can deal more seriously with the other differences that may indeed be more specific to nongenetic epidemiology, including the range of prior odds and the highly dense correlation pattern.19,20 Davey Smith2 shows how Mendelian randomization bridges the gap between genetic and nongenetic epidemiology and offers a wealth of new possibilities. We believe that it is important to bring different platforms of measurement to coexist within the same cohorts and biobanks.20 To date, most studies that have extensive genomic measurements have limited or no information on the envirome, and studies that are strong in environmental measurements often lack interest in genomics. Hopefully, this unfortunate dissociation will be gradually remedied. Finally, what does this all mean for policy and decision-making? As we pointed out originally4 and as our commentators further elaborated, scientific appraisal of the evidence is not the same as making policy decisions based on this evidence. We simply suggest that scientists should give transparent estimates about the FP:FN ratio of the field in which the information was obtained, systematic and transparent synopses of the prior evidence in the field, and estimates about the posterior odds that take the range of existing uncertainty into account. Scientific studies, when correctly executed and validated, inform us about what is, but not what ought to be. This may seem obvious to some, but one need only follow the newspapers or TV news to see what happens when scientists become advocates based on their own research findings.
Abstract Neurobiology-based interventions for mental diseases and searches for useful biomarkers of treatment response have largely failed. Clinical trials should assess interventions related to environmental and social stressors, with long-term follow-up; social rather than biological endpoints; personalized outcomes; and suitable cluster, adaptive, and n-of-1 designs. Labor, education, financial, and other social/political decisions should be evaluated for their impacts on mental disease.
Boucher presents several arguments for substantial benefits from vitamin D supplementation,1 but this is not clearly shown in our review.2 Given that we selected only one review per outcome for our analysis, the risk of overlap of original studies is minimal for any given outcome. Boucher’s comments on the importance of supplement doses …
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery phase, using strict protocols for analyses, pursuing complete and transparent reporting of all results, placing emphasis on replication, and being fair with interpretation of results.
Our website uses cookies to enhance your experience. By continuing to use our site, or clicking "Continue," you are agreeing to our Cookie Policy | Continue JAMA Internal Medicine HomeNew OnlineCurrent IssueFor Authors Podcast Publications JAMA JAMA Network Open JAMA Cardiology JAMA Dermatology JAMA Health Forum JAMA Internal Medicine JAMA Neurology JAMA Oncology JAMA Ophthalmology JAMA Otolaryngology–Head & Neck Surgery JAMA Pediatrics JAMA Psychiatry JAMA Surgery Archives of Neurology & Psychiatry (1919-1959) JN Learning / CMESubscribeJobsInstitutions / LibrariansReprints & Permissions Terms of Use | Privacy Policy | Accessibility Statement 2023 American Medical Association. All Rights Reserved Search All JAMA JAMA Network Open JAMA Cardiology JAMA Dermatology JAMA Forum Archive JAMA Health Forum JAMA Internal Medicine JAMA Neurology JAMA Oncology JAMA Ophthalmology JAMA Otolaryngology–Head & Neck Surgery JAMA Pediatrics JAMA Psychiatry JAMA Surgery Archives of Neurology & Psychiatry Input Search Term Sign In Individual Sign In Sign inCreate an Account Access through your institution Sign In Purchase Options: Buy this article Rent this article Subscribe to the JAMA Internal Medicine journal
Since 2007, genome-wide association (GWA) studies have identified numerous well-supported, novel genetic risk loci for common cancers; however, there are concerns that this technology is reaching its limits. We provide an overview of GWA-identified genetic associations with solid tumors. We simulated the distribution of population risk alleles for colorectal, prostate, testicular, and thyroid cancers based on genetic variants identified in GWA studies. We also evaluated whether statistical power to detect typical genetic effects could be improved with studies performing GWA analyses of all available samples rather than multistage designs. Fifty-six eligible articles yielded 92 eligible associations between cancer phenotypes and genetic variants with a median per-allele odds ratio (OR) of 1.22 (interquartile range = 1.15-1.36). Half of the associations pertained to prostate, colorectal, or breast cancer. Individuals at the upper quartile of simulated risk had only 2.1- to 4.2-fold higher relative risk than those in the lower quartile. Comprehensive evaluation of currently available samples with GWA platforms would yield few additional variants with per-allele OR = 1.4, but many more variants with OR = 1.2 could be detected; statistical power to detect weak associations (OR = 1.07) would still be negligible. The GWA approach is effective in identifying common genetic variants with moderate effect; however, identifying loci with very small effects and rare variants will require major new efforts. At present, the utility of GWA-identified risk loci in risk stratification for cancer is limited.
Polymorphisms in chemokine receptor genes may explain some of the heterogeneity in sustaining viral suppression observed among patients receiving potent antiretroviral therapy.
