A famous Harvard researcher. A blockbuster finding about honesty. Suspicious data, failed replications, and a $25 million lawsuit. Welcome to one of the strangest research misconduct scandals in recent memory. In the first of two episodes about the Francesca Gino case, we investigate a famous experiment claiming that signing an honesty pledge at the top of a form makes people less likely to cheat. We follow the trail from a failed replication to some extraordinary detective work involving an Excel spreadsheet—and conduct a little statistical sleuthing of our own. Along the way, we discover how suspicious changes to a handful of observations can alter the story a scientific study tells.

Statistical topics

• Statistical sleuthing
• Data integrity
• Data management
• Replication
• Sensitivity analysis
• Effect sizes
• Statistical significance
• Reproducibility
• Spreadsheets
• Conceptual replication
• Direct replication
• CalcChain

Methodologic Morals

• “There's cleaning your data, and then there's laundering your data.” 
• “Good data record keeping is still sexy. Version control, baby.”

References

References

Published research

• Shu LL, Mazar N, Gino F, Ariely D, Bazerman MH. Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proc Natl Acad Sci U S A. 2012;109(38):15197-15200. Retracted. doi:https://doi.org/10.1073/pnas.1209746109
• Kristal AS, Whillans AV, Bazerman MH, Gino F, Shu LL, Mazar N, Ariely D. Signing at the beginning versus at the end does not decrease dishonesty. Proc Natl Acad Sci U S A. 2020;117(13):7103-7107. doi:https://doi.org/10.1073/pnas.1911695117
• Retraction for Shu et al., Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proc Natl Acad Sci U S A. 2021;118(38):e2115397118. doi:https://doi.org/10.1073/pnas.2115397118
  
  

Data Colada investigations

• Simonsohn U, Simmons JP, Nelson LD. Evidence of fraud in an influential field experiment about dishonesty. Data Colada. August 17, 2021. https://datacolada.org/98
• Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 1): “Clusterfake.” Data Colada. June 17, 2023. https://datacolada.org/109
• Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 2): “My class year is Harvard.” Data Colada. June 20, 2023. https://datacolada.org/110
• Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 3): “The cheaters are out of order.” Data Colada. June 23, 2023. https://datacolada.org/111
• Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 4): “Forgetting the words.” Data Colada. June 30, 2023. https://datacolada.org/112
• Simonsohn U, Simmons JP, Nelson LD. Exhibits 3, 4, and 5. Data Colada. September 16, 2023. https://datacolada.org/114
• Simmons JP. Harvard's Gino report reveals how a dataset was altered. Data Colada. July 9, 2024. https://datacolada.org/118
  
  

Additional resources

• Lessig L. The Gino Case. Website and podcast series presenting arguments challenging Harvard's research misconduct findings. https://theginocase.info/
• Gino v Harvard University. CourtListener. Federal court docket and publicly available filings. https://www.courtlistener.com/docket/67659904/gino-v-harvard-university/
• Simonsohn U, Simmons JP, Nelson LD. Data Falsificada. ResearchBox. Research data, analysis code, and supporting materials accompanying the Data Colada investigations. https://researchbox.org/1630.34
• Kristal AS, et al. Research materials and data accompanying Signing at the beginning versus at the end does not decrease dishonesty. Open Science Framework. https://osf.io/4b7mu/overview
• Correction for Kristal et al., Signing at the beginning versus at the end does not decrease dishonesty. Proc Natl Acad Sci U S A. 2026;123(27):e2610441123. doi:https://doi.org/10.1073/pnas.2610441123
• Harvard Business School. Investigative report concerning allegations of research misconduct involving Francesca Gino. Harvard Business School Research Integrity Investigation Report. 2023. Accessed October 18, 2026. https://datacolada.org/wp-content/uploads/Harvard-Report-on-Gino.pdf
  
  

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Program that Kristin teaches in::

Epidemiology and Clinical Research Graduate Certificate Program 

Kristin’s manuscript writing course:

https://instats.org/seminar/from-data-to-discussion-writing-a-scient

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Cold Open: Two Frauds in a Paper About Honesty
• (02:41) - The Claim: Spreadsheets Are the Villain
• (05:43) - The 2012 Honesty Pledge Study
• (09:17) - Signing at the Top Wins
• (13:31) - Failed Replications: Five Conceptual Tries, Then...

Can you spot problems in a scientific study just by reading the published paper? We put our statistical sleuthing skills to the test with a famous study on procrastination and deadlines that was retracted more than 20 years after it was published. But were there clues hidden in plain sight in the published results—and would we have noticed anything suspicious at the time? We put on our data-detective hats and find a few statistical clues of our own before turning to Data Colada’s investigation of the original data, where the story gets even stranger. Along the way, we hunt for missing participants, question suspiciously enormous effects, meet some unlikely data twins, and discover why sometimes the best statistical tool is simply asking: Do these data behave like real people?

Statistical topics

• Statistical sleuthing
• Effect sizes
• Degrees of freedom
• Replication
• Missing data

Methodologic Morals

• “A data detective's toolbox is surprisingly simple. Check the degrees of freedom, look for twins, question giant effects, and ask whether the data actually make human sense.”
• “You don't have to be a statistician to catch a statistical criminal.”

References

• Data Colada blog posts:
  • https://datacolada.org/138
  • https://datacolada.org/139
• Data and code: https://researchbox.org/7135
• Retraction Watch article: https://retractionwatch.com/2026/09/03/procrastination-study-duke-dan-ariely-psychological-science-data-colada-tampering-retraction/ 
• Original 2002 paper: https://journals.sagepub.com/doi/10.1111/1467-9280.00441 
• Retraction notice: https://journals.sagepub.com/doi/10.1177/09567976261488042 
• Hyndman K, Bisin A. Replication of "Procrastination, Deadlines, and Performance: Self-Control by Precommitment". Psychol Sci. 2026;37(8):557-571. doi:10.1177/09567976261460772

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Program that Kristin teaches in::

Epidemiology and Clinical Research Graduate Certificate Program 

Kristin’s manuscript writing course:

https://instats.org/seminar/from-data-to-discussion-writing-a-scient

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Introduction
• (02:15) - The Procrastination Paper and Study Two
• (08:51) - Suspiciously Tidy Results
• (16:18) - Study One: Real Classroom Stakes
• (20:06) - Missing Data and First Red Flags
• (28:17) - How Data Colada Got the Data
• (33:43) - When the Replication Failed
• (35:36) - Red Flags: Data Twins and Sanity Checks
• (47:21) - Study One Gets the Spreadsheet Treatment
• (51:45) - The Grading Scheme Unravels
• (57:51) - Manuscript Versions Reveal the Truth
• (01:08:11) - Rating the Claim and Methodological Morals

What happens when you discover that your published paper is wrong? Does retracting it ruin your reputation—or could owning the mistake actually make things better? In this episode, we meet the inaugural winners of the Ctrl-Z Awards, which recognize scientists for correcting or retracting their own work. Behavioral ecologist Kate Laskowski takes us inside a scientific detective story involving suspicious spider data, three retracted papers, and the fear that going public might tank her career. Then computational social scientist Alyssa Smith tells us how one small error in a figure led her to retract the paper at the center of her PhD and job talk. Along the way, we talk about data forensics, the stigma around retractions, and why self-correcting scientists are the heart of science.

Statistical topics

• Data management and documentation
• Open science
• Reproducible workflows
• Research integrity
• Retractions
• Scientific misconduct
• Statistical forensics

Methodologic Morals

• “Science is self-correcting, which means scientists need to be self-correcting too.”
• “Good documentation is boring—right up until it saves you.”
• “If you want scientists to correct their mistakes, reward them for correcting their mistakes.”
• “I'd rather be considered an honest idiot than for people to wonder if I'm untrustworthy.”

References

• Retraction Watch: https://retractionwatch.com/
• Ctrl-Z Awards: https://retractionwatch.com/ctrl-z-award/
• Announcement of inaugural Ctrl-Z winners: https://centerforscientificintegrity.org/2026/08/04/2026-winners-ctrl-z-award-courage-retracting-own-work/
•  Kate Laskowski’s blog post: https://laskowskilab.faculty.ucdavis.edu/2020/01/29/retractions/
• Kate Laskowski’s Data Detective Course syllabus
• Alyssa Smith’s site: https://asmithh.github.io/ 
• Laskowski KL, Montiglio PO, Pruitt JN. Individual and Group Performance Suffers from Social Niche Disruption [retracted in: Am Nat. 2020 Feb;195(2):393. doi: 10.1086/708066.]. Am Nat. 2016;187(6):776-785. doi:10.1086/686220
• Laskowski KL, Pruitt JN. Evidence of social niche construction: persistent and repeated social interactions generate stronger personalities in a social spider [retracted in: Proc Biol Sci. 2020 Jan 29;287(1919):20200077. doi: 10.1098/rspb.2020.0077.]. Proc Biol Sci. 2014;281(1783):20133166. Published 2014 Mar 26. doi:10.1098/rspb.2013.3166
• Modlmeier AP, Laskowski KL, DeMarco AE, et al. Persistent social interactions beget more pronounced personalities in a desert-dwelling social spider [retracted in: Biol Lett. 2020 Feb;16(2):20200062. doi: 10.1098/rsbl.2020.0062.]. Biol Lett. 2014;10(8):20140419. doi:10.1098/rsbl.2014.0419
• Smith AH, Green J, F Welles B, Lazer D. Emergent structures of attention on social media are driven by amplification and triad transitivity [retracted in: PNAS Nexus. 2026 May 07;5(5):pgag137. doi: 10.1093/pnasnexus/pgag137.]. PNAS Nexus. 2025;4(4):pgaf106. Published 2025 Apr 1. doi:10.1093/pnasnexus/pgaf106

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

Statistical power is one of the most important ideas in statistics—and one of the most misunderstood. In this episode, we explain what statistical power really means, why researchers need to think about it before they collect their data, and what goes wrong when they calculate “post hoc power” after the study is over. Along the way, we hunt for UFOs, test Kristin’s psychic abilities, explore the ethics of underpowered studies, and discover why post hoc power is really just a p-value wearing a fake mustache. Then statistician and powerlifter Andrew Althouse joins us for the inside story of a years-long scientific feud over post hoc power, where we discuss what happens when being wrong about statistics turns into doubling down, angry letters to the editor, and debates about how science corrects itself.

Statistical topics

• Effect size
• False negatives
• P-values
• Post hoc power
• Power calculations
• Research ethics
• Sample size
• Scientific controversy
• Sensitivity analysis
• Statistical power
• Statistical significance
• Study design
• Type II error

Methodologic Morals

• “Big questions need studies powerful enough to answer them.”
• “Post hoc power is circular reasoning masquerading as new information.”
• “Know when to cut your losses. A mistake gets more expensive every time you double down.”

References

• Althouse AD, Chow ZR. Comment on “Post-hoc power: If you must, at least try to understand.” Ann Surg. 2019;270:e78-e79.
• Althouse AD. Post hoc power: Not empowering, just misleading. J Surg Res. 2021;259:A3-A6.
• Altman DG. Statistics and ethics in medical research: III. How large a sample? BMJ. 1980;281:1336-1338.
• Bababekov YJ, Hung YC, Hsu YT, et al. Is the power threshold of 0.8 applicable to surgical science?—empowering the underpowered study. J Surg Res. 2019;241:235-239.
• Bababekov YJ, Stapleton SM, Mueller JL, et al. A proposal to mitigate the consequences of type 2 error in surgical science. Ann Surg. 2018;267:621-622.
• Chang DC, Stapleton SM. Response: The proliferation and misinterpretation of “as safe as” statements in surgical science: A call for professional discourse to search for a solution. J Surg Res. 2021;259:A12-A15.
• Freiman JA, Chalmers TC, Smith H Jr, et al. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial: Survey of 71 negative trials. N Engl J Med. 1978;299:690-694. 
• Hoenig JM, Heisey DM. The abuse of power: The pervasive fallacy of power calculations for data analysis. Am Stat. 2001;55:19-24.
• Nuzzo RL. Statistical Power. PM&R. 2016;8(9):907-912. doi:10.1016/j.pmrj.2016.08.004
• Nuzzo RL. Post hoc Power. PM&R. 2021;13(4):422-424. doi:10.1002/pmrj.12476

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Intro
• (00:43) - What is Statistical Power?
• (03:22) - The UFO Telescope Analogy
• (06:19) - The Psychic Ability Experiment
• (10:36) - Planning Power Before You Study
• (12:46) - History and the Landmark 1978 Paper
• (17:42) - Enter Post-Hoc Power: The Problem
• (19:26) - Three Problems with Post-Hoc Power
• (24:33) - The Surgery Journal Disaster
• (29:59) - Guest Interview: Andrew Althouse
• (32:30) - The 2018 Paper and the Controversy Begins
• (39:54) - Doubling Down and the Cost of Being Right
• (53:03) - Rating the Claim

Can weed protect your brain as you get older? We unpack a study linking lifetime cannabis use to larger brain volumes and better cognitive test scores in older adults—and investigate how those findings became headlines urging readers to “pass the pot to grandma.” Along the way, we explore collapsing categorical variables, confounding, clinically versus statistically significant results, multiple testing, and how a press release subtly changed a story. We also meet permanently stoned mice, learn about Carl Sagan’s surprising connection to cannabis research, and discover how a single missing word can change the entire meaning of a study.

Statistical topics

• Categorical variables
• Confounding
• Effect size
• False discovery rate
• Healthy volunteer bias
• Measurement error
• Multiple testing
• Observational studies
• Recoding variables
• Science communication
• Statistical vs practical significance

Methodologic Morals

• “Always follow the chain from data to result to conclusions and pay attention to the words along the way.”
• “Huge samples can give tiny differences delusions of grandeur.”

References

• Guha A, Fu Z, Calhoun V, Hutchison KE. Lifetime Cannabis Use Is Associated With Brain Volume and Cognitive Function in Middle-Aged and Older Adults. J Stud Alcohol Drugs. 2026;87(4):739-751. doi:10.15288/jsad.25-00346
• Press Release: https://news.cuanschutz.edu/news-stories/study-finds-cannabis-usage-in-middle-aged-and-older-adults-associated-with-larger-brain-volume-better-cognitive-function
• Bilkei-Gorzo A, Albayram O, Draffehn A, et al. A chronic low dose of Δ9-tetrahydrocannabinol (THC) restores cognitive function in old mice. Nat Med. 2017;23(6):782-787. doi:10.1038/nm.4311
• NY Post: https://nypost.com/2026/02/07/health/cannabis-may-benefit-aging-brains-study-finds/
• David Grinspoon: https://www.nytimes.com/2020/07/02/science/lester-grinspoon-dead.html?unlocked_article_code=1.7lA.CyS1.neeqGGNgx8g1&smid=url-share

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Opening: The Brain on Drugs
• (00:37) - The Cannabis Headline
• (03:38) - The Claim: Brain Benefits for Older Adults
• (06:53) - The Mouse Study: Pothead Mice vs. Control Mice
• (11:25) - From Mice to Humans: UK Biobank
• (15:01) - Measurement Problems and Category Collapse
• (21:39) - Results: Statistically Significant, Clinically Meaningless
• (27:13) - Individual Brain Regions and Cherry-Picking
• (32:32) - Cognitive Tests: What Actually Matters
• (40:09) - How It Gets Lost in Translation
• (45:32) - Rating the Claim

Can the meningitis vaccine do double duty against gonorrhea? We examine a fascinating randomized trial inspired by years of encouraging observational studies—and what happened when those earlier results were finally put to the test. Along the way, we discuss ecological studies, case-control studies, why statisticians distrust phrases like “trend toward significance,” how intention-to-treat differs from per-protocol analysis, and why no amount of statistical adjustment can fully substitute for randomization. Along the way we celebrate outrageously named clinical trials, ask whether “flirting with statistical significance” belongs in a romance novel or a medical journal, and discover that “practice safe statistics” may be the best advice of the episode.

Statistical topics

• Case-control studies
• Confounding
• Ecological studies
• Generalizability
• Intention-to-treat
• Meta-analysis
• Null results
• Observational studies
• P-values
• Per-protocol analysis
• Randomized clinical trials
• Survival analysis

Methodologic Morals

• “Don't mistake a p-value of 0.06 for a p-value of 0.04 with bad luck.”
•  “Practice safe statistics; randomize whenever possible.”

References

Seib KL, Donovan B, Jin F, et al. Meningococcal B Vaccine to Prevent Neisseria gonorrhoeae Infection. N Engl J Med. 2026; 395: 349-61. 

Petousis-Harris H, Paynter J, Morgan J, et al. Effectiveness of a group B outer membrane vesicle meningococcal vaccine against gonorrhoea in New Zealand: a retrospective case-control study. The Lancet. 2017; 390: 1603–10.

Wang, B., Mohammed, H., Andraweera, P., McMillan, M., Marshall, H., 2024. Vaccine effectiveness and impact of meningococcal vaccines against gonococcal infections: A systematic review and meta-analysis. Journal of Infection. 2024; 89: 106225. 

Molina JM, Bercot B, Assoumou, L, et al. Doxycycline prophylaxis and meningococcal group B vaccine to prevent bacterial sexually transmitted infections in France (ANRS 174 DOXYVAC): a multicentre, open-label, randomised trial with a 2 × 2 factorial design. The Lancet Infectious Diseases. 2024; 24: 1093–1104.

Thng C, Eskandari S, Jin F, et al. Efficacy of the meningococcal vaccine against Neisseria gonorrhoeae: a randomised clinical trial (MenGO). npj Vaccines. 2026.

Still Not Significant | Probable Error: List of funny disguises scientists use to describe p>.05

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:15) - - Introduction
• (00:51) - - The Claim: Can a Meningitis Vaccine Prevent Gonorrhea?
• (04:10) - - The Biology Behind the Idea
• (10:37) - - How Observational Studies Made the Link
• (14:44) - - Earlier Clinical Trials: MENGO and DOXYVAC
• (17:30) - - Flirting with Significance: A Statistical Rant
• (22:40) - - GoGoVax: Study Design and Analysis
• (33:31) - - The Shocking Null Result
• (35:38) - - Why These Results Don't Generalize
• (37:42) - - Rating the Claim

Can you improve your endurance by sitting in a hot tub instead of doing another hard workout? We dig into a study that put elite runners through five weeks of what we affectionately call “endurance hot tubbing” to see whether heat alone could trigger the same cardiovascular adaptations as altitude training. Along the way, we explore crossover study designs, multiple testing, why significant physiological changes don’t necessarily translate into better athletic performance, and how an exploratory regression analysis ended up answering the wrong scientific question. We also discover that serious hot tub research involves surprisingly little relaxation, debate whether bubbles should count as an experimental condition, and learn why distance runners pay good money to sleep in tents with some of the oxygen sucked out.

Statistical topics

• best subsets regression
• causal inference
• crossover studies
• exploratory analyses
• mediation / mechanisms
• model specification
• multiple testing
• peer review
• sample size

• (00:15) - Introduction
• (00:44) - Can Hot Tubs Replace Exercise?
• (01:27) - Crazy Runners and Crazy Training
• (05:44) - The Amazon Delivery System Metaphor
• (12:12) - From Heat Training to Hot Tubs
• (13:33) - The Study: 10 Runners in Hot Water
• (22:31) - Results and the VO2 Max Question
• (29:26) - Statistical Sleuthing: Best Subsets Regression
• (36:50) - The Peer Review Problem
• (41:39) - Rating the Claim

Methodologic Morals

• “A model can be very good at answering the wrong question.”
•  “The label ‘peer reviewed’ does not guarantee ‘carefully reviewed’.”

References

• Jenkins EJ, Killick JA, Zerilli O, et al. Long-term passive heat acclimation enhances maximal oxygen consumption via haematological and cardiac adaptation in endurance runners. J Physiol. Published online November 20, 2025. doi:10.1113/JP289874
• Stembridge M, Jenkins E. Marathon training: Why hot baths might help you run faster. The Conversation. Published March 16, 2026. doi:10.64628/AB.6vknm9esh

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:15) - Introduction
• (00:44) - Can Hot Tubs Replace Exercise?
• (01:27) - Crazy Runners and Crazy Training
• (05:44) - The Amazon Delivery System Metaphor
• (12:12) - From Heat Training to Hot Tubs
• (13:33) - The Study: 10 Runners in Hot Water
• (22:31) - Results and the VO2 Max Question
• (29:26) - Statistical Sleuthing: Best Subsets Regression
• (36:50) - The Peer Review Problem
• (41:39) - Rating the Claim

Could four one-minute bursts of exercise really improve blood sugar? We try “exercise snacks” ourselves before taking a close look at the clinical trial that inspired headlines. We explain why the study’s main result wasn’t statistically significant, how 34 secondary outcomes complicated the story, and what pre-registration can reveal about a study after it’s published. Along the way, we compare notes on our own exercise-snacking adventures, debate continuous glucose monitors, and ask how much evidence a single study should generate before it becomes health news.

Statistical topics

• Crossover design
• Multiple testing
• Pre-registration
• Primary vs secondary outcomes
• Randomized controlled trial
• Research transparency

Methodologic Morals

• “It's good relationship advice to be transparent. It's also good research advice.”
• “If the primary outcome is not significant, say it up top.”

References

• Babir FJ, Marcotte-Chénard A, Sandilands RE, et al. Exercise snacks performed in real-world settings reduce postprandial hyperglycaemia and glycaemic variability in individuals living with type 2 diabetes: a randomised crossover study. Diabetologia. 2026;69(8):2200-2211. doi:10.1007/s00125-026-06741-2
• https://www.washingtonpost.com/wellness/2026/05/28/4-minutes-exercise-day-could-help-control-blood-sugar/
• clinicaltrials.gov pre-registration with changes: https://clinicaltrials.gov/study/NCT06382246?term=NCT06382246&rank=1&tab=history&a=1&b=2#version-content-panel
  
  

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Intro
• (04:06) - The claim: four minutes a day
• (07:05) - Our own N of 1 experiments
• (13:24) - The study
• (21:54) - Primary outcome: complete miss
• (25:16) - Secondary outcomes to the rescue?
• (35:02) - Statistical sleuthing and transparency
• (44:23) - Rating the claim

How do you decide whether a clinical trial “worked”? In Part 2 of our Galleri series, we examine the landmark randomized trial of a blood test designed to detect more than 50 cancers. We explore why different outcome measures led to dramatically different headlines, discuss primary versus secondary outcomes, pre-registration, hierarchical testing, and post hoc analyses, and explain why mortality remains the outcome everyone is waiting for. Along the way, we uncover a statistical mystery involving dozens of missing cancers and discover how a little arithmetic can sometimes reveal more than a press release.

Statistical topics

• cancer screening
• exploratory analyses
• hierarchical testing
• missing data
• multiple testing
• outcome measures
• post hoc analyses
• pre-registration
• primary and secondary outcomes
• randomized clinical trials
• screening tests

Methodologic Morals

• “When the simple numbers don't add up, pay attention. The arithmetic may be trying to tell you something.”
• “The first question should not be, did it work? It should be, what counts as success?”

References

• Giridhar KV, et al. Safety and performance results from PATHFINDER 2, a registrational study of a multi-cancer early detection test in an intended-use population. Presented at the 2026 American Society of Clinical Oncology (ASCO) Annual Meeting. May 2026.
• Hubbell E, Clarke CA, Aravanis AM, Berg CD. Modeled Reductions in Late-stage Cancer with a Multi-Cancer Early Detection Test. Cancer Epidemiol Biomarkers Prev. 2021;30(3):460-468. doi:10.1158/1055-9965.EPI-20-1134
• Neal RD, Johnson P, Clarke CA, et al. Cell-Free DNA-Based Multi-Cancer Early Detection Test in an Asymptomatic Screening Population (NHS-Galleri): Design of a Pragmatic, Prospective Randomised Controlled Trial. Cancers (Basel). 2022;14(19):4818. Published 2022 Oct 1. doi:10.3390/cancers14194818
• ASCO slides: https://grail.com/wp-content/uploads/2026/05/Swanton_ASCO-2026_NHS-Galleri_FINAL-Slides-05.26.2026.pdf
• UK registry protocol:  https://www.isrctn.com/ISRCTN91431511 
• Clinicaltrials.gov protocol: https://clinicaltrials.gov/study/NCT05611632 

Common biases in cancer screening studies

Cancer screening studies are subject to several well-known biases that can make a screening test appear more effective than it actually is. Three of the most important are:

Lead-time bias: Screening advances the time of diagnosis, making survival from diagnosis appear longer even if the patient's lifespan is unchanged. For example, if a screening test detects a Stage II cancer at age 60 that otherwise would have been diagnosed because of symptoms at age 62, but the patient dies at age 68 regardless, survival from diagnosis appears to increase from 6 years to 8 years even though the patient did not live any longer. 

Length bias: Screening preferentially detects slower-growing, less aggressive cancers because they remain detectable for longer than fast-growing cancers. For example, a slow-growing cancer that remains in Stage I for 5 years is much more likely to be found by screening than an aggressive cancer that progresses to symptoms within months. This can make screened patients appear to have better survival simply because screening preferentially found the less aggressive cancers. 

Overdiagnosis: Screening detects cancers that would never have caused symptoms or death during a person's lifetime, leading to unnecessary diagnosis and treatment. For example, a screening test may detect a very slow-growing prostate or thyroid cancer in an older adult that would never have become clinically important if it had remained undiscovered. 

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Intro
• (03:39) - The Claim: Not Ready for Primetime
• (03:58) - Trial Design: 142,000 Participants
• (07:50) - The Primary Outcome Problem
• (20:29) - The Primary Endpoint: Complete Miss
• (22:14) - Three Arguments for the Defense
• (28:29) - - Statistical Sleuthing: Missing Cancers
• (41:14) - - The Stage Shift Argument
• (50:30) - - Rating the Claim

Can a single tube of blood really detect dozens of cancers before symptoms appear? We dive into the science behind Galleri, a blood test that claims to detect more than 50 types of cancer from a simple blood draw. Recent headlines about the test ranged from “breakthrough” to “bust” after the release of results from a massive randomized clinical trial. In this Part 1 episode, we explore cell-free DNA, DNA methylation, machine learning, sensitivity, specificity, and positive predictive value. Along the way, we revisit the prenatal screening revolution, ask why detecting cancer earlier doesn’t always help patients, and learn how escaped DNA convicts end up swimming in a giant molecular pool party. And for the first time ever, Normal Curves ends on a cliffhanger: we’ll save the controversial results of that landmark trial for Part 2.

Statistical topics

• cancer screening
• case-control studies
• counterfactuals
• machine learning
• negative predictive value
• overdiagnosis
• positive predictive value
• randomized clinical trials
• screening tests
• sensitivity and specificity
• validation

References

• Bianchi DW, Chudova D, Sehnert AJ, et al. Noninvasive prenatal testing and incidental detection of occult maternal malignancies. JAMA. 2015; 314:162-9. 
• Liu MC, Oxnard GR, Klein EA, et al. Sensitive and specific multi-cancer detection and localization using methylation signatures in cell-free DNA. Ann Oncol. 2020. 31:745-59. 
• Schrag D, Beer T, McDonnell C et al. Blood-based tests for multicancer early detection (PATHFINDER): a prospective cohort study. The Lancet. 402: 1251-60.
• Giridhar KV, et al. Safety and performance results from PATHFINDER 2, a registrational study of a multi-cancer early detection test in an intended-use population. Presented at the 2026 American Society of Clinical Oncology (ASCO) Annual Meeting. May 2026.

Statistic discussed in the episode

PATHFINDER 2 investigators reported that adding Galleri to routine screening increased the number of screen-detected cancers by 6.5-fold. This figure compares 31 cancers detected through USPSTF-recommended screening (for breast, cervical, lung, and colon) with 204 cancers detected when Galleri was added, counting the same 31 conventional-screening cancers in both totals. Thus, describing the increase as 6.5-fold is misleading, since the combination of Galleri plus conventional screening is, by definition, guaranteed to detect at least as many cancers as conventional screening alone. Moreover, everyone in the study received Galleri, whereas conventional screening depended on which tests participants happened to be due for and completed during the study period. The comparison therefore does not involve two equally applied screening strategies.

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - - Introduction
• (00:44) - - The Holy Grail of Cancer Testing
• (04:31) - - Headlines: Same Data, Opposite Stories
• (07:38) - - How Cell-Free DNA Works
• (13:54) - - DNA Methylation: GRAIL's Fingerprint
• (15:19) - - The Origin Story
• (22:18) - - The Pathfinder Studies
• (35:01) - - The Paradox: Why Earlier Detection Doesn't Always Help
• (40:32) - - The Cliffhanger

Odds ratios show up everywhere in medical research—but do readers, journalists, and even researchers always know what they mean? In this episode, we tackle one of the most common statistical misunderstandings in science: treating odds ratios like risk ratios. Along the way, we explore puppy photos, fish photos, first-date hookups, sugary drinks, cardiac care, and a listener challenge that started with an informal study of five medical residents and a box of chocolate truffles. We explain why logistic regression produces odds ratios, when odds ratios can wildly exaggerate effects, and why some famous headlines turned out to be much less dramatic than they sounded.

Statistical topics

• binary outcomes
• case-control studies
• logistic regression
• odds ratios
• risk ratios
• odds vs risk 

Methodological morals

• “Just because logistic regression gives you an odds ratio does not mean you have to report it.”
• “A lot of bad science communication starts long before the journalist even enters the story.”

References

• Bleich SN, Herring BJ, Flagg DD, et al. Reduction in purchases of sugar-sweetened beverages among low-income Black adolescents after exposure to caloric information. Am J Public Health. 2012;102:329–35.
• Sainani KL. How Statistics Can Mislead. Am J Public Health. 2012. 2012;102:e3–e4.
• Bleich SN, Herring BJ, Flagg DD, et al. Bleich et al. respond. Am J Public Health. 2012;102:e4.  
• Press video: https://www.youtube.com/watch?v=IFyrqbf1XWs 
• Sainani KL, Schmajuk G, Liu V. A Caution on Interpreting Odds Ratios. Sleep. 2009;32:976.
• Schulman KA, Berlin JA, Harless W, et al. The Effect of Race and Sex on Physicians' Recommendations for Cardiac Catheterization. NEJM. 1999;340:618–26.
• Schwartz LM, Woloshin S, Welch HG. Misunderstandings about the Effects of Race and Sex on Physicians' Referrals for Cardiac Catheterization. NEJM. 1999;341:279–83.
• Associated Press. Study Finds Bias in Doctors' Care of Women and Blacks. The New York Times. February 25, 1999.
• Knol MJ, Duijnhoven RG, Grobbee DE, et al. Potential Misinterpretation of Treatment Effects Due to Use of Odds Ratios and Logistic Regression in Randomized Controlled Trials. PLoS ONE. 2011;6:e21248.  

More information on logistic regression and odds ratios:

• Sainani KL. Logistic Regression. PM&R. 2014;6:1157–62.
• Sainani KL. Understanding Odds Ratios. PM&R. 2011;3:263–67.
• Nuzzo RL. Communicating measures of relative risk in plain English. PM&R. 2022;14:283-287.

When outcomes are common, odds ratios can exaggerate effect sizes. Alternatives include:

• Presenting raw percentages (absolute risks)
• Presenting adjusted percentages from logistic regression (these may be calculated by plugging in means for the covariates)
• Converting odds ratios to risk ratios
• Reporting risk ratios directly when appropriate

Converting Odds Ratios to Risk Ratios:

• Zhang J, Yu KF. What's the Relative Risk? A Method of Correcting the Odds Ratio in Cohort Studies of Common Outcomes. JAMA. 1998;280:1690–91.
• ClinCalc. Odds Ratio to Relative Risk Calculator.
•  https://clincalc.com/stats/convertor.aspx
• RR = OR / [(1 − P0) + (P0 × OR)]

Example:

OR=0.51, baseline risk=93.3%

RR = 0.51 / [(1 − 0.933) + (0.933 × 0.51)]

= 0.51 / (0.067 + 0.476)

= 0.51 / 0.543

= 0.94

Thus, an odds ratio of 0.51 corresponds to a risk ratio of 0.94 when the baseline risk is 93.3%.

The corresponding unadjusted risk ratio is 86%/93.3%=0.92

Correction: In the episode, we stated that the adjusted risk ratio was 0.92. In fact, it is 0.94, as shown above. 0.92 is the unadjusted risk ratio. 

Kristin and Regina’s online courses: 

Demystifying Data: A Modern Approach to Statistical Understanding  

Clinical Trials: Design, Strategy, and Analysis 

Medical Statistics Certificate Program  

Writing in the Sciences 

Epidemiology and Clinical Research Graduate Certificate Program 

Programs that we teach in:

Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:

Kristin -  LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Introduction
• (02:54) - What Are Odds Ratios?
• (04:02) - Puppy Photos and First Dates
• (06:09) - Risk Ratio Explained
• (08:10) - Calculating Odds Ratios
• (11:09) - Fish Photos and Reversed Numbers
• (16:01) - Real-Life Example: Sugary Beverages
• (22:08) - How Logistic Regression Works
• (31:53) - The Video: Researchers Made the Mistake Themselves
• (36:30) - The Cardiac Catheterization Study
• (39:24) - The New York Times Printed a Correction
• (46:10) - Using OR and RR Interchangeably for Case Control
• (47:00) - Reye Syndrome and Aspirin
• (49:37) - Rating the Claim and Methodological Morals