Data

About this day

Data is not a neutral input — it is generated by a process, for a purpose, and that process shapes what questions you can credibly ask. The morning lecture treats health data as a first-class citizen: the data-generating process, data collected as a byproduct of care versus designed for research, de-identification and governance, and the practical mechanics of access. The afternoon lecturers from Sage Bionetworks cover the broader health data landscape — finding, accessing, and ethical use of biomedical and genomics research data. The mode is interrogate: treat your dataset as a suspect.

Schedule

Time	Block
9:00–9:15	Morning handoff
9:15–10:30	AM lecture (Pollard)
10:30–11:00	Coffee
11:00–12:30	Small Group 1
12:30–1:30	Lunch
1:30–2:45	PM lecture (Sieberts & Banerjee)
2:45–4:15	Specialty Breakout #2 — data clinic
4:15–5:45	Small Group 2 (closing) — refine your data plan
5:45–6:00	Daily wrap — recap & look ahead
6:00	Dinner

Morning lecture outline

Data: the foundation, constraint, and failure mode of health AI. A working outline of the lecture’s content.

• The day on the spine. Day 1 turned a topic into a well-posed problem; today goes from the problem to the data that actually exists. The goal is not more data but the right data — and problem and data co-evolve.
• A stress test: COVID-19 prediction models. Of hundreds of pandemic models, reviews found essentially none clinically usable and some harmful — built on “Frankenstein” datasets spliced from many sources, learning spurious cues and evaluated on their own training data. A failure of scientific validity, not data management.
• The Surgisphere scandal. A fabricated dataset drove high-profile NEJM and Lancet papers, halted global trials, and was retracted — yet unreliable data persists. What was missing was infrastructure: provenance, cohort logic, leakage control, evaluation, governance.
• The central claim. Data is not a neutral input. It is generated by a process, for a purpose, and that process shapes what you can credibly ask.
• The data-generating process. Data collected as a byproduct of care (abundant, messy, workflow-biased) versus data designed for research (protocol-driven and cleaner for its purpose, but still biased).
• Care data are selected by action. Missingness is rarely at random; “controls” reflect clinical policy, not sampling; conditioning on a measurement can open collider bias. The data encodes the policy that produced it.
• Every modality is a measurement decision. Labs, vitals, notes, diagnosis/ billing codes, and imaging each carry their own bias — including labels that partly encode reimbursement rules, and the timing of a test carrying more signal than its value.
• A clinical dataset doesn’t just exist — it’s made. Using MIMIC to illustrate the choices behind extraction, structuring, and years of de-identification, and why reproducing a result needs shared, versioned cohort logic (the MIMIC Code Repository).
• De-identification, governance, and access. What Safe Harbor / expert determination do and don’t guarantee; why membership inference and memorization make re-identification a live risk (acute for clinical LLMs); and the real access path — credentialing, data use agreements, IRB — on a timeline of weeks to months, so start early.
• Standards versus representation. A common schema like MEDS solves syntactic portability, but representing irregular, multimodal event streams is still an open research problem (picked up on Day 4).
• What the data silently encodes. Temporal and case-control selection that can manufacture signal; measurements that are not neutral — pulse oximetry less accurate on darker skin, models reading race off chest X-rays, sepsis cohorts that shift with their definition; and the most dangerous bias of all: the patients who never enter the data, an absence that survives every downstream fix.
• Combining data is fusion, not addition. Pooling heterogeneous sites can compound bias; transportability requires stated invariance assumptions. And data only answers the question it was generated for — spending data, for instance, tracks the health system, not health.
• Two data worlds. Institutional clinical EHR data versus governed research-cohort and multi-omics data (consortium-scale, via platforms like Synapse) — setting up the afternoon’s Sage Bionetworks guest lectures.
• Interrogate your dataset. Where did it come from? Who is missing? What does it silently encode? Does it support your Day 1 problem? — feeding the day’s data-specification workbook part and the gap check between problem and data.

What you’ll be able to do

• Characterize a data-generating process and what it was built for.
• Anticipate provenance-driven bias before modeling.
• Produce a realistic data access plan.

Small group & specialty breakout

In Small Group 1, the group works through the Day 2 Workbook prompt: each person describes their data-generating process and what data exists independent of their AI system, and the group asks what bias that provenance creates. See how the small groups run.

The specialty breakout then runs as a data clinic: the main datasets, access routes, and shared biases in your domain.

The closing small group ends with each person noting how their thinking on data shifted over the day.

Project Workbook — Part 2: Data

The data realistically in scope at the point your solution must act, the data-generating process behind it, and what its provenance hides. Data and the problem aren’t separable — the problem already implies a data scope — so if the data you’d need won’t be there when you act, re-scope the problem or make collecting it part of the solution, and note that here.

Additional resources

• Slide decks: Morning lecture — Pollard (PDF) · Afternoon lecture — Sieberts & Banerjee.
• Project Workbook template — Part 2.
• PhysioNet — MIMIC-IV and credentialing.
• The MEDS (Medical Event Data Standard) documentation.
• Synapse — Sage Bionetworks’ governed-sharing platform for research-cohort, genomics, and consortium data.
• DREAM Challenges — community benchmarking on shared biomedical data.
• The datasets list (posted before the camp).

Suggested readings

1. Johnson et al. (2023), MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data.
2. Roberts et al. (2021), Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence.
3. Mehra et al. (2020, RETRACTED), Hydroxychloroquine or chloroquine for treatment of COVID-19: a multinational registry analysis (the Surgisphere scandal). The Lancet.
4. Yuan et al. (2021), Temporal bias in case-control design: preventing reliable predictions of the future. Nature Communications.
5. Bareinboim & Pearl (2016), Causal inference and the data-fusion problem. PNAS.
6. Einav et al. (2018), Predictive modeling of U.S. health care spending in late life. Science.
7. Chandra, Cutler & Song (2011), Who Ordered That? The economics of treatment choices in medical care. Handbook of Health Economics.
8. Fleming et al. (2024), MedAlign: a clinician-generated dataset for instruction following with EHRs. AAAI.
9. Gebru et al. (2021), Datasheets for Datasets. Communications of the ACM.
10. Gianfrancesco et al. (2018), Potential biases in machine learning algorithms using electronic health record data. JAMA Internal Medicine.
11. Sjoding et al. (2020), Racial bias in pulse oximetry measurement. NEJM.
12. Gichoya et al. (2022), AI recognition of patient race in medical imaging: a modelling study. Lancet Digital Health.
13. Johnson et al. (2018), A comparative analysis of sepsis identification methods in an electronic database (how the cohort definition changes the data). Critical Care Medicine.
14. Homer et al. (2008), Resolving individuals contributing trace amounts of DNA to highly complex mixtures (the genomic re-identification result behind controlled access). PLoS Genetics.
15. MEDS (2025), An emerging data standard and ecosystem for health-AI research. NEJM AI.
16. Overhage et al. (2012), Validation of a common data model for active safety surveillance research (the OMOP CDM). JAMIA.
17. Agniel, Kohane & Weber (2018), Biases in EHR data due to processes within the healthcare system — when a lab is measured can be more informative than its value. BMJ.