SPEAKER NOTES — Slide 1 (Title) Open. ~75 min, INTERROGATE mode. You built the field's reference clinical dataset — you carry unusual authority on both the concepts and the practical reality. This deck is an expanded starter (26 slides); see guides/day2-build-guide.md for the figure plan and injection map. INSTRUCTOR tags in the notes mark where to add your own material.

SPEAKER NOTES — Slide 2 (Handoff & where we are) - Run the 10-minute handoff before this slide, or fold it in — invite 2-3 participants to state their Day 1 problem in one sentence. - INSTRUCTOR: pick problems you can reference later in the lecture - (e.g., when you reach the gap check, point back to one of them). - This is the first day of the standard daily arc; model the rhythm well.

"The general idea [behind Guardian Angel] was correct. The connection to the real world was absolutely lacking."

SPEAKER NOTES — Slide 9 (The wrong frame) Set the day's thesis. Yesterday: "problem before method." Today, complicate it productively — the problem and the data co-evolve. INTERROGATE mode: the cohort interrogates whether the data they assume exists actually does, in the form they need.

- Pandemic was a wake up call for data sharing - When COVID-19 hit us in 2020, machine learning researchers around the world turned their focus to building predictive models to help beat the pandemic. - Report by the Turing Institute concluded that of the hundred of algorithms built, - None useful, some even caused harm. - What went wrong? ... - This is why today is not a “data management” lecture. It is a scientific validity lecture.

- Talk about "Frankenstein datasets" - Researchers didn't have access to the data they needed. - So they went to the web...

- As a result, the algorithms were highly problematic

- Perhaps clearest example of academic misinformation came from Surgisphere! - Led by Sepan Desai, vascular surgeon, MD/PHD Chicago - Early in the pandemic, offered dataset of 100,000 patients over 671 hospitals. - High profile papers in NEJM and The Lancet on Chloroquine (see website) - Huge global impact. Clinical trials stopped around the world.

- People began to question whether the data really existed. - All came falling down - Monumental waste of time - Worse, it caused mass confusion and distrust.

- WHO admitted it had cancelled trials without seeing the data

- If you didn’t follow along at the time.. - If you are interested in the gory details.. - I recommend this article in the Scientist. - They say this prompted calls for “reviews of how science is conducted, published, and acted upon”

- https://www.medrxiv.org/content/10.64898/2026.02.24.26347028v1.full.pdf

SPEAKER NOTES — Slide 21 (Section divider) The data-generating process is the conceptual core of the day — give it a clean break.

SPEAKER NOTES — Slide 22 (Byproduct vs designed) Core conceptual point. Hammer the byproduct-of-care idea: every quirk of clinical data — missingness, timing, coding — traces back to the fact it was generated to deliver and bill care, not to answer the participant's question. That mismatch is the day's recurring theme.

SPEAKER NOTES — Slide 23 (The central claim) The thesis, stated plainly so it holds for 75 minutes. Everything after is evidence: the generating process determines what the data can and cannot answer. Say it slowly.

We often think about isolated sets of data. Helpful to remind ourselves that this is a timeline. A set of interconnected events, each generating data. When we are looking at ICU data, we are looking at a small component.

SPEAKER NOTES — Slide 25 (Case study: MIMIC) Use MIMIC to illustrate the generating process — NOT as a product pitch. You have first-hand history. INSTRUCTOR: tell the cohort what surprised YOU about how hard "ready-to-use" was to achieve, and one shaping decision you made that downstream users inherit without knowing. The reproducibility angle is yours to press — the MIMIC Code Repository and "can you reproduce this published mortality-prediction result?" make the shaping-decisions point land; the datathon is the natural live version. → guides/instructor-resources.md. The lesson generalizes to any dataset.

SPEAKER NOTES — Slide 36 (Data is sampled from a policy) For this audience, "data is messy" is not news — formalize it. - The data is a draw from a clinical decision policy, so missingness is MNAR by construction and control definitions are selection mechanisms. - Reach for selection diagrams (Bareinboim & Pearl, data-fusion / PNAS 2016), not just the MCAR/MAR/MNAR taxonomy. - The collider point (conditioning on measurement) is the one to dwell on — it is how naive analyses fabricate signal. - a **selection diagram** is the right object (Bareinboim & Pearl). - Bareinboim & Pearl: https://www.pnas.org/doi/epdf/10.1073/pnas.1510507113 - MCAR/MAR/MNAR labels are the floor.

SPEAKER NOTES — Slide 37 (Each modality is a decision) Replaces a reference-card table with the identifiability lens this audience needs. The right two columns are the content: every modality's *presence* is a clinical decision, and conditioning on it has consequences (collider bias, spectrum bias, measurement-rate-as-severity). Pick one row to work through; the labs/collider row is the sharpest.

SPEAKER NOTES — Slide 39 (Section divider) Pivot from "where data comes from" to "how you actually get it." This section is the highest-utility content of the day for a room of PhD students.

SPEAKER NOTES — Slide 40 (De-identification & governance) Keep the practical floor (Safe Harbor vs. expert determination) to one line, then give this audience the privacy frontier they actually care about: de-identification of a dataset does not de-identify a *model* trained on it (membership inference, memorization in clinical LLMs), and the DP/synthetic-data "solutions" carry real utility and distribution-shift costs. INSTRUCTOR: name the de-id standard your institution uses.

- Time-consuming but important step is manual review. - One powerful approach is to group values and order by frequency. - Infrequent values often contain PHI. - We also use a script to gather news articles mentioning Beth Israel patients. - Remove anyone featuring in the articles. - This Yankee's fan will not appear in MIMIC

- Pattern matching is very important part of the deidentification process. - e.g. we know patient names, dates, etc, so we use this information. - e.g. dociments often follow consistent structure, such as Name: XXX - e.g. PHI entities also follow consistent structure, e.g. years, names. - While very effective, also fragile. - What I mean by fragile is that small inconsistencies in the data can lead to complete failure (e.g. a malformed date).

- We combine the previous approaches with machine learning models. - This paper describes a BERT model that we have used in the past. - Benefit is flexibility. Picks up PHI that patterns miss. - e.g. finds malformed dates that patterns missed.

SPEAKER NOTES — Slide 45 (Access mechanics) The slide participants most need. A room leaving able to actually get data is a real outcome. INSTRUCTOR: give the CURRENT PhysioNet steps and one or two pitfalls (training certificates expiring; DUA institutional sign-off lag). Be concrete about who signs the DUA at a typical university.

SPEAKER NOTES — Slide 52 (Standards vs. representation) Don't sell "structure is nice" to this audience. The framing: a schema like MEDS solves the *syntactic* portability problem, but how to *represent* irregular multimodal event streams (tokenization, temporal encoding, vocabulary) is unsolved and contested — a live research area. You plant it; McDermott harvests it on Day 4. The (patient, time, code, value) tuple is the one concrete anchor.

SPEAKER NOTES — Slide 53 (Section divider) Pivot from "how to get data" to "what is silently wrong with it." This is the INTERROGATE heart of the day.

SPEAKER NOTES — Slide 54 (Provenance-driven bias) - Time - Looks like patient at time of covid - Cohort - Signal partly represents clinician suspicion - Selection - People are missing if they never sought care - Documentation - Codes may be for billing.

SPEAKER NOTES — Slide 55 (Label-as-billing-artifact) Give the concrete "the label is really a billing artifact" example the outline asks for. INSTRUCTOR: a specific code from your experience (e.g., a sepsis or AKI definition that shifted with a billing or guideline change) makes this vivid. Connects back to Pete's Day 1 "label noise" feature and forward to Day 3 estimands.

- “Data” are not just the patient’s oxygenation. They are oxygenation as mediated by a device, skin pigmentation, clinical thresholds, and treatment protocols. - Photoplethsymogram is a device that measures your blood O2 with lght. - We know that PPGs are less accurate for patients with darker skin. - Study demonstrated this resulted in different O2 therapy for different patients.

- This is the multimodal version of “measurement decision.” An image is produced by referral, protocol, scanner, positioning, and acquisition workflow. The model receives all of that, not just the disease. - Example of the kind of research people are doing. - Study published in Lancet Digital Health - Demonstrated that AI models could detect race of patients using Chest X-rays. - Important implications in AI ethics and bias.

-

this is a multimodal physiologic record showing one cardiovascular event propagating across different measurement systems. From top to bottom, the channels appear to be: II, V, aVR — ECG leads, showing the heart’s electrical activity ABP — arterial blood pressure waveform CVP — central venous pressure PAP — pulmonary artery pressure Pleth — pulse oximeter plethysmography waveform, highlighted in orange Resp — respiration waveform - The ECG spike comes first, then the arterial pressure pulse, then the pleth pulse a little later in time - Pleth is on a different clock to ECG

-

SPEAKER NOTES — Slide 64 (Who is missing) "Who is missing entirely" is the bias that survives every downstream fix because it was never in the data. Plant the equity thread Days 3 (Salaudeen) and 5 (Raji) develop. Tie absence to access, not health. Leads into the data-fusion frontier slide.

SPEAKER NOTES — Slide 65 (Data fusion) — CORE The advanced version of the day's own thesis, for a strong room. Data fusion under heterogeneous selection can compound rather than cancel bias; "more sites" is not a generalization guarantee. Load-bearing for this audience — keep it. Callback to Pete's Day 1 shift slide; sets up Mamdani's Day 5 transport story.

SPEAKER NOTES — Slide 66 (Data answers its own question) Connect to the economics readings. Data carries the imprint of its purpose: utilization data tells you about access, incentives, and billing as much as about disease. INTERROGATE in one sentence: ask of any variable, "what was this generated to do?" — the answer bounds what it can tell you.

SPEAKER NOTES — Slide 67 (Section divider) Set up the guests. This frames the Sage Bionetworks guest lectures (Sieberts & Banerjee) as the other half of one picture, not separate topics.

SPEAKER NOTES — Slide 68 (Interrogating your dataset) This IS the structure of the deliverable and the small-group prompt. Walk the four questions slowly. Question 4 is the gap check — give it its own slide next.

SPEAKER NOTES — Slide 69 (The gap check) This is the pivotal move of the day and the reason data follows problem on the spine. INSTRUCTOR: point at one of the morning's handoff problems and run the gap check live, out loud — it models exactly what they do in small group. Some will discover their Day 1 problem needs revising; that is a success, not a failure.

SPEAKER NOTES — Slide 70 (Activity / transition) Hand off and preview the guest's contrasting half. Name the deliverable's four parts clearly — they map onto the small-group prompt and the four questions on Slide 24. End by reminding them: a problem statement without a data spec is still a wish.