Back to the atlas

SQYER · write-up

We ran a week of FDA device reports through TypeSafe Jev. Here's what we built.

If a medical device breaks, hurts someone, or just does something odd, somebody is supposed to tell the FDA. Those reports pile up in a public database called MAUDE. Each one has a few coded fields and, most of the time, a paragraph or two of free text describing what happened. The coded fields get counted. The paragraphs mostly don’t get read, because there are millions of them and they are written in all caps by whoever had the form open that day.

We wanted to know what a text-reading model does with that pile. So we took one week of it and asked.

The week

The slice is every report FDA received between August 24, 2026 and August 30, 2026, as published in the openFDA export of September 16, 2026. That is 116,623 reports. We picked a week rather than a month or a year for a dull reason: it is the largest chunk we could process, check, and publish in a day, and the most recent complete week available at the time. FDA’s export had nothing for September yet.

One thing surprised us. 64,085 of those reports, 55%, had no narrative text at all in the export. Not redacted, just absent. The same week a year earlier has text on nearly every report, so this looks like a lag between a filing arriving and its narrative being published, not a change in what people write. We counted those reports in the totals and left them out of everything else. The 52,538 that did have text are what the rest of this page is about.

Weekday volume ran a little over twenty thousand reports a day. Saturday the 29th had 480.

Eight questions, asked 52,538 times

Jev is a model from TypeSafe that answers typed questions about a piece of text. It does not write anything. You hand it the text and a question with a fixed shape, and it hands back probabilities: a distribution over a list of options, a score on a scale, or a yes/no with a number attached. We were curious whether that shape holds up on regulatory prose, which is repetitive, formulaic, and full of holes where the redaction markers used to be.

We wrote eight questions and asked all eight of every narrative. Same wording every time. No follow-ups, no retries to get a nicer answer.

  1. Failure the narrative describes. Which best describes the device failure described in the narrative? choice
  2. One clear failure?. Does the narrative describe one identifiable device failure clearly enough to categorize it? noul
  3. Harm the narrative describes. What harm to the patient does the narrative text describe, setting aside the coded fields? choice
  4. Narrative says device was returned and analyzed. Does the narrative say the device was returned to the manufacturer and examined or tested? noul
  5. Manufacturer says it could not confirm. Does the manufacturer state that the reported failure could not be confirmed, duplicated, or reproduced on analysis? noul
  6. Where the narrative places the cause. Where does the narrative place the cause of the event along this device-versus-user axis? score
  7. Trend / CAPA / recall. Does the narrative mention similar prior events, a trend, a CAPA, a field action, or a recall? noul
  8. Manufacturer says information was not received. Does the manufacturer say it requested but did not receive further information (e.g. from the user facility)? noul

Two of those questions are there to keep the others honest. The second one asks whether the text describes one identifiable failure at all; if the answer to that is shaky, we do not count the failure mode from the first question, and the report shows up as “not categorized.” And for every yes/no question we treat anything between 0.2 and 0.8 as uncertain. It is drawn on the page as a hatched band and it never counts toward a yes or a no. That rule cost us a lot of clean-looking bars. We kept it.

What the page does

The atlas is one screen. Pick a device type, a manufacturer, a coded event type, or a harm tag, and every chart recomputes for that slice in your browser. There is no server behind it. The whole week’s tags come down as one file, and the charts are just counting.

The grid we look at most compares the harm tag from the narrative with the event_type box the reporter ticked. Most of the time they agree. When they don’t, you are looking at two descriptions of the same report that disagree with each other, one coded and one written. In this week, 195 reports were coded Death. The narratives of 30 other reports were tagged as describing a death while the box said something else. That is a reason to open those 30 and read them. It is not a finding that anyone died, or that the coding was wrong, and we have tried to say so everywhere that number appears.

Which brings up the last part of the page: the reports themselves. We published 10,000 full narratives, chosen so that every report coded Death or tagged as describing death is included, with injuries and malfunctions filling the rest in proportion. Each one shows the original text, the FDA codes, all eight answers with their probabilities, and a link to the record on FDA’s site. If a tag looks wrong to you, the text is right there.

What it isn't

This is our test of a model on public text. It is not a finding by SQYER, and it is not a finding by Jev. Neither of us is making a medical, regulatory, or legal determination about any device, company, or event, and nothing here is medical advice or a substitute for the FDA record.

MAUDE has its own caveats and they apply to everything built on it, including this. FDA does not verify the reports. A report does not show that a device caused anything. Counts are not rates, because reporting depends on who was required to file and who bothered. And the date on a report is the day FDA received it, which can be weeks or months after whatever it describes.

We also don’t publish performance claims about the model here. Look at the answers next to the text and decide for yourself.

Why we bothered

Almost every organisation has a MAUDE. Complaint logs, service tickets, inspection notes, the free-text field at the bottom of a form that nobody built a report for. The structured columns get dashboards. The text gets skimmed by whoever has time, which is nobody.

A model that answers the same fixed questions about every record, and tells you how sure it is, turns that text into something you can count and, more importantly, something you can check. That is the idea we wanted to try on a dataset where anyone can look over our shoulder. The plumbing is not exotic: pull the files, flatten them, scrub anything that looks like a name or a phone number, ask the questions, store the answers next to the source fields, publish a static snapshot. If you have text like this and want to see what it would look like as answers, SQYER does this for a living.

Hire SQYER

Notes for the careful