When hallucinations become real
My weekend adventure to invalidate a Gemini hallucination.
Troy Tassier is a professor of economics at Fordham University and the author of The Rich Flee and the Poor Take the Bus: How Our Unequal Society Fails Us during Outbreaks.
Last week I began reading about Ave Maria University in Florida where a current measles outbreak that began in early January has resulted in 46 student infections and 57 more students in quarantine.
I wondered, what is Ave Maria’s vaccination rate? Doing what we all do when seeking an answer to a simple question, I typed my question into Google. At the top of the page popped up Gemini’s answer to my question, 98%. (If not familiar, Gemini is Google’s AI tool.)
That’s a high rate. Now, I was even more curious. How could a small university with such a high vaccination rate have an outbreak this large moving through the student population so quickly?
I clicked on the “Campus Health Update” link that provided the source of Gemini’s answer. Here I found the official outbreak update provided by the university.
As you can see, the information had been misinterpreted by Gemini. The update does not state that 98% of students have been vaccinated, instead it says that the “vast majority” of its campus population are vaccinated and that a vaccination or prior infection provide 98% protection. Simply put, Gemini got it wrong.
An AI hallucination like this is not unusual. What I discovered next was unusual, or at least I hope it is.
Realizing that I didn’t have the answer, I returned to my Google search and looked at the list of links related to my question. Near the top of the list was a recent news article from a highly respected periodical. “Ah,” I thought, “this should provide a trustworthy answer.” I clicked. What popped up next surprised me, and troubled me even more.
This periodical provided the same 98% vaccination rate as the Gemini hallucination. Again, my curiosity was piqued. Maybe 98% really was the answer and Gemini had simply provided the wrong source. I clicked on the link that the appears in the more trusted source hoping to get confirmation of the stated vaccination rate. It took me to the same exact health update that was linked in the Gemini answer.
Now I was even more troubled. Gemini and this author had the same source. Had both misinterpreted the information from the source in the same way? Unlikely - the source was pretty clear. More likely I thought, the author simply trusted Gemini, provided Gemini’s answer as the truth, and cited its source without reading it carefully.
However, I wanted to be generous. I thought perhaps there is another source for the 98% value and continued scrolling.
Another article title caught my attention: “Florida Measles Outbreak Among Highly Vaccinated Population.”
Inside this article was the following statement: “Alarmingly, university officials have noted that approximately 98% of the student population is vaccinated against measles...” Here, no source was provided.
Now I was more than troubled. Not only had this online news article taken what appeared to be an erroneous piece of information as fact, they had titled their article in a way that misled the public. They insinuated that we can have a large measles outbreak even when the population is 98% vaccinated. With this information might people think, “why bother being vaccinated?” That would be dangerous.
At this point I stopped again to consider the possibilities and, being an economist, I turned to the numbers. Fifty infected students out of a population of 1,500 is 3%. If the rate of vaccination was in fact 98% that would mean that all of the unvaccinated students (2%) plus another 1% of the vaccinated students had been infected (or something approximating that). Even though no vaccine is perfect at providing immunity, a measles outbreak of this size, occurring this quickly (over just a few weeks), was ridiculously improbable if the university truly had a 98% vaccination rate. It would take an absurd bout of bad luck. At that point I was fully convinced that the answer could not be 98%. The journalists were relying on a hallucination or the university really did claim a 98% vaccination rate (somewhere) but was lying.
I emailed the author inquiring about their source for the vaccination rate and put my question aside for the day while I awaited an answer. (To date I have not received a response.)
However, being an academic, I couldn’t kick my curiosity aside that easily. The next morning I spent two hours combing the internet and the Florida Department of Health website searching for Ava Maria’s vaccination rate. I couldn’t find it. (The closest I came was a Florida DoH webpage listing the percentage of religious exemptions by census track. If you are curious, about 10% of children between ages 4 and 18 in the census track containing Ave Maria University have a religious exemption to the state vaccine mandate.)
My final piece of validation came yet another day later when the New York Times reported on the Ave Maria outbreak. Their article contained the following sentence: “The university did not respond to a request for comment. On its website, it says that the ‘vast majority’ of people on the campus are vaccinated, but it’s unclear exactly what percentage of students have exemptions.”
At this point, I decided that if the New York Times could not track down the information I was seeking, the answer probably wasn’t available. And, I became fully convinced that the journalists had relied on an AI hallucination for their reporting.
In this case however, the answer wasn’t the point.
Instead, I was left with concern about truth. When information provided by AI is taken as fact, the truth can become muddy. How many people would take 98% , either in the Gemini version or in one of the other more trusted media versions, as fact? How many times would the hallucination be repeated? At what point does a simple hallucination impersonate the truth so well that it actually becomes true in the public consciousness? If the hallucination becomes true in this sense, how does it get invalidated in the future? How difficult is it to do so? I don’t know how many students are vaccinated at Ave Maria and I don’t know the answer to any of these other questions either and that scares me.
Note: I do not have verification that either journalist relied on the answer provided by Gemini so I am not including their identities or the identities of their publications here.
Troy Tassier is a professor of economics at Fordham University and the author of The Rich Flee and the Poor Take the Bus: How Our Unequal Society Fails Us during Outbreaks.