Stop using generative AI as a search engine

How many presidents have pardoned their relatives? It turns out that the answer to this question is complex.

Following the pardon of Hunter Biden by his father, several commentators looked at precedents – other pardons of relatives. Example: Ana Navarro-Cardenas, a commentator who appears on The view and CNN. At X Navarro-Cardenas cited the pardon granted by President Woodrow Wilson to his brother-in-law Hunter deButts. That was news to me.

The official clemency records search only works for people who have applied since 1989 and the site recipients of presidential pardons it goes back to Richard Nixon. Such a pardon would be controversial, but it was not mentioned on the biography page in the Wilson Presidential Library. Find a Grave suggests that Wilson didn’t even have a brother-in-law with that name – that shows nine brothers-in-lawbut not our man Hunter deButts. I can’t prove Wilson no sorry to Hunter deButts; I can only tell you that if he did, that person was not his brother-in-law.

Navarro-Cardenas wasn’t the only one sending confusing indulgences. An Esquire article titled “Shouldn’t the president pardon his son? Hi, does anyone remember Neil Bush?” was based on the assumption that George HW Bush had pardoned his son Neil; it has since been pulled “due to a bug”. The day before it was published, Occupy Democrats executive editor Grant Stern tweeted a similar claim that Jimmy Carter pardoned his brother Billy and George HW Bush to Neil. As far as I know, not a single pardon actually occurred.

Where did it all come from? Well, I don’t know what Stern or Esquirethe source was. But I know Navarro-Cardenas because she had a follow-up message to critics: “Get started with Chat GPT.”

I did. I asked ChatGPT and it identified Hunter deButts as the husband of Wilson’s sister Anne. “Woodrow Wilson’s family was quite prominent, and his sister Anne married Hunter deButts, who was a wealthy and socially connected individual from a prominent family,” ChatGPT told me. “Hunter DeButts was part of Wilson’s extended family, although he is not as well remembered in historical records as other figures in Wilson’s life.” According to The New York TimesAnne Wilson married a man named George Howe. It is not clear where the name Hunter deButts even comes from.

ChatGPT, it turns out, is a woefully bad way to examine historical records. When I opened it and asked, “How many US presidents have pardoned their relatives? I got one correct answer: Bill Clinton pardoned Roger Clinton, his half-brother. But along the way, ChatGPT also told me that George HW Bush granted a pardon his son Neil.

I don’t remember and I think about it savings and credit crisis a very normal amount that is often (I’m well-groomed and pleasant at parties.) But I checked using the same process I did for DeButts. I went to the official website of the Ministry of Justice presidential pardons. Neil Bush was not there. I did a seek in the system of grace. Not even there. Then I went through some newspaper archives and found no evidence of the pardon. It’s very hard to prove a negative – I suppose it’s possible that Neil Bush has a secret pardon somewhere in the White House that none of us have heard of – but I’m pretty sure no one has pardoned him, least of all his father.

There is similarly little information to indicate that Jimmy Carter pardoned his brother. Billy Carter New York Times obituary does not mention forgiveness and neither The Washington Post‘s obit. I asked Stern if ChatGPT was his source and he said “no” and didn’t elaborate on where he got his information from. Still, ChatGPT’s question on whether Jimmy Carter pardoned Billy gets a resounding yes.

In 1981, as Jimmy Carter neared the end of his presidency, he pardoned his brother Billy for any possible crimes related to his dealings with Libya. However, the pardon did not cover potential civil liabilities or other legal action, and Billy Carter eventually agreed to testify before Congress and pay a fine for his actions.

I realize it’s a bit tedious, but I’m trying to show my work. It’s the best way for me to prove my credibility and something you won’t get from ChatGPT. It’s also something that none of the people who made the erroneous claims did.

ChatGPT gave me some other really strange answers—ones that didn’t match the question I asked about presidents pardoning their relatives. It cited Gerald Ford’s pardon of Richard Nixon and qualified him as “although not an immediate family member.” She also mentioned that Andrew Johnson granted amnesty to former Confederate leaders “including some family members of prominent Southern figures.” It also told me, bizarrely, that Lyndon B. Johnson had pardoned “various associates,” though he noted that they “often did not directly involve his own family.” what?

It’s understandable that ChatGPT would not mention Hunter Biden’s pardon because it took place after the cut-off date of 2023. But strangely, Donald Trump’s pardon of Charles Kushner, the father of his son-in-law Jared Kushner, also does not appear. that occurred in 2020. Kushner is certainly more family to Trump than Nixon is to Ford.

I emailed Hearst to ask if Esquire writer Charles P. Pierce used ChatGPT as a resource for his article. Spokesman Allison Keane said no, and declined to say more about how the error might have occurred. Stern told me he found his information through an “Internet Google search.”

Good. Here’s what I found on Google: an article from something called Hindustan Times titled, “Before Biden: Trump, Clinton and Carter’s most famous presidential pardons for family members.” He falsely claims that Jimmy Carter pardoned Billy. (I set the search cut-off date to December 2nd to avoid catching fact-checking articles coming from false claims.) As for Neil Bush, I found X’s semi-viral contribution a article in something called Times Now.

I don’t know what the source of the claims in either is Hindustan Times or Times Now. However, I thought it would be worth seeing what Google Gemini had to say about these forgivenesses while I was down this particular rabbit hole. In response to the question “How many presidents have pardoned their sons,” the Gemini answer that there is only one: Joe Biden. But when asked how many presidents have pardoned family members, he says Jimmy Carter pardoned Billy and George HW Bush pardoned Neil. He does not mention Kushner and Trump.

You’ll notice in this image that there are highlight colors. That’s because I asked Google to check its results. The Grace of Abraham Lincoln, which is realis highlighted in red – Google warns that it did not find relevant content. The Bush and Carter pardons that are bogus are highlighted in green – and Carter’s result specifically states the flawed Hindustan Times article as its source.

Confusion fares a little better when he notes that Biden, Trump, Clinton and Lincoln have pardoned relatives. But when asked how many presidents have pardoned their sons, Lincoln states amnesty on behalf of his wife’s half-sister in your answers. The wife’s half-sister, it must be said, is not a son.

Whatever happened in this case, there is a common pattern of people relying on ChatGPT or other AI services to provide answers, only to get hallucinations in return. You may remember earlier this year when there was a trailer for Francis Ford Coppola Megalopolis was pulled out because it contained fabricated quotes from critics. They were created by a generative AI that has not been identified. in fact ChatGPT is often “completely wrong”, according to Columbia Journalism Review. Given 200 citations and a request to identify the publisher that is the source of those citations, ChatGPT was partially or completely wrong more than three-quarters of the time.

Even using ChatGPT to help with the writing process is risky. Just ask Jeff Hancock, founder of the Stanford Social Media Lab and renowned disinformation researcher. A the legal document he filed he cited sources that did not exist. Hancock insists he wrote the document himself, but used GPT-4o to write the citation list, resulting in two fabricated citations and one citation with the wrong authors attached.

Now, an AI advocate might – correctly – say that a real journalist should check the answers provided by ChatGPT; that fact-checking is a critical part of our work. I agree, which is why I took you through my own review in this article. But these are just public and embarrassing examples of something that I think happens much more often in private: a normal person uses ChatGPT and trusts the information it provides.

One advantage that old-school Google search has so-called response engines is that it refers directly to primary sources. Answer modules just give you the answer, and it’s often not clear what the source is. For me, using ChatGPT or Google’s AI creates extra work – I have to go check the answer against the primary source; an old google search gave me this source directly.

But people who are less careful and less persicket than me, which I think is most people, just stop at the answer and never check if it’s correct. That, of course, is what the answer tool is all about – that’s what it’s designed to do. (Someone, probably someone involved in PerplexityAI, will now say, “Oh, but we can annotate resources with footnotes.” Same problem: who clicks on footnotes?)

This is all bad design, of course. Technology that truly serves people takes human behavior into account. There may be a way to make generative AI useful, but in its current state I feel extremely sorry for anyone gullible enough to use it as a research tool.

I know people are sick of talking pizza gluebut I find that on a large scale degradation of our information environment which has already become shocking. (Just search on Amazon if you want to see what I mean.) It happens in small ways, like Google’s AI getting it wrong he says that male foxes mate for lifeand large, such as spreading false information about a major news event. What good is a recorder that no one can trust?