AI Overview is still 'yes, and'-ing completely made up idioms despite Google's best efforts to restr

Anyone studying a second language will tell you that learning idioms is often a stumbling block. I mean, just take my mother tongue, British English—'raining cats and dogs', 'on it like a car bonnet ', 'Bob's your uncle, Deborah's your aunt'—I mean, what bizarre fairytale creature would even talk like this?

Because idioms spring up from rich etymological [[link]] contexts, AI has a snowball's chance in Hell of making heads or tails of them. Okay, I'll stop over-egging the pudding and dispense with the British gibberish for now. The point is, it's a lot of fun to make up idioms and watch Google's AI overview try its hardest to tell you what it means (via ).

We've had a lot of fun with this on the hardware team. For [[link]] instance, asking Google's AI overview to decipher the nonsense idiom 'Never cook a processor next to your GPU' returns at least one valiant attempt at making sense via an explanation of hardware bottlenecking.

When our Andy asked the AI overview, it returned, "The saying [...] is a humorous way of highlighting the importance of not having a CPU [...] and GPU [...] that are poorly matched, especially for gaming. It implies that if you try to run a game where the CPU is weak and the GPU is powerful, or vice versa, you'll end up with a frustrating experience because the weaker component will limit the performance of the other."

However, when I asked just now, it said, "The saying [...] is a humorous way of suggesting that you should never attempt to repair a faulty GPU by heating it up in an oven, as this can cause more damage than it fixes. This practice, sometimes referred to as the "oven trick," has been discredited due to its potential to melt solder joints and cause further issues."

Alright, fess up: who told the AI about the 'oven trick'? I know some have sworn by it for older, busted GPUs, but I can only strongly advise against it—for the sake of your home if not your warranty.

Image 1 of 2

Because a Large Language Model is only ever trying to predict the word that's most likely to come next, it parses any and all information uncritically. For this reason—and their tendency to return different information to the same prompt as demonstrated above—LLM-based AI tends not to be reliable or, one might argue, even particularly useful as a referencing tool.

For one recent example, turned to three different for a solution to his storage woes. It took two days and nearly 300 different prompts for even one of the AI chatbots to spit out something helpful.

Google's AI Overview is almost never going to turn around and tell you 'no, you've just made that up'—except I've stumbled upon a turn of phrase that's obviously made someone overseeing this AI's output think twice. I asked Google the meaning of the phrase, 'Never send an AI to do a human's job,' and was promptly told that AI Overview was simply "not available for this search."

Image 1 of 4

Our Dave, on the other hand, got an explanation that cites Agent Smith from The Matrix, which I'm not going to read too deeply into here. At any rate, there are than you may have been led to believe, and I'm seeing those fingerprints on Google's AI Overview refusing to play ball with me.

Indeed, that it has been attempting to clamp down on "nonsensical queries that shouldn’t show an AI Overview" and "the use of user-generated content in responses that could offer misleading advice."

Undeterred, I changed the language of my own search prompt to be specifically gendered and got told by the AI Overview that a 'man's job' specifically "refers to a task that requires specific knowledge, skills, or experience, often beyond the capabilities of someone less experienced." Right, what about a 'woman's job', then? Google's AI overview refused to comment.