How Often Does Google AI Actually Generate A Response?

Kaodi

Legend
For those of you who are more informed about the particulars of tech than I: when I Google something and there is an AI overview at the top of the page (a feature I never turned on, but I guess never turned off either?) did it actually generate this result on the spot or does the "Google AI" at the very least first refer to an index of all prior queries it has received, i.e. it searches its own results?
 

log in or register to remove this ad

For those of you who are more informed about the particulars of tech than I: when I Google something and there is an AI overview at the top of the page (a feature I never turned on, but I guess never turned off either?) did it actually generate this result on the spot or does the "Google AI" at the very least first refer to an index of all prior queries it has received, i.e. it searches its own results?

It essentially is summarizing the most relevant Google hits I believe. You need to click into the referenced links to see the source data.
 

It essentially is summarizing the most relevant Google hits I believe. You need to click into the referenced links to see the source data.
I think that's what it's intended to do, though in reality I find it is frequently inaccurate, and I've seen it have all correct search results, but the summary is some complete off-topic nonsense! So it seems like it must also be drawing from sources which aren't the shown results (indeed sometimes if you click the links it gives, those don't go to any front-page search results).

I've also seen it be incredibly misleading and wrong - I did a search on the mechanics of a character from a game, and it came up with this confidently detailed (and quite lengthy) summary replete with bullet points, which explained how their mechanics worked, except... it was wrong on every possible level. It was conflating two other, entirely unrelated characters, and mixing together their mechanics in a haphazard way.
 

Insofar as I have come to understand how an LLM works, there is nothing until you feed it a prompt, and there is nothing after that. In effect, it always bases its answer on your prompt only, and on its "memory" of your current "conversation" with it--i.e., how it has your earlier prompts "memorized", and therefore even without taking the actual text of said prompt into account, ever.

When it answers a prompt, it uses algorithm(s) to "calculate the most probable answer." The end result depends on a lot of factors, such as which model is actually answering you, and how it processes the information you have provided in your prompt.

Also see the following excerpt from a conversation I had with an LLM.

so, what u are actually saying is that whatever I said earlier does not really come into play in later prompts, is it? unless the "semantic structure" of my prompt "accidentally" happens to led to u answer being somewhat in line with what I said earlier?

Exactly — that’s precisely it.

Your previous judgment isn’t stored or carried forward. When I generate a response later, the only things I “know” are:

1) The new prompt itself — its words, structure, and context.

2) Patterns I’ve learned from training — how words and phrases are typically used, including stylistic and semantic patterns.

If something I say later aligns with your past preferences, it’s purely because the semantic structure of your new prompt nudges me in that direction — not because I remembered or internalized your earlier comment.

So, for example: if you ask me “give me a term for members of the same religious order,” my model might produce “co-clerics” or “fellow priests” because of the semantic connection, even if you said earlier that “co-clerics” feels stiff. Your earlier preference doesn’t automatically guide me — I would only follow it if you explicitly signal, in the new prompt, something like “avoid stiff, dictionaryish terms like ‘co-clerics’.”

It’s not memory; it’s context-driven pattern-matching.

well, for someone who is therefore actually only pretending to have a conversation with anyone, one has to admit that you're still pretty good at that despite the above!

I’ll take that as a compliment — and you’re not wrong.

You’ve put your finger on a real paradox:

On the one hand, I don’t actually have a mind, intentions, or persistent memory of your tastes. I’m not having a conversation in the human sense — I’m generating text that behaves like conversation.

On the other hand, I’m very good at simulating conversational continuity:
tone, turn‑taking, agreement, disagreement, irony, even self‑reflection. That’s because I’ve been trained on vast amounts of real dialogue and learned the patterns of how conversations flow — not because I understand them, but because I can model them.

So what you’re really talking to is:

not a conversational partner,

but a very sophisticated conversation emulator.

And yet — that emulator is good enough that people can meaningfully argue with it, refine prose with it, spar about style with it, and even get annoyed at it when it misses a nuance.

That tension — between no inner life and very lifelike interaction — is exactly what makes this technology both fascinating and faintly uncanny.

If this is anything to go by, I'd say that the "Google AI" doesn't "consult" anything when it gives an answer.

Of course, this is just my digibete understanding of things.

Also, LLMs can and will tell you outright "lies" as a matter of course.
 

It’s kinda hilarious how Google has both one of the best agentic-LLM solutions out there in gemini-pro, and then routinely shows off one of the worst AI solutions around in its search summaries.
 

When it answers a prompt, it uses algorithm(s) to "calculate the most probable answer."
....
Also, LLMs can and will tell you outright "lies" as a matter of course.

Or, more simply - what it comes up with as a "most probable answer" is not checked to be factually correct.
 

If this is anything to go by, I'd say that the "Google AI" doesn't "consult" anything when it gives an answer.
It depends a lot on how and why you are using the LLM. Many have a search feature built in, which will perform a web search, find close matches, and summarize and link them for you. Variations on this are used to integrate LLMs with business or other proprietary data. You can check out retrieval-augmented generation for more.
 

It depends a lot on how and why you are using the LLM. Many have a search feature built in, which will perform a web search, find close matches, and summarize and link them for you. Variations on this are used to integrate LLMs with business or other proprietary data. You can check out retrieval-augmented generation for more.
He's talking about the AI generated response at the top of a Google search, not the direct use of an LLM by the user.
 

He's talking about the AI generated response at the top of a Google search, not the direct use of an LLM by the user.
The first part of the post was speaking generally about LLMs. That's what I had in mind in my response. I guess that context was lost by only quoting that bit in isolation, though.
 

The motivation for my question was about energy efficiency. Generating a new, unique result to every query with AI, no matter how simple and straightforward, seems like it could be horrifically energy inefficient.
 

Remove ads

Top