for the low end Qwen 27b is pretty close to Luna other than being slower and you can run that on commodity hardware. I have an overly quantized bumbling idiot version of it running on 20gb of GPUs.
Google could use the LLM to find the best hit in the SRPs and put those before the synopsis. It would take more context but if it has an idea of what the answer is it could review the text content of the top hits and check them against what it think the user is looking for.
Of course more often than not the reference links within LLM responses don't really relate to what they're saying - LLMs act like highschoolers, "I need X references to back up my essay, so I'll find some things that sound relevant and put them in as footnotes".
Fair point, but still: anything that messes up other users is generally identified as "don't do this outside of personal stuff nobody else has touched, once others have pulled it it's forever."
For example all the history rewriting has that rule -- "don't force push after somebody has pulled that branch".
A lot of things that should be one bash command turn out to be dozens of lines of incantations and handling edge cases. Microsoft I'm looking in your direction.
reply