Because things that sound good sometimes are indeed good. So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess. The final evaluation is if it is cheaper to dig the info ourselves or we can use the AI to narrow down the search space. I think the second, but only if people do the verification part.
First of all, no. Everyone using ChatGPT does not know that verification is necessary. I'm sure you know, and so do many other people. But there's also a lot of people who clearly do not understand this. Including some of the people developing products based on this stuff. And that's a problem.
Second, the fundamental issue here is that as long as verification is needed, this technology is only ever useful for domains in which you're already an expert. That severely limits the technology, and yet this keeps being touted as some kind of general knowledge base when clearly it's at best a very narrow one.
And it seems to me like this is unlikely to be solved in the current paradigm. This is just intrinsic to the architecture of LLMs. There is no getting away from it.
That's fascinating to me, since the moment you sign up to use it, it blatantly states that it is often incorrect.
And below every chat box there is a warning that says:
```ChatGPT can make mistakes. Consider checking important information.```
So I guess I am overestimating the average user by assuming that people read the text written on every single page they use to query the system.
Even so, this entire comments section reads like a giant saltmine of people that are terrified of embracing tech that in every sense of the word increases turnaround time on productivity.
The ability to literally feed a screenshot into it and ask questions regarding it, with insane results is astounding.
I have input a picture of baby rabbits and asked it to rate my chickens, and got a detailed explanation of how the picture contains baby rabbits and not chickens.
There are things the system is great at, and others it sucks at currently.
But you are all mistaken if you think its going to stay this bad.
If you haven't even bothered to check how quickly the field has moved from single-modal to multi-modal, well, damn guys, I am sorry, but you are getting automated first.
The biggest problem with current models is that they don't know how to verify their own answers, and there's no way to allow them to without putting a ministry of truth on it (similar to what twitter was during covid).
To combat this, we need to figure out how to let the model generalize truth, which is something it can only do if fed enough (unfiltered, unbiased) data.
With enough data, it will find the patterns underlying false information and eventually figure out how to use it.
In fact, most likely, the future models will simply be fed unlabeled data, in any format (audio, video, telemetry, analytics) and generalize from that.
The concept of feeding normalized data to a model for training is archaic and stupid.
The world is not normalized.
Data is not always the same, and will not always even be guaranteed to be present.
Anyways, what does it matter.
Why am I trying to sell sugar to people addicted to salt?
> So I guess I am overestimating the average user by assuming that people read the text written on every single page they use to query the system.
Yes, you definitely are. Speaking from experience of supporting users for many years. People will not only ignore such text, they will repeatedly make mistakes or ask questions the text answers directly, even after you’ve explained it five different ways.
The rest of your post is full of assumptions and dismissals, so I think I may be wasting my time, but let’s please stop with the rhetoric that someone who doesn’t agree with you is somehow afraid or has an agenda. As way of example, your comment ignores the bigger picture of bad actors using LLMs to push propaganda, sow discord, or scam people. The “bullshit” part of the problem isn’t limited to “it gave me a wrong answer on my homework”.
Students have gone under academic investigation because their professor asked GPT "was this paper written by AI?" and pasted in their submitted assignment.
> So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess.
Why is verification part of the process if not because ChatGPT generates bullshit? I don’t think the author is disregarding that it is sometimes correct.
Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet? The difference is that the ai gives answers faster and with much vaguer prompts than a handcoded system is usually able to.
> Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet?
What authority does random information on the internet have? How do I verify the information without looking at random information on the internet?
> while everyone using ChatGPT knows that verification is part of the proccess.
No, everyone very much does not know that. Including the lawyers who though it made sense to ask ChatGPT questions, than asking it if the answers were correct.
