The video looks like it's making this whole thing out to be a joke. Everything about it conveys a lack of seriousness. The dancing cartoon character, the ascii art, the music, the whiz-bang visual effects that obscure anything interesting or useful.
The author is making the same mistake that they're claiming other news outlets have made. They're placing too much responsibility on the AI chatbot rather than the end-user.
The problem that needs correcting is educating the end-user. That's where the fix needs to happen. Yet again people are using a new technology and assuming that everything it provides is correct. Just because it's in a book, or on TV or the radio, doesn't mean that it's true or accurate. Just because you read something on the Internet doesn't mean it's true. Likewise, just because an AI chatbot said something doesn't mean it's true.
It's unfortunate that the young man mentioned in the article found a way to reinforce his delusions with AI. He just as easily could've found that reinforement in a book, a youtube video, or a song whose lyrics he thought were speaking directly to him and commanding him to do something.
These tools aren't perfect. Should AI provide more accurate output? Of course. We're in the early days of AI and over time these tools will converge towards correctness. There should also be more prominent warnings that the AI output may not be accurate. Like another poster said, the AI mathematically assembles sentences. It's up to the end-user to figure out if the result makes sense, integrate it with other information and assess it for accuracy.
Sentences such as "Tech companies have every incentive to encourage this confusion" only serve to reinforce the idea that end-users shouldn't need to think and everything should be handed to us perfect and without fault. I've never seen anyone involved with AI make that claim, yet people write article after article bashing on AI companies as if we were promised a tool without fault. It's getting tiresome.
Yes, although resources might be a better word than tools in that case. If I'm at the library and I'm asking the librarian to help me locate some information, they are definitely an educated resource that I'm using. The same for interacting with any other person who is an expert whose opinion or advice I'm seeking.
Those experts will generally ask clarifying questions if they don't understand what you're asking, rather than spin you in circles. The reason they're considered experts in the first place is they understand the topic they're sharing information on better than you do. It's not the end users fault that the LLM is spewing nonsense in a way that can be mistaken for human-like.
This is why I dislike the word "hallucination" when AI outputs something strange. It anthropomorphizes the program. It's not a hallucination. It's an error.
It's not an error though. From is training it's outputting things most likely to come next. Saying it's an error means that being accurate is a feature and a bug that can be fixed.
It's of course not actually hallucinating. That's just the term that's been chosen to describe what's going on
Being accurate is a feature and it is a bug that can be fixed though.
Given various models, one that always produces statements that are false and another that only sometimes produces false statements, the latter model is preferable and the model which most people intend to use, hence the degree to which a model produces correct statements is absolutely a feature.
And yes, it's absolutely possible to systematically produce models that make fewer and fewer incorrect statements.
It's nice that you feel that way, but reality is at odds with your sentiment. Even if the LLM is trained on completely 100% factual human-checked data, its mechanism is still predicting the next word, and what it is not is a mechanism designed to return only factual data. There is no such thing as an infallible LLM, no matter the model or how it was trained.
Sure, some may return results that are sometimes more true than others, but a broken clock is also right twice a day. The more broken clocks you have, the more chance there is that one of them is correct.
No, the user you replied to is correct. Accuracy is indeed a feature, and can be incrementally improved. "Predicting the next word" is indeed a mechanism that can be improved to return increasingly accurate results.
Infallibility is not a feature of any system that operates in the real world. You're arguing against a strawman.
It's nice that you feel that having one LLM that generates entirely incorrect statements is equally as functional as an LLM that does not, but reality in terms of what LLMs people will actually use in real life and not for the sake of being pedantic over an Internet argument is very much at odds with your sentiment.
How a product happens to currently be implemented using current machine learning techniques is not the same as the set of features that such a product offers and it's absolutely the case that actual researches in this field, those who are not quibbling on the Internet, do take this issue very seriously and devote a great deal of effort towards improving it because they actually care to implement possible solutions.
The feature set, what the product is intended to do based on the motivations of both those who created it and those who consume it, is a broader design/specification goal, independent of how it's technically built.
>LLM that generates entirely incorrect statements is equally as functional as an LLM that does not
And yet they would both be operating within the normal design parameters, even the supposed "LLM that does not" when it spits out nonsense every so often.
Your current zeitgeist is not much better than a broken clock, and that is the reality many people are witnessing. Whether or not they care if they are being fed wrong information is a whole other story entirely.
Like cubic splines, the data will be on the line. Everything in-between the points may or may not be true. But it defiantly conforms to the formula.
Wonder if it would be possible to quantify margin of error between different nodes in these models. But even what is 'in between' still conforms to the formula. But not necessarily what it should be. A simple 2 node model should be 'easy' to quantify but these models with thousands of nodes what does it mean to be +/- x percent from the norm. Is it a simple sum or something else to quantify it.
> 5. Mathematics The difference between a computed or measured value and a true or theoretically correct value.
^ this is the definition that applies. There is a ground truth (the output the user expects to receive) and model output. The difference between model output and ground truth ==> error.
--
> From is training it's outputting things most likely to come next
Just because a model has gone through training, does not mean the model won't produce erroneous/undesirable/incorrect test-time outputs.
--
> Saying it's an error means that being accurate is a feature and a bug that can be fixed.
Machine learning doesn't revolve around boolean "bug" / "not bug". It is a different ballgame. The types of test-time errors are sometimes just as important as the quantity of errors. Two of the simpler metrics for test-time evaluation of natural language models (note: not specifically LLMs) are WER (Word Error Rate) and CER (Character Error Rate). A model with a 3% CER isn't particularly helpful when the WER is 89%. There are still "errors". They're just not something that can be fixed like normal software "errors".
It is generally accepted some errors will occur in the world of machine learning.
I don't agree that that's the right definition to use though. LLMs do not output computed or measured values.
If I expect Windows to add $5 to my bank account every time I click the Start button, that's not an error with Windows, it's a problem with my expectations. It's not a thing that's actually made to do that. The start button does what it's supposed to (perhaps a bad example, because the windows 11 start menu is rubbish), not my imagined desired behavior.
LLMs output a vector of softmax probabilities for each step in the output sequence (the probability distribution). Each element in the vector maps to a specific word for that sequence step. What you see as a "word" in LLM output is "vector position with 'best' probability in softmax probability distribution".
And that is most definitely a computed value. Just because you don't see it, doesn't mean it's not there.
The real danger of the word "hallucination" is it implies that the model knows what's real and erroneously produced a result that is not. All LLM output is basically an interpolation, most people just aren't used to thinking of "words" as something that can be the result of interpolation.
Imagine The real high temperature for 3 days was: 80F on Monday, 100F on Tuesday, 60F on Wednesday. But if I'm missing Tuesday, a model might interpolate based on Monday and Wednesday that it was 70F. This would be very wrong, but it would be pretty silly to say that my basic model was "hallucinating". Rather we would correctly conclude that either the model doesn't have enough information or lacks the capacity to correctly solve the problem (or both).
LLMs "hallucinations" are caused by the same thing: either the model lacks the necessary information, or the model simply can't correctly interpolate all the time (this possibility I suspect is the marketing reason why people stick to 'hallucinate', because it implies its a temporary problem not a fundamental limitation). This is also why tweaking prompts should not be used as an approach to fixing "hallucinations" because one is just jittering the input a bit until the model gets it "right".
That's the exact opposite of what the term "hallucination" is intended to imply. If it knew what was real and produced the wrong result anyway that would be a lie, not a hallucination.
I've heard the term "confabulation" as potentially more accurate than "hallucination", but it never really caught on.
"Confabulation" would never catch on because it's a word that most people don't know and couldn't remember. "Hallucination" is easier to remember, easier to understand, and easier for laypersons to build a mental model of.
Error isn't exactly correct, either. Barring some kind of weird hardware failure, the LLM generally does the text completions correctly. The word "error" only comes into play when that LLM output is used as part of a larger system.
The program is producing correct output, for technical values of correct.
The LLM is a statistical model that predicts what words should come next based on current context and its training data. It succeeds at that very well. It is not a piece of software designed to report the objective truth, or indeed any truth whatsoever.
If the LLM was producing nonsense sentences, like "I can't do cats potato Graham underscore" then yes, that's "incorrect output". Instead, it's correctly putting sentences together based on its predictions and models, but it doesn't know what those sentences mean, what they're for, why it's saying them, if they're true, what "truth" is in the first place, and so on.
So to say that these LLMs are producing "incorrect output" misses the key thing that the general public also misses, which is that they are built to respond to prompts and not to respond to prompts correctly or in a useful or reasonable manner. These are not knowledge models, and they are not intended to give you correct sentences.
I hope you cut that client lose. If they'll do something like that on the POC they'll do it on the big contract and in numerous other ways. You'll have to double-check everything with a client like that and not be able to trust them. Life's too short to put up with that.
I feel that we've done ourselves a disservice by referring to them as ad blockers rather than content blockers or, better yet, virus blockers. With malicious ads being the primary vector for compromising a person's computer, calling these tools virus blockers would not only be more accurate, but it would make it harder for sites to counter their use from a public relations point of view.
Could you imagine visiting a web site only to have it tell you to disable your virus blocker to view the web page? Yet, web sites make the same requests of your "ad blocker" even though the end result is the same. They want you to disable your security protections and risk infection to view a web page.[1]
Sadly, I believe we've already lost this battle and are worse off for it.
Ads in general should be considered malware. They have adware and spyware components and both types of software used to be detected and removed by anti-malware software. Apparently a few lines in the terms of service nobody reads is all it takes to legitimize this abusive software. Because users have "agreed" to be subjected to malware, their use of content blockers somehow becomes immoral.
> With malicious ads being the primary vector for compromising a person's computer, calling these tools virus blockers would not only be more accurate, but it would make it harder for sites to counter their use from a public relations point of view.
I don't understand this. How are malicious ads the primary way to compromise a computer?
It depends on the ad. In the before time, long ago, ads were a simple static image or body of text. And in that case, while not impossible, it's rather hard to compromise a computer with a static image or block of text. You'd have to have a fairly specific image crafted to adversarially target a specific bug in the rendering of images to get code execution, and thus compromise a computer.
It is no longer the before time.
In the now time, ads frequently contain not just text or images, but javascript as well. And already having code execution by virtue of javacript, it is a lot easier to escalate the privileges of that code execution from the limited environment of the browser to installing code on the computer running that browser. Want to deploy your bot? Buy an ad that includes your malicious javascript payload. Now, anyone who goes to a site and views your ad will execute your javascript for free in addition to your offer to sign them up for credit score monitoring.
Every single ad seller's been caught by companies selling these dangerous ads. Google, yahoo, etc.
They usually catch it within 3-5 days. That's too long.
In short, yes. Advertisements are a huge way to get malware onto people’s computers. This just one random article from a google search for “malware advertising” but there are tons and it still happens today.
What is in the interest of the ad networks is income and plausible deniability.
“Sorry we didn’t realise that there was malicious content in that ad, we’ll do better in the future, and we’ll also work on taking less than four days to remove malicious content.”
In the meantime the reason ads are being bought is the ability to deploy customer enrichment experiences (the new name for malware) which will help customers better discover your product (by redirecting their browser, or rewriting links).
I have seen ads trigger site redirects, app download redirect to app stores etc. and even initiate apk(Android app package) file downloads automatically.
That metric may mean in an indirect sense as well as a direct one - i.e. ads are the most common first step in a pathway that begins with getting a user to take some action that eventually leads them toward doing something that ends in compromise, even if it's several (mis)steps later.
Even if you have a zero-day exploit for a browser, you still need to get that exploit to your victims. Ads are the easiest way to get custom media and javascript into as many browsers as possible. Ptherwise, you need to tempt users to your malicous site or find an exploit in a widely used site to get your payload to users (since most sites do not allow users to post arbitrary HTML, CSS or javascript).
Ad networks are supposed to vet ads to make sure they are safe, but they're bad at it and the system is not set up to make it easy (ads are dynamically generated by whoever's buying them as the page loads).
Because most ads are served by a handful of ad networks (Google Ads, etc) you have a single entity through which you can get your malicious ad spread across a wide range of websites, increasing its yield. Without the ad networks you'd have to negotiate with every single site one by one to get your ad displayed, and some sites might do more due diligence and refuse to serve your ad because of its maliciousness.
I think it's better to just be open about it. I don't like ads so I block them with an ad-blocker. There's nothing wrong with that. I don't have a moral obligation to waste my time watching things that don't interest me or being manipulated in a way that is detrimental to my psychological and material well-being.
When I'm feeling tetchy I call them malware blockers. RATs, browser toolbars, adware, etc. might not technically be viruses, but they sure as heck are being used as malware.
> The ZFS on Linux project gets around this restriction by running ZFS in user space
ZFS on Linux is a kernel module. You may be thinking of ZFS-FUSE which runs in user space using FUSE, but I'm not sure if it's being maintained any more.
Yes, it's normal. Just people scanning for SSH ports to attack.
What I did was take the IANA IPv4 address space list[1] and add firewall rules to drop packets to port 22 coming from /8's that I would never connect from. I'm in the US, so it made sense for me to drop connections from subnets assigned to RIPE, APNIC, etc. It didn't eliminate the problem, but it reduced the connection attempts considerably.
reply