A lot of the annoying AI writing is actually good writing patterns done far too often. Rule of three is something every writer should use but not three times in one paragraph, etc.
the problem in my opinion comes down to a sort of poor "theory of mind" on the part of agents at our current moment.
The AI is really bad at guessing what its reader already will know. Not just before reading, but also from reading the text. This means that it so often explains things that don't need explaining, and emphasizes things that don't need emphasis.
It's not Y but X. except no sane person would ever think it was Y, and it was clear to everyone it is X already (or X is not a widely understandable thing in the first place, or the AI hasn't established a satisfying reason for why this distinction isn't important, or or or).
This is, I think, due to a combination of training on human feedback rewarding surface level good text (people marking preference on their 300th answer of the day are not trying to understand whatever point the agent is making they're just seeing whether it seems like it knows what its talking about) combined with later training by automated grading systems (other llms) that always have perfect context - if I also went and read all the code the agent referenced, read all of its 12 paragraph answers in gave at every turn of this back and forth conversation, knew all the files on the hard drive and had read all the diffs it made, then yeah it would be fine to assume some of the things it does, but I don't do that when im talking with my boss, because he asked me to do it so he doesn't have to understand everything, and he expects me to surface important stuff and minimise other stuff in a sensible way
I am on a LitRPG site (RoyalRoad) and there are routinely AI Gen novels in the top 5 novels on Rising Stars on the site which is basically traffic and rating metrics for newish novels. People say they hate AI writing and downvote it when they know it’s AI but seem to read it just fine without noticing fairly often as well.
AI-written stories are definitely extremely common in low-brow fiction, but it's notable that they're not restricted to it. There's an ongoing scandal with an award-winning French novel C'était ça ou mourir which seems to be entirely AI-written. I fear the lesson here is that the AIs have actually correctly learned that the average person loves this style, and only starts disliking it once it becomes repetitive.
And there are people who exclusively read AI generated fiction. These people mostly weren’t reading a lot before. AI in a lot of cases isn’t replacing existing fiction, it’s offering a different product.
Genres where ghost writing is a already a big part of it and where authors routinely publish multiple books a year might end up incorporating AI more but right now you’d have to lie to readers to do it.
I think this mostly comes down to what work is productive for a person. People who are fascinated by research topics are likely to work on them and people who are enamoured with creating art will do so. But people who's productivity was largely being some cog in some machine will be all too happy to say good riddance.
It seems to me that recent changes have dramatically exacerbated this problem. Pangram is still claiming their extremely low false positive rate from before adding humanization detection and now very frequently detects actual human works as humanized AI.
There seems to be a huge backlash to existing works where AI was detected already at least in my art (creative writing). So I think we already have evidence consumers care. I don't think blockchaining is the solution for the general public, but I'd probably do exactly that if I was taking a university course these days since they tend to be hyper sensitive about AI and err on the side of the accuser so I want strong evidence my writing is mine. Which of course is ridiculously insufferable because I used to do a good portion of my homework in paper notebooks on the subway without stable internet.
LLMs still write middling quality prose. Aspects of it are good but they write in a very tropey manufactured style that isn't something we are fatigued of, it's something that's inherently flawed and disliked.
A lot of writing publications are so overwhelmed by submissions that they run an AI tool on everything and don't even read the ones that fail (for whatever threshold they think is best). Plus look at the recent backlash against the book where the author according to detectors used about three quarters AI gen. If you do your art for anyone but yourself you care a great deal about your work showing as human and being treated as having soul rather than testing as machine and being treated as witch hunt grade slop.
Pangram’s market claims are very different from independent validations of the platform. The 99.98% rate is on a toy dataset not resembling reality.
Even if 2/10000 is true that’s nowhere near accurate enough to make aggressive accusations that create anxiety at levels people need to medicate with potentially fatal consequences.
the text in above is all ai generated. i uploaded a picture of two kittens to gemini and asked it to guess their age. then asked gemini to change the language to of someone who is born in deep Appalachia but has Japanese parents. then asked it to change the bullets to paragraphs written by someone who is talking fro life experience rather than grounded in published science. pangram gave it 100% human generated. go ahead, try it. it boggles me how people believe these ai detectors.
if it makes you feel good the original response from gemini gets 100% AI detection and for that I think pangram should get the credit but in practical terms the empty pots in my yard are more useful than pangram at this point.
You are talking about false negatives, but this thread started with a discussion around the potential for false positives causing writers anxiety
> Even if 2/10000 is true that’s nowhere near accurate enough to make aggressive accusations that create anxiety at levels people need to medicate with potentially fatal consequences.
False positives and false negatives are different problems that have different impacts.
Does your single example prove that Pangram doesn't work, or that it doesn't work on short snippets of text? Try to get a detection error on a longer run of text. I'd be curious to see the results, particularly if you can get a false positive.
Well, false positives must anyway be zero for a product which is built around detecting AI. let me explain - the value proposition of this product is that we detect AI. Its not we don't call human writing AI generated. The latter is implied and must not happen ever. The value is that pangram detects AI generated content which it did not in the above example. if this explanation doesn't make sense then think of pangram as a pregnancy test where presence of AI == presence of baby. If a pregnancy test keeps saying you are not pregnant when you actually are then that's a problem, right? - you can't argue that at least its not saying you are pregnant when you are not.
But here's a bigger problem - Imagine a teacher grading 50 students - 40 of them use AI to write their answers and 10 write honestly. All 40 of them use the hack that I used and get 100% human from pangram and the other 10 also get 100 human from pangram. what is the product really adding to the workflow? Nothing, zero or zilch!!! - but then you come along and say hey at least those 10 honest students also got 100% from pangram. This whole argument of false positives being very very low works if your false negatives are tight which they are not.
