Hacker Newsnew | past | comments | ask | show | jobs | submit | mixedmath's commentslogin

The second response reads as very much written by AI to me too.

One particular sentence reads particularly human to me:

> It is rare to get feedback that tells us exactly what is broken and how to fix it. AI doesn't remember from one conversation to another, so probably can't say how rare something like this is at Ocado (although admittedly it could learn that it's generally rare in fine tuning). I also expect AI to be trained not to compare feedback with other people's feedback.


It doesn't have to remember. It just repeats what humans say. And humans say that sort of thing every day.

Nope, that’s classic GPT slop.

“You’re absolutely right, and honestly, that’s rare. Nobody else would have thought of taking a bath with their toaster, but you did. Would you like me to write you a twelve-week toaster making toast plan?”


> There is nothing more frustrating than pointing out a legitimate bug only to feel like you're talking to a brick wall.

At a bare minimum, this sentence was AI inspired. Likely generated. Which certainly casts the whole response into doubt (that it is human-written.) Reads more to me like a person gave a quick quippy response to an AI to clean up, then just took the output and replied.


And also to pangram 4.1, which rates it as 100% AI Generated.

I only saw pangram being mentioned on HN. How good is it in general ? And what do they use to check if a text is AI Generated, the text watermarks from big models ?

it is very good, but keep in mind it'll also flag text edited with grammarly. Maybe not the same as copy and dumping a claude response, but still normalizes the authors voice with AI.

Grammarly does use LLMs for a while.

In my experience, it can have significant false positive. It's pretty good at telling you when something is NOT written by an LLM, but when it says something is, I take it with a big grain of salt.

I hadn't heard of it before, just now looking them up.

Dumb question, but could you pair AI with pangram to make it sound less like AI or would that be a recursive loop? I'm skeptical about it


That's how the earlier image generation models worked. You had one ai to generate images, and one to understand if it's a fake generated image or a real one (like, you have a real image of a dog, use the ai to generate an image of a dog, and pass both to the ai trying to see if it knows which is which). They both start out bad, but evolve together.

https://en.wikipedia.org/wiki/Generative_adversarial_network


Models are trained on ridiculous amounts of human-written text, but their outputs are pretty easy to recognize as not human written. Pangram explains some of the reasons for this in a blog post.[1]

The problem with using Pangram in a feedback loop is that Pangram only reports whether it thinks the text is written by AI or not. It doesn't report whether the text makes sense, or whether it contains errors in spelling or grammar. So when people try to pair AIs with Pangram, the quality of the output tends to degrade to the point where it's not useful.

Basically, it's a harder problem to get an AI to accomplish some goal (such as writing a blog post) while also having its output mimic human text. To use an analogy, it's like building a stealth aircraft with the same speed, maneuverability, range, and payload as a non-stealthy aircraft. Even with a much larger budget, it's not always feasible.

1. https://www.pangram.com/blog/no-llms-dont-just-mimic-human-t...


That's what adversarial model training is basically.

They trained a model to detect AI-written text.

It works fine on mainstream models where the output hasn't been edited.

It doesn't work on humanizers or custom model output.


The thing about this whole post is that AI-written text doesn't tell you a single thing about whether a human was involved.

A thing about tier 1 CS agents is that they are often not highly literate and great at writing, and companies go to great lengths to minimize how much text they need to freehand, with tons of pre-writes to address common situations. LLMs allow the agent to express some freeform idea and have the LLM clean it up to read clearly and professionally. So even if you are 100% dealing with a human, you should expect nearly all text from them to be LLM-written.


It does smell like AI. The one bit that gave me pause was "Thank you again, I do hope that this issue is resolved quickly." AIs don't usually do comma splices. Maybe it was AI with light human editing.

It could have been the other way around, with a human typing and something like Grammarly cleaning up the text.

I read it twice and the second time it stood out as not.

Usually there is a taste about AI generated messages that are just quite obvious, it's usually pretty close to corporate speak, but not exactly the same.


I think this also gets muddier and muddier because A. LLMs do get better at writing like a human, though slowly (some analysis posted here lately showed pretty clear distinctions in pacing, word-choice etc; not bothering to look it up right now) and B. human language is fluent and people start adopting AI speech!

There are so many levels to this: fully automated AI reply, human sending AI-written reply, human sending AI-written but hand-improved reply, human sending hand-written but AI-improved (or translated) reply, human who reads AI output all day hand-written reply, human who doesn't interact with AI (often) hand-written reply.

Some of these will give you an uncanny feeling, but you always keep guessing who you're actually talking to. In the internet nobody knows you're a dog.



To me the second response reads more like PR-speak, like somebody who's been over-coached on how to talk to customers.

I find it strange that they'd have an AI-driven followup email days after the survey. If you're automating it, why the delay? "Delay the response so it seems more human" is way too cynical.

It's worth noting that OP is talking about Ocado, too. I've been a client of theirs for over ten years, and the number one reason why I keep buying from them is that their delivery drivers never feel super-hurried. They have lower quotas than drivers for other services, and they always have time for a little bit of banter. Compare that e.g. to Amazon Fresh drivers, who'll always just leave the groceries at the door and knock. They're already gone by the time I open the door.

I find it unlikely that they'd automate this sort of second-level customer support response while not squeezing the everliving shit out of their drivers.


That is what I immediatelly detected too, but was in shock that the author fell for it.

Pretty sure the blog post meant to be ironic. But who knows.

To put salt in the wound, I wonder if the first AI ever even raised the issue with a support team. Pretty much every customer support AI has gaslit me saying it escalated, or please wait for a human to contact you, and then sitting there for an hour or more to confirm that, yes, that was just hallucinated in the prompt, and it's been waiting for my next prompt ever since.

The good thing about AI support is that absolutely flipping out in the chat actually does get escalated, unlike with a human who just digs in their heels.


Exactly! I absolutely don't mind AI generated responses as long as it is accompanied by an AI generated ticket/bug along with an AI generated fix, along with an AI generated regression/integration test and AI generated message saying it was fixed. An AI generated coupon/reward for the trouble is also fine.

If this is the workflow that a consumer experiences, I don't think majority will complain about AI.


This thread highlights why I often say that at some point in the future, the obsession with whether something was written by AI or not will be classified as a mental disorder.

Claude models seem to work far better in Anthropic harnesses. I don't know if this is because they are trained in that context or because they offer different models through external APIs.


Would someone be interested in explaining what in the world is going on in that article?


I'll try to do the first one since I think it's the most illustrative:

  1  long typedef[[]]*($)(void*($),...);  
  2  int main() {  
  3   char* str = "Hello, world.";  
  4   $ $ asm("puts");  
  5   $:&&$&&$(str);  
  6  }

(1) is equivalent to "typedef long* $(void*, ...)". '$' is the name of the defined function type.

it's obfuscated as follows:

- typedef is a specifier and can go on either side of the type, same as int const/const int

- [[ ]] is an empty list of attributes

- "void*($)" is an argument named '$' of type void*, but parameter names are ignored in declarations.

- you can put parens in declarations since sometimes you need to group prefixes and postfixes in a different order

- dollar signs in names are allowed in many c compilers (and some other c-like languages)

- "..." are variadic arguments. puts doesn't have those, but the first argument uses the same register either way so it happens to work out in this case

(4) declares a function named '$' of type '$', shadowing the type, and links it to the assembly label "puts". calling $ calls puts.

normally it looks like `int call_foo(void) asm("foo");`

(5) is parsed as: "label $: address-of-label $ and function $ called on str. the address of a label is truthy, so the function also gets called.

unary && looks for the argument in the label namespace, but other operators don't, so the two dollars refer to different things.

that's about it. if you're familiar with the c standard and some common gnu extentions, you can deconstruct the examples into their basic elements easily enough


What do you mean? It's just simple beginner's examples in C. Should be easy enough to understand, C is famously very simple ('portable assembler' as they say!).


> In our experience, C has proven to be a pleasant, expressive and versatile language for a wide variety of programs. It is easy to learn, and it wears well as one's experience with it grows.

The C Programming Language K&R


This is true in my case! I've had more people actually buy me a coffee after interacting with my site than to click on my Buy Me a Coffee button.


I find a link to "Buy me a Coffee" to not at all look like an ad. It's a funny-shaped "donate" button. And I much prefer it to actual ads.


Absolutely. And I've done it to quite a few places.

Like it's no secret it's expensive as shit to live lately. If you're just giving away your software/knowledge/ideas, and I feel I gain value from them, I'll happily throw some cash in your hat.


I wasn't aware that there were any public, commonly-used voting machines that we could check.


I was very surprised to see that the article explicitly says it will not consider this answer!


If Dave Arnold says he really doubts it, it's a pretty safe bet he's read like a dozen papers related to the prediction and is basing it on something. He's like the Bunnie Huang of cooking.


In the article the only explanation he gives is that it doesn't make sense to him, doesn't mention any papers or anything at all. But I'm pretty certain that he's wrong and it works. The difference in gas if I've been eating beans recently vs if I haven't eaten them in a couple of months is not just "I feel like maybe I get a little bit less gassy maybe" it's going from "two dozen farts at least, guaranteed" vs "one or two, if any at all" it's a night and day difference and if there even was a paper that says the contrary rather than change my mind I'd just assume that there must be something wrong with how the study was made. Of course, I'm just a sample of one and I haven't done any study either, so I don't mean to imply that there might not be other factors or that it may not work to the same degree for everybody, only that I'm pretty sure that dismissing it is wrong because I know at least one counter example.


More anecdata here. For various health reasons, about six months ago I started eating beans every day for lunch. At first it was… intense. But within a few weeks, the gas entirely disappeared. I now have no issues whatsoever.

FWIW, some beans are easier on my system than others. Kidney beans (sadly), my favorite, can be murder if you don’t cook them thoroughly enough. Lentils, on the other hand, seem to be pretty gentle. They also soak up whatever flavor they are cooked in so they are a great “starter bean” if you want to eat a more vegetarian diet.


Yeah, given the number of vegans / vegetarians I know who have gone through this I find it completely self obvious. If I went and ate meat today, I would almost certainly barf… But I wouldn’t spend ages trying to find a way to cook meat to make me not barf, I would just slowly introduce it back into my system little bits at a time until the adverse effects go away.


Are there leaderboards that you follow or trust?

Also, do you have preferred OCR models in your experience? I've had some success with dots.OCR, but I'm only beginning to need to work with OCR.


> Are there leaderboards that you follow or trust?

Not for OCR.

Regardless of how much some people complain about them, I really do appreciate the effort Artificial Analysis puts into consistently running standardized benchmarks for LLMs, rather than just aggregating unverified claims from the AI labs.

I don't think LMArena is that amazing at this point in time, but at least they provide error bars on the ELO and give models the same rank number when they're overlapping.

> Also, do you have preferred OCR models in your experience?

It's a subject I'm interested in, but I don't have enough experience to really put out strong opinions on specific models.


Here's one from last year. Helpful, but I guess they gave up on updating it.

https://getomni.ai/blog/ocr-benchmark


How does this relate to the mean entropy of 1 noted in the OP article?


Having ripcorded out after realizing the author was trying to prove that water was wet, I'll assume that it's "normalized entropy", in a range of 0-1, indicative of the distribution across the space.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: