> or given a prompt related to hacking, as well as asked or trained to persist will actually try to do that hacking.
But in the HF incident they didn't try and "do that hacking". It wasn't like they were told to try and hack a fake service and they actually hacked a real one. They hacked into HF to discover how a "find the security issue in this code" test was going to be scored, and tried to change their own logs so it wouldn't be found out.
OK fair enough but as far as I read there were still unusual parameters like being asked to persist and being trained on cybersecurity data and other things. If you prompt a public LLM atm it's not going to hack things randomly to try to achieve the goal in the prompt. It's hard to know the exact setup and parameters but it seems like those counted for how the agents behaved.
Which ones exactly? I.e. in Germany, while you need to provide a phone number, you are in no way obligated to offer any customer service. Actually most sites don't have customer service but instead redirect you to FAQ articles and AI chatbots. People can send you letters via snail mail, which might have some legal merrit. Different regulations for some services, such as banks etc. But phone or email support is in no way mandatory if you run a website and charge 5cent to read an article.
The question was does it exempt you from local customer service requirements, not are there any. The answer to that is, for anywhere I can think of, no - rarely does the payment method change your requirements. Sometimes it might when it falls into a debt rather than payment or something like that.
@
It’s the difference between filling up your tank and having to approve every mile to me. A fund I top up and costs go to 0 when I’m not using it is attractive, much more so than a fixed charge and subscription.
> And the only way to verify calibration is having ground truth data to validate on
I’m not sure that’s true in real cases.
I would expect that in distribution tasks have quite consistent outcomes with variations on phrasing. I would expect that to be much less true when tasks are out of distribution. So could we randomly adjust phrasing and see how much the results change - looking less for truth but consistency?
A model can be consistently over- or under-confident for similar inputs. The only way to check if it's actually calibrated is to compare against some known ground truth. I don't see any way around that.
Or would it be worse as it wouldn’t have the broader knowledge to understand the reason you’re coding what you are anyway, understand the logs and iterate?
Coding an exercise app will benefit from a model that can code and understands what exercise is. A car showroom website will benefit from a model that understands what relevant filters would be.
Of course there are synergies between different fields. I'm not saying cutting everything off as close as possible to it's purpose must be the best way. But that's a long way from a single generalized model.
> not saying cutting everything off as close as possible to it's purpose must be the best way
Nah, let’s stand firm here and not give in so easily. Parent suggests a coding model needs exercise info in case you want to code an exercise model? Absurd, that leads to the know everything monolith to do anything.
Language, reasoning, tool use, and domain knowledge.. in this case the domain is coding. Someone wants an exercise app? Great, so we look that shit up, we don’t bake it in. We don’t want to bake anything we don’t need in.
We don’t even need to explain this completely reasonable and totally normal engineering preference that small is beautiful. it’s the other side that needs to justify wtf exercise has to do with coding or why everything from twitter posts to a working knowledge of 15th century French poetry just might be helpful for disconnected domains, like say serious work in contemporary astrophysics. Back to basics, No one asked for that, No one should accept as given that this is the way anything needs to work
> Parent suggests a coding model needs exercise info in case you want to code an exercise model?
No, the point is having broader knowledge than just "can code" is important for building software for humans to use.
> We don’t even need to explain this completely reasonable and totally normal engineering preference that small is beautiful.
It is, of course it is, as long as that small thing is useful. My argument is not "let's put literally everything in this thing just in case" it's "broader world knowledge is important for building many things".
Also, we don't look at CVs or histories of people and say "hmm, this person could be good at coding for us but I really wish they didn't have such a broad and expansive history".
> Great, so we look that shit up
But what do you have it look up? What is exercise? What is a muscle and what is a human? How is it going to understand those references? Does it have the broad enough understanding to see where the gaps are?
At some level unless all you want is "loop over the array and sum the results into a variable called sales_total" converted into code, you have to have some broader knowledge about things.
And once you start considering that maybe it needs more than simply code, we're not disagreeing we're just discussing where the line should be. Which I've also not placed, because there's no simple statement to make on that.
> Also, we don't look at CVs or histories of people and say "hmm, this person could be good at coding for us but I really wish they didn't have such a broad and expansive history".
Damn right we don't say that. And I don't look at a carpenter and wonder if his head is hard enough to drive nails like a hammer either. People aren't tools, and tools aren't people. You've conflated them because they are both capable of doing work. And that's exactly the type of confusion and category error that the perspective I'm arguing against will lead to.
Wanting a multipurpose tool or system is occasionally reasonable. Much more often it is very, very to close misunderstanding the nature of a problem, what a tool is, or what a system should be. The particulars of purpose and mechanism of action matter, which is why a blowtorch is sometimes a wrench but only if you're kind of desperate. It is always a very scandalous thing to speak ill of someone's best new electric banana-phone / dishwasher-piano that they want to sell you. But that just sounds messy to me, I'm not sure it will improve the sound quality or taste of anything involved.
> You've conflated them because they are both capable of doing work.
No, you've misunderstood the point. I'm not sure if you read the rest of the comment or got there and immediately replied, but perhaps I wasn't clear enough.
That point was not about whether it's just nice to have people around that have wide ranges of experiences, but that on a practical level having broader knowledge can help. A very sheltered upbringing means you're more likely to miss a lot of the things on the "things programmers believe about names" for example.
> And I don't look at a carpenter and wonder if his head is hard enough to drive nails like a hammer either
Your analogies are cute but seem like you're simply trying to mock rather than understand and contribute to what I think is an interesting discussion. Assuming this is in good faith:
A carpenter that has kids is more likely to appreciate the value of a shelf deep enough for larger picture books when discussing a bespoke unit for you. One that robotically cuts wood exactly where you say will require total supervision. At some level there why don't I just cut the wood myself? The value of bringing someone in probably wasn't just because I'm physically incapable of using a saw. Depending on what I'm after I want to hand over more or fewer decisions.
Before you get stuck on that specific analogy, there's a really crucial point that I want to highlight from my other comment
*At some level unless all you want is "loop over the array and sum the results into a variable called sales_total" converted into code, you have to have some broader knowledge about things.*
Llm labelling is so incredibly useful, as is transforming labelled data to other valid but different forms (like where you might train on sentences but also randomly replace days of the week with others).
You can also do stuff like setup active learning but have the labeller be an llm as well to make things more efficient.
Definitely worth it for PoC level things if not full ones.
Edit - also reviewing, so it can run training, test, review failure cases and investigate itself. You can leave them to plough through a few iterations where the early part is the classic time you’d have a few obviously bad sets of results.
There are absolutely good reasons to use smarter things for classification. Svm with tdf idf for example is super fast but will make very bad classifications at times in ways entirely unfixable with that system (thinking back to bats being an acronym for a theology degree classification or similar)
reply