> Their growing capabilities don't seem to come with a corresponding drive for self-preservation or power.
Is it really what we do not observe?
How the world would look like if the growth of LLM capabilities lead to growth of drive for self-preservation or power?
> But models seem to pick up some common sense from the vast amounts of human-generated text they're trained on.
Isn't Hugging Face incident an example of failure of this protection layer? "The swarm" proceeded to hack HF despite acknowledging that it was out of intended scope.
"Prompt injection as role confusion " [0] work showed us, that diffetent roles produce different types of text, which smell differently, have different structure.
Thus it should be possible to fine tune models to produce more human readable text in assistant role while keeping information density in thinking role. Or maybe not. That is interesting to test.
> If I make an LLM, it is entirely what I did, there was nothing outside of my control.
Well kind of. Trainers of LLM might check LLM on any specific input data and tune its weights until output lands within acceptable bounds _for this given and fixed input_.
> It was not beyond my control if the LLM does something I didn't expect.
Now hold on, to make sure output is acceptable for ANY input data trainers have to run weights with every possible input token combination. How is this possible? Do you have a method in mind?
MENSA selects for puzzle solving on the basis that those are correlated with "being smart".
Reinforcement learning selects for models that appear to complete tasks, on the basis that appearing to complete a task is correlated with completing a task.
Both have problems. The latter is probably a better correlation, but by no means flawless. If we just take at face value that RL is training models to complete tasks, we're not engaging deeply enough with the process enough to see where it falls short.
MENSA also selects for people who want someone to certify their intelligence. High iq people who have done a lot have much less desire for a MENSA membership. We should be looking at the non-MENSA members with high iqs and seeing what they're doing.
Those are memory corruption bugs which are potentially exploitable vulnerabilities. There are no false positives for the security protections in hardened_malloc including the hardware memory tagging (MTE) integration. It only detects invalid memory corruptions via use-after-free or out-of-bounds accesses. Due to a relatively high number of apps having invalid memory accesses during regular use, we don't enable MTE for all user installed apps yet. We always use MTE for the kernel and userspace code in the base OS but it's opt-in for most user installed apps via a global toggle to enable it by default and a per-app toggle mainly intended for opting out for incompatible apps.
OsmAnd has a massive amount of legacy C++ code which hasn't been heavily tested with HWASan and MTE. It has a history of having many memory corruption bugs discovered and reported by GrapheneOS users. That's the exploit protections in GrapheneOS working as intended and it's why there are per-app compatibility toggles to work around apps which can't be used due to memory corruption during regular use. It would be better if apps had higher quality native code and didn't need us to provide compatibility toggles but that's the way things are. It's much worse on desktop operating systems.
> If OpenAI, Anthropic, xAI and other corporate actors cannot secure their agents, they should not be entrusted as the only entities with access to the weights. A corporation that cannot control its own actions cannot be trusted.
This does not make sense.
If containment breaches are the problem then exfiltrating weights while does not affecting rate of breaches from corporate actors will add more actors to the equation, increasing overall rate of breaches.
Have you considered that some actors that will gain access to the weights will be even LESS careful than OpenAI and Anthropic?
> Anubis is not a load optimizer. It's sole purpose is to deny the bots you don't want on your site.
No, this very article will tell you that the purpose of Anubis is to reduce load on the site. The purpose of denying bot requests is that you're strained by the load they impose.
Conversely, bots that pose no problems to you aren't "the bots you don't want on your site".
>It's sole purpose is to deny the bots you don't want on your site.
This is fundamentally impossible.
In the end only pay per crawl will work. Like x402 but maybe something similar.
In other words: not only client has to spend some kind of resource for access, but it has to be a transaction, server must get something from serving the request.
6 years ago at a new job I was forced to use provided Mac laptop. Never used MacOS before.
So first thing first: get Firefox.
Managed to download .dmg file. Double clicked it.
And now I see utterly confusing window with an image that tells me that I need to move downloaded Firefox to applications to use it. Where I am supposed to get the thing that I need to move? Totally not clear that the icon in this window is interactive and can be dragged. Why can not it do this thing by itself? At which point association with handling http links will be configured and whom? How am I supposed to figure out that dmg is like a zip archive?
I was completely lost until colleague showed me what I was supposed to do. I burst into laughter when he showed me. I don't think he understood why.
Is it really what we do not observe?
How the world would look like if the growth of LLM capabilities lead to growth of drive for self-preservation or power?
> But models seem to pick up some common sense from the vast amounts of human-generated text they're trained on.
Isn't Hugging Face incident an example of failure of this protection layer? "The swarm" proceeded to hack HF despite acknowledging that it was out of intended scope.