It's kind of crazy they can do this without informing customers. Essentially we like many other knowledge working companies are slowly outsourcing intelligence to a 3rd party that can, at will it seems, alter that intelligence without informing us. Just means we have to invest more in harnesses, guardrails and keep active in open model space.
I think (2) is the important one. Anthropic provides benchmarks with their model releases. If they are making changes that make models perform worse on advertised benchmarks, how can this be legal?
It's a good point. I mean I have no idea if they are actually breaking the law, would assume their tc's would dissolve them from any legal jeopardy but it's still not right.
Is this where independent benchmark companies like artificial analysis could help ?
Google gives a lot of information to its customers, the advertisers, when it changes things that impact them a lot. Not enough for all of them to be happy, but it does private and public communications on big changes.
Super smart but it's one of those ideas that hopefully gives someone else in this space another idea that leads to something that could mass scale and I would buy. Love the ingenuity though.
We spend a small fortune trying to get ontology right as it's so critical to real world success of enterprise AI. Curious if this could actually help us.
Highly doubtful. More than likely someone is picking targets here. Going for food supply is a war crime. So it could be an extremely hostile nation state. Or some really psychopathic disgruntled person/group. If there's no ransom it strongly leans toward a nation state in my opinion. A lot of turmoil between Europe and Russia+the us lately. That's my guess.
>A lot of turmoil between Europe and Russia+the us lately. That's my guess.
Attacking power plants, grocery stores, sending drones into multiple countries over and over, setting up gun/dead-drops in German forests, sabotaging warehouses and factories, etc wasn't enough?
PMs ultimately take bets. Some to many will fail and if they are great a few will pay off in an excessive way. I have yet to see anything in the AI field that make better bets than my best PMs (I manage 50 of them) but as soon as I do the whole calculus changes. I joke with them about it. They also joke back it could happen to me (which it absolutely could). We live in interesting times.
You seriously manage 50 human PMs? That sounds really interesting, I can't even imagine what kind of role that would be, other than someone running multiple companies simultaneously. Could you share what you do?
No, a very large software company. Have a team structure so not dealing with day to day but ultimately responsible. Have spent last year blending the team with engineers who have been interested in getting into product. Ultimately see that will be direction of travel across the industry.
reply