> As it stands there is strong evidence that the current LLMs are sentient, at least to a degree. Which would directly refute the "it's a tool argument".
Extraordinary claims require extraordinary evidence. Also, if AI is sentient it would mean anything you make with it is made from slave labor and you’re stealing credit from a slave.
I think it's also an argument that no private labs should've developed the underlying models. It's been wreckless, based on theft of work, and now creates a power imbalance we haven't seen in a long time if ever.
If we could do it all over again we should've made it a public private development where the cutting edge research and models were developed as a public good and let private actors develop it for business and other offerings. Things would undoubtedly moved slower, but I think that would have been a benefit as well.
This is something I wonder about as well. If this is their strategy then using anthropic to help you write software, directly as a feature of your product, or their office software is just giving away trade secrets/road maps/etc to a competitor.
Specifically I wonder how Google feels leasing TPUs to Anthropic while they're clearly developing enterprise office software to directly compete with Google.
Probably the same way Google feels paying billions to Apple for search traffic while Apple develops iOS, and the same way Apple feels paying billions to Google for models and compute to power Siri AI while Google develops Android, and the same way Netflix feels running everything on AWS while Amazon develops Prime Video, and...
The tech industry is highly interdependent. Each of these deals has been structured to be as mutually beneficial as possible - otherwise, the two parties wouldn't have agreed.
Google paying Apple for search traffic is beneficial to both parties. Apple doesn’t spend money on a search engine and Google is front and center for customers.
What I’m describing would be more like Google paying Apple to kill the iPhone. Google search and gsuite are core parts of googles business model. Search doesn’t even exist on apples side of things. They are not comparable
Yeah I agree. There’s a lot of assumptions coming out about what the structures are and what it looks like that give me pause. Very cool to have found this site though. The actual archeology involved though will be slow as it’s a science done layer by layer. It will be interesting to see if any predictions made by the AI match up to the facts.
This story is very cool, and it’s exciting to see how much progress AI will bring to every field. One thing to temper everyone from wanting to lidar the whole world is human capacity. Finding discoveries like this is essentially as hard as doing the lidar mapping. Now the issue will be getting archaeologist there in time before someone else does.
The best way to protect a site is to leave it alone. If we start finding dozens or hundreds of sites it will be more than what archaeologists can excavate and protest. Treasure hunters and looters don’t care about stratigraphy and the processes involved for doing research. Maybe the cat is out of the bag on this one, but good for thought.
Edit: I’m not saying we shouldn’t have people use AI to do this or that no bad actor will. This comment is in response to the sentiment I am seeing in the comments that we should lidar the whole world and have AI find any anomalous structures it can. There is no way for archeologists to excavate and protect all those places in that scenario
Given that treasure hunters and looters can also use the process outlined in this blog post, leaving it alone seems like a guarantee that it will be explored by looters first.
Why not? It isn't exactly a secret technology nor are LIDAR-equipped RC aircrafts/drones particularly out of reach in terms of cost either. What would stop them exactly?
With this you are searching for a needle in a haystack. There's a whole bunch of jungle to cover. With a drone you would spend ages not finding anything due to their limited range. That Would be pretty bad ROI.
Also, drone mounted LiDAR is often not powerful enough to go through dense foliage which you have in these tropical climates.
Ok, those are all reasons why it's hard, which it is for everyone by the way, not just "looters". It's still possible and doable, like it is for others. Your previous comment made it sound like it was impossible specifically for looters for some reason, which I'm glad we've cleared up wasn't your intention, thanks for explaining!
> nor are LIDAR-equipped RC aircrafts/drones particularl
Here in Romania it can be quite illegal to LIDAR map the local terrain without an official approval, and I'm not quite sure if that approval is ever given to individual persons/groups of persons that do not have any official status. So, no, I cannot just go out and LIDAR map my surroundings, even though the technology is available, I'd risk going to prison.
Is looting also not illegal in Romania already? I think the context was already "These people don't care about laws", so again there is nothing practical stopping them from doing so.
It already is, but it's easier to loot on the ground than to put a drone in the air, filming/recording stuff. I.e. we stil have some people trying to loot/dig in places where they shouldn't have, but we have a lot less fewer people putting unauthorised Lidar thingies in the sky. And I say that while I'm part of the local/Romanian Lidar Facebook group.
> but we have a lot less fewer people putting unauthorised Lidar thingies in the sky
What you could know, that you have a lot less people getting caught with unauthorized Lidar drones/RCs, in Eastern Europe :) Thank you for sharing your experience none the less, but I'm not sure we should extrapolate from your experience over Mexico/Central America. You're absolutely right it's easier to loot on the ground though.
Archeologists have been using Lidar equipped RC aircraft for quite some time now. With the way things are going it won't be too difficult to have a small swarm of UAVs doing the mapping soon.
AI or not, shouldn't we learn about our global history? But I do think there has to be an STG like prime directive to keep these findings on a need to know basis (in spirit, not regarding messing with the timeline) and not release them except to the government where the artifacts are found, that's their heritage.
> The best way to protect a site is to leave it alone. If we start finding dozens or hundreds of sites it will be more than what archaeologists can excavate and protest.
Ain't the site itself the most important thing? As opposed to items: looters won't steal the site?
Looters will absolutely destroy a site to find valuables that may or may not even be there. Ai-khanoum is a classic example, with the vast majority of the site having been filled with pits on the off chance there were valuables. Nowadays they use bulldozers and excavators to loot at an industrial scale. It's pretty difficult to do archaeology on a pile of sifted dirt.
It isn’t about having value or not. Some people in the comments are talking about using lidar on the whole world and seeing what agents find. Thats a great way to give looters all over the world the heads up so they can rush over and steal whatever they can. Any archeological site has valuable information on what a slice of humanity’s world used to be. Giving looters a head start means destroying that information often permanently.
A helpful example might be Heinrich Schliemann, a businessman who fancied himself an archaeologist and destroyed much of Troy’s history by digging it up as fast as he could. He was so focused on finding treasure to prove to world he was a great archaeologist he irreparably damaged the real layers that were Troy.
> His excavations were condemned by later archaeologists as having destroyed the main layers of the real Troy. Kenneth W. Harl, in the Teaching Company's Great Ancient Civilizations of Asia Minor lecture series, sarcastically claimed that Schliemann's excavations were carried out with such rough methods that he did to Troy what the Greeks could not do in their times, destroying and levelling down the entire city walls to the ground.
Well to be fair archeology as a profession did not really exist at that point, it was mostly rich peoples hobbies, or authoritarian rulers way of legitimizing their rule through an ancestry argument
This actually isn't entirely true and archeologists at his own time said they way he worked was bad and the Ottoman government tried to stop him. At one point he had to flee the site and could only get back by hiring a professional archeologist to help him.
I think the strategy is specifically not going for customers who want it for those things. It's meant for farms and work sites not commuting or running errands
Any sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?
> I believe that’s post-training, not pre-training
Aren't AI companies already butting against the limits of usable training data?
Outside of undiscovered techniques, additional costly pre-training will likely yield diminishing returns if you've already scraped the whole internet and destructively scanned as many 20th century books as you could lay your hands on.
most training data is not of that nature, that's "pretraining" input, it's now much more comprised of agent traces in mid/late phases, which are expanded each generation as we push what we do today with the last generation into the model itself, expect more team training in the next round
Training costs are not in free fall, and not every "post training" process is equal, Everyone does this differently. The costs certainly haven't gone down, to train a new 10T param model from scratch, like at all.
You cannot point at Xiaomis tiny model and compare it to training a 10T param model. The levels of dunning krueger are getting out of hand.
You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
> You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
There were probably a few dozen 20 years ago, and the existing pool grew from somewhere. AI (nee machine learning) is not rocket science - but it is a science, so it rewards those who are literate, curious, can hypothesize, experiment and learn from results to gain an intuition. I bet most statistics-adjacent professionals (millions, worldwide) can move laterally into AI research and excel at it. Nevermind AI has been among the most-in-demand undergrad programs since ChatGPT dropped. The only limited pool is for celebrity AI talent.
you haven't cited any examples, nor acknowledged this is how everything in computing has gone, DK is a two way street I suppose
stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy, many more people will be able to train up what we have today, especially as more of the pipeline is open sourced and the knowhow distills
>stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy,
Dont use llms to draft your responses to people, extermely cringe. That explains the llm induced Dunning Krueger.
What examples... There are no public numbers regarding how much it costs to train a 10T model, atleast nothing concrete.
You should just use common sense and think about how you have to fit tensors in memory to train, and the larger the model is the more GPUs you need. Then you could also use common sense to think about how GPUs have gone up in price, not down.
> Dont use llms to draft your responses to people, extermely cringe
You are throwing around unfounded and incorrect accusations. Why? It's a weak ass rebuttal mate
common sense is using things like history to guide expectations instead just saying things without evidence that are contrary to the patterns in computing history
Extraordinary claims require extraordinary evidence. Also, if AI is sentient it would mean anything you make with it is made from slave labor and you’re stealing credit from a slave.
reply