You can throw a ball up in the air and predict that it will fall again
and bounce. You have no understanding of mass, gravity, acceleration,
momentum, impulse, elasticity...
You can press a button that makes an Uber car appear in reality and
take you home. You have no understanding of apps, operating systems,
radio, internet, roads, wheels, internal combustion engines, driving,
GPS, maps...
This confusion of understanding and prediction affects a lot of people
who use technology in a "machine-like" way, purely instrumental and
utilitarian... "how does this get me what I want immediately?"
You can take any complex reality and deflate it, abstract it, reduce
it down to a mere set of predictions that preserve all the utility for
a narrow task (in this case visual facsimile) but strip away all depth
of meaning. The models, of both the system and the internal working
model of the user are flattened. In this sense "AI" is probably the
greatest assault on actual knowledge since the book burning under
totalitarian regimes of the mid 20th century.
What if the model actually understands that the ball will fall and bounce because of mass, gravity, acceleration, momentum, impulse, elasticity? I mean you can just ask ChatGPT and Claude, I guess you would answer that in this case it's just prediction, but if they were human then it would be understanding.
> I guess you would answer that in this case it's just prediction,
No I would answer that it is indeed understanding, to upend your
"guess" (prediction) and so prove that while you think you can
"predict" the next answer you lack understanding of what the argument
is really about :)
I think I understand the topic quite well, since you deliberately deviate from answering the question. You made a practical example that doesn't really work in practice.
The MLM has no idea what it’s making, where you are in the map, what you left behind, and what you picked up. It can accurately predict what comes next, but if you pick up an item and do a 360° turn the item will be back and you can repeat the process.
We are talking about neural networks in general, not this one or that one, if you train a bad model or the model is untrained it would not indeed understand much or anything.
Generalisable how? The model completely hallucinates invalid input, it's not even high quality and required CSGO to work. What's the output you expect from this and what alternatives are there?
None of those questions are relevant are they? I get the impression you've already decided this isnt good enough, which is basically agreeing with everyone else. No one is talking about what it's capable of today. Read the thread again. We're imagining the great probability a few permutations later this thing will basically be The Matrix.
It did not require CSGO, that was simply one of their examples. The very first video in the link shows a bunch of classic Arati games, and even the video which is showing CSGO is captioned "DIAMOND's diffusion world model can also be trained to simulate 3D environments, such as CounterStrike: Global Offensive (CSGO)" — I draw your attention to "such as" being used rather than "only".
And I thought I was fairly explicit about video data, but just in case that's ambiguous: the stuff you record with your phone camera set to video mode, synchronised with the accelerometer data instead of player keyboard inputs.
As for output, with the model as it currently stands, I'd expect a 24h training video at 60fps to be "photorealisic and with similar weird hallucinations". Which is still interesting, even without combining this with a control net like Stable Diffusion can do.
You do the same thing at a larger scale, and instead of video game footage you use a few million hours of remote controlled drone input in the real world.
To answer your question directly, the benefit is that we could make something different from counter strike.
You see, there are these things called "proof of concept"s that are meant to not be a product, but instead show off capabilities.
Counterstrike is an example, meant to show off complex capabilities. It is not meant to show how the useful thing of these models is to literally recreate counterstrike.
Which capabilities are being shown off here? The ability to take an already existing world-model and take lots of compute to have a worse, less correct model?
The capability to have mostly working, real time generation of images that represent a world model.
If that capability is possible, then it could be possible to take 100 examples of seperate world models that exist, and then combine those world models together in interesting ways.
Combining together world models is an obvious next step (IE, not showed off in this proof of concept. But it is a logical/plausible future capability).
Having multiple world models combined together in new and interesting ways, is almost like creating an entirely new world model, even though thats not exactly the same.
Gosh I’m so excited for AAA games to get _even more garbage_ once this gets good enough for investor-driven studios to start slinging out AI driven, hyper-derivative, low-effort slop with even less effort.
We all thought Sports-Game $CURRENT_YEAR or CoD/MW MicroTransaction-Garbage was bottom of the barrel, a whole new world of fresh hell awaits us!
You have it all wrong. It's not going to be AAA studios doing this. Instead, it will be randos who have a cool/fun concept that they want to try out with a couple friends.
Sure, most will be bad. But there might be some gems in there that go viral.
Making it easier for regular people to experiment with games is a good thing.
Or even better, what if this allows people to throw something together to see if a game mechanic is fun, and once it's tested out, then they can make the game for "real".
Allowing quicker experimentation times is also a good thing.
You'd have to have the rights to do the training wouldn't you ? Or does that mean I should close off all of my creations legally just so you can't use them ?
I don't have a problem with you using this with a Camera and the real world or your creations ; but I do have a problem when people are able to use someone's work and use these blend and call them original.
It's just as I would have taken a 3d model static mesh, apply some blend on it and call it my own.
> you'd have to have the rights to do the training wouldn't you ?
Thats undecided by the courts. Everyone is training using other people's data right now though, and very few companies are even being sued (let alone have finished the multi-year process to actually be punished for it).
> Or does that mean I should close off all of my creations
When you put something out there, you should expect that other people are going to use those creations. Almost certainly in many ways that you don't approve over.
Don't release something publicly if you don't want other people to use it.
> but I do have a problem
Your anger isn't particularly useful. What are you going to do about it? This particular proof of concept model was made with 1 single GPU running for 2 weeks. You can't stop that.
> It's just as I would have taken a 3d model static mesh, apply some blend on it and call it my own.
Something that I am sure many people are doing all the time, already. Transforming and using other people's content is as old as the internet. AI does little to change that.
You misunderstand that I'm also a consumer and that I look upon these things. If I see an someone bring up some AI stuff and call themselves an artist I know how I look at them.
This is a minor update of the "Little Book of Deep Learning", a phone-formatted free book for people with a STEM background that happened to be quite popular (580k+ download in a year).
This update adds in particular a chapter about methods to run or fine-tune LLMs with limited compute and memory (prompt engineering, quantization, low-rank adapters, and model merging).
I have no idea what this means.