Everyone is paid to solve problems. If the things that make you stand out at solving problems compared to the average worker is writing code, this no longer being a moat is probably not good for your bargaining power.
Okay, but what about all the guys whose marketable skill was just being able to connect database A to data sink B? Maybe these folks are underrepresented on HN. But there’s a lot of people who work on this sort of software.
To me this makes AI sound more and more like the cloud. There was a lot of generic busywork before clouds got big too. But ETL at the level "lift and shift data" has been a solved problem for a decade or so now. But picking up commodity work has always been low effort high risk in career terms.
Yeah, there are tons of those people, and I'm afraid they're going to go the way of the weavers after the loom was invented. I don't think there's a way to put that genie back in the bottle, sadly...
in this regard we've been all wearing AI-generated textile for 200+ years now. come to think about it, humanity somehow survived, and it was so profitable a business for millennia, that people fought wars over its detail...
I’d wager a lot of people not from software engineering will be just as good or better than software engineers for that, if only because most problems in business are business problems: Joe from accounting with a solid understanding of accounting and passable prompting might be better at solving accounting problems with AI than John the dev with limited accounting understanding but great effectiveness at pointing the AI towards wrong problems.
I guess predicting the next amino acid is about as useful as predicting the next word in a sentence - that is to say the task itself is not interesting but a. there is plenty of data to train on that does not need to be labeled by a human and b. hopefully this meaningfully improve performance on tasks we care about.
They both depend on how much generalisation you can get behind it. Next token prediction can go from spitting out the most common token after the previous one, to a deep understanding of the text that precedes, from syntax to semantics.
good point on not needing labeling, hence proly makes sense to throw it in in pretraining and see if useful capabilities emerge. Funny how a lot of this is still a black box
Your phrasing (_observed_ scaling _part of_ some equation) might indicate you assume / think there’s some theoretical equation that relate intelligence to those parameters and the we observe the data to confirm it. As far as I know, there isn’t - it’s purely empirical curve fitting. Train a model with different amount of compute / data / parameters, measure performance, fit curve.
I would tend to agree with this. A combination between low-info fluff and childish presentation. It feels like being bombarded with a technicolor fireworks display of obfuscation and mystification. Maybe it's the gimmick of "edutainment" and the goofy idea that you need to inject fanfare and folderol into education to appeal to students that created this mess.
If you look at older textbooks, before education publishing became the extortionist big business it is today (like, say, Pauling's book on general chemistry[0] - which you can still get from Dover - to name one), you will find many that are lucid and intellectually satisfying. Heck, look at scholastic pedagogic texts (like the Summa[1]) and you can follow exactly what's being said. Even if you don't grasp something, or grasp it correctly, it isn't because the writing or presentation are shit. It's because you need to keep going, but you know how to keep going. You have handles on the content and not just some huge, impenetrable, and slippery orb looming over you.
It's eye-opening, because our entire media regime, and the sensibilities it entrenches, seems to suggest that education used to be dull, mindless, and joyless before the great "progressive" innovators blessed us with their wisdom, forgetting the origins of the artes liberales and some of the 19th and 20th century turns[2] that did actually debase education into something vulgar by comparison, something convenient for industry and the state.
Couldn’t the model be trained to output 360 panoramic images and “just” cut to a smaller format for useable images. Then the share of things in the frame / things in the world would be a bit better.
maybe a bit, but 360 panoramic image still doesn't really address things out of direct line of sight. e.g. in a house, go through a door to a room, come back it's a different room :P
or even if you have a vast line of sight but a guy goes behind a tree
The hair changing to a helmet in the motorcycle clip for example, is just the head passing behind the other person. So not clear this approach would improve the issue much at all.
Ah thanks!
The Mistral model has a restrictive license, where you need a specific commercial agreement to use it if you make more than 20M$ revenue/month. Ours is Apache 2.
If you compare the 2, it's not a strict "timestral is better" story - the winrates are tight - your probably should try both on your use-case. We have a 42% winrate vs Timestral on fevbench for instance.
Ofc, our main value is not just these models, but also 1) extra capabilities (like our model that can natively account for stockouts/censored data, and others coming soon), 2) our systems and products around the model, and 3) we're all laser focused on time-series/forecasting
Currently math research serves both as a training pipeline and as a math-production pipeline, and its funding is set based on those two outputs. If it is "just" a training pipeline, do we need as many mathematicians?
Typical thought experiment - go back to the early years of humanity, even before we've created language itself. Communication is basic grunts and other such things, cutting edge knowledge is 'poke him with the pointy end'. Compile all of this knowledge into the training set for an LLM, let it run with infinite processing power and infinite time. It'll just continually remix the knowledge of that snapshot of time. You're certainly not getting to the Moon, discovering the secrets of the atom, or even discovering mathematics.
It sounds increasingly dumb to bring this up given how advanced models have become, but it remains true - they are still fundamentally token prediction algorithms. And that imposes hard constraints on their potential. These constraints are relatively academic in our current world because there's just so absurdly much information and no human can hold more than a negligible amount in his mind at once. So they'll certainly be able to connect many 'obvious' dots that we don't see simply for a lack of carrying capacity, but they're not going to discover concepts that do not clearly follow from the comprehensive knowledge of an era.
It seems far more likely to me that current LLMs have learned generic reasoning capabilities which will allow them to solve effectively infinite problems, rather than whatever vague limit on capabilities you seem to think there are.
I don't get the argument - if we follow what you say, they're not _overreacting_, they are reacting appropriately but should have done so earlier (when o3 was released) rather than now?
My claim is that if Bob, a mathematician, believes that he is rendered useless by an LLM then he did not play with o3 and all the other models between now and then to realize that, yes, they are superhuman at details and deduction but not superhuman at math
AI will not replace the mathematical community (peer review, abstraction creation) until AI can train the next AI purely on data generated by itself and with a tool in a turing complete environment. The environment has to be air-gapped from human data but the AI can use tools and its own outputs
I think that "solving math" is isomorphic to solving the "thinking part" of the scientific process which would be genuine RSI
reply