Hacker Newsnew | past | comments | ask | show | jobs | submit | dfdydx's commentslogin

Everyone is paid to solve problems. If the things that make you stand out at solving problems compared to the average worker is writing code, this no longer being a moat is probably not good for your bargaining power.

The things that make me stand out at solving problems is that I can effectively convince machines to solve problems at massive scales.

Okay, but what about all the guys whose marketable skill was just being able to connect database A to data sink B? Maybe these folks are underrepresented on HN. But there’s a lot of people who work on this sort of software.

To me this makes AI sound more and more like the cloud. There was a lot of generic busywork before clouds got big too. But ETL at the level "lift and shift data" has been a solved problem for a decade or so now. But picking up commodity work has always been low effort high risk in career terms.

Yeah, there are tons of those people, and I'm afraid they're going to go the way of the weavers after the loom was invented. I don't think there's a way to put that genie back in the bottle, sadly...

Everyone is now a member of the weaving business. AI exists to solve wages.

in this regard we've been all wearing AI-generated textile for 200+ years now. come to think about it, humanity somehow survived, and it was so profitable a business for millennia, that people fought wars over its detail...

Do you know how many people “that sort of software” displaced just a few decades ago? Probably a lot…

I’d wager a lot of people not from software engineering will be just as good or better than software engineers for that, if only because most problems in business are business problems: Joe from accounting with a solid understanding of accounting and passable prompting might be better at solving accounting problems with AI than John the dev with limited accounting understanding but great effectiveness at pointing the AI towards wrong problems.

Yeah A.I is pretty good at that

Now everyone can.

I know the incantations to summon the daemon and bend it to my will.

There are a lot more people able to do that than there were able to code.

I guess predicting the next amino acid is about as useful as predicting the next word in a sentence - that is to say the task itself is not interesting but a. there is plenty of data to train on that does not need to be labeled by a human and b. hopefully this meaningfully improve performance on tasks we care about.

They both depend on how much generalisation you can get behind it. Next token prediction can go from spitting out the most common token after the previous one, to a deep understanding of the text that precedes, from syntax to semantics.

good point on not needing labeling, hence proly makes sense to throw it in in pretraining and see if useful capabilities emerge. Funny how a lot of this is still a black box

Your phrasing (_observed_ scaling _part of_ some equation) might indicate you assume / think there’s some theoretical equation that relate intelligence to those parameters and the we observe the data to confirm it. As far as I know, there isn’t - it’s purely empirical curve fitting. Train a model with different amount of compute / data / parameters, measure performance, fit curve.

It’s a simple exploration/exploitation trade off. Always explore, never exploit is probably not the optimal strategy every time.

It seems to me that everyone is in "always exploit" mode.

Seems weird when it's become apparent that LLMs aren't really special sauce outside of just having the compute.


Coming from somewhere else, US textbooks feel super watery - low signal to noise ratio, too much hand holding.


I would tend to agree with this. A combination between low-info fluff and childish presentation. It feels like being bombarded with a technicolor fireworks display of obfuscation and mystification. Maybe it's the gimmick of "edutainment" and the goofy idea that you need to inject fanfare and folderol into education to appeal to students that created this mess.

If you look at older textbooks, before education publishing became the extortionist big business it is today (like, say, Pauling's book on general chemistry[0] - which you can still get from Dover - to name one), you will find many that are lucid and intellectually satisfying. Heck, look at scholastic pedagogic texts (like the Summa[1]) and you can follow exactly what's being said. Even if you don't grasp something, or grasp it correctly, it isn't because the writing or presentation are shit. It's because you need to keep going, but you know how to keep going. You have handles on the content and not just some huge, impenetrable, and slippery orb looming over you.

It's eye-opening, because our entire media regime, and the sensibilities it entrenches, seems to suggest that education used to be dull, mindless, and joyless before the great "progressive" innovators blessed us with their wisdom, forgetting the origins of the artes liberales and some of the 19th and 20th century turns[2] that did actually debase education into something vulgar by comparison, something convenient for industry and the state.

[0] https://archive.org/details/generalchemistry00paulrich

[1] https://aquinas.cc/la/en/~ST.I.Q1

[2] I am not claiming everything was perfect centuries ago, or that there were no good pedagogical developments since then.


Couldn’t the model be trained to output 360 panoramic images and “just” cut to a smaller format for useable images. Then the share of things in the frame / things in the world would be a bit better.


maybe a bit, but 360 panoramic image still doesn't really address things out of direct line of sight. e.g. in a house, go through a door to a room, come back it's a different room :P

or even if you have a vast line of sight but a guy goes behind a tree


The hair changing to a helmet in the motorcycle clip for example, is just the head passing behind the other person. So not clear this approach would improve the issue much at all.


So is Mistral's model, right? Genuine question because I had been following your company - is there any use case where t0 is better than timestral?


Ah thanks! The Mistral model has a restrictive license, where you need a specific commercial agreement to use it if you make more than 20M$ revenue/month. Ours is Apache 2.

If you compare the 2, it's not a strict "timestral is better" story - the winrates are tight - your probably should try both on your use-case. We have a 42% winrate vs Timestral on fevbench for instance.

Ofc, our main value is not just these models, but also 1) extra capabilities (like our model that can natively account for stockouts/censored data, and others coming soon), 2) our systems and products around the model, and 3) we're all laser focused on time-series/forecasting


interesting. The github repo (https://github.com/mistralai/timestral ) says the license is Apache 2. Where does the 20M$ revenue limit comes from?


It's in the licence of the weights, which you can see on Huggingface.



Currently math research serves both as a training pipeline and as a math-production pipeline, and its funding is set based on those two outputs. If it is "just" a training pipeline, do we need as many mathematicians?


Well humans do that and you expect them to continue to do that - why can't AI models do it as well?


Typical thought experiment - go back to the early years of humanity, even before we've created language itself. Communication is basic grunts and other such things, cutting edge knowledge is 'poke him with the pointy end'. Compile all of this knowledge into the training set for an LLM, let it run with infinite processing power and infinite time. It'll just continually remix the knowledge of that snapshot of time. You're certainly not getting to the Moon, discovering the secrets of the atom, or even discovering mathematics.

It sounds increasingly dumb to bring this up given how advanced models have become, but it remains true - they are still fundamentally token prediction algorithms. And that imposes hard constraints on their potential. These constraints are relatively academic in our current world because there's just so absurdly much information and no human can hold more than a negligible amount in his mind at once. So they'll certainly be able to connect many 'obvious' dots that we don't see simply for a lack of carrying capacity, but they're not going to discover concepts that do not clearly follow from the comprehensive knowledge of an era.


Perhaps they could some day. I'm basing my understanding on the current day capabilities. Once that changes I will update what I think.


It seems far more likely to me that current LLMs have learned generic reasoning capabilities which will allow them to solve effectively infinite problems, rather than whatever vague limit on capabilities you seem to think there are.


I don't get the argument - if we follow what you say, they're not _overreacting_, they are reacting appropriately but should have done so earlier (when o3 was released) rather than now?


My claim is that if Bob, a mathematician, believes that he is rendered useless by an LLM then he did not play with o3 and all the other models between now and then to realize that, yes, they are superhuman at details and deduction but not superhuman at math


Can you come up with a concrete prediction on what AI can’t do in math? Something testable.


Yes, here is a concrete prediction

AI will not replace the mathematical community (peer review, abstraction creation) until AI can train the next AI purely on data generated by itself and with a tool in a turing complete environment. The environment has to be air-gapped from human data but the AI can use tools and its own outputs

I think that "solving math" is isomorphic to solving the "thinking part" of the scientific process which would be genuine RSI


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: