I always find so much cognitive dissonance in these kinds of posts.
> My most successful work this year was hand-writing a tool, replacing ~4,000 lines of vibe-code with a few hundred. My system has proven more flexible, easier to test, and more generalizable than what it replaced.
> agents still need human taste and experience to produce good work.
Where do you think human taste and experience in software comes from?
> overstimulating & exhausting in a way that coding never was.
> for many of us, honing the code, finding the right design, seeing the parts mesh together — that was the meditative purpose we found in our jobs.
> That’s not the only time AI waylaid me. LLMs could not figure out how to annotate that model’s training data correctly. I spent days going back and forth with them
> LLMs are not yet good companions when brainstorming complicated things.
Why do we expect this to get better? The training data will degrade in quality if more people rely on LLMs so I'd expect models to eventually start getting worse even if they currently enjoy apparent improvements in output.
> Writing code is solved.
What a strange thing to say given all the above experiences, which I think we all share.
Why are we willing to give up so much in terms of purpose and understanding for perceived productivity gains?
I am not OP, obviously, but I am writing to address the "Where do you think human taste and experience in software comes from?" point, hoping that I provide an alternative perspective.
As someone who wrote code for 27 years in all sorts of languages, not necessarily impressive code, but got paid for it, made a career, and changed life for the better multiple times, I don't think coding was ever an important part of the journey! Maybe it's cognitive dissonance, as you wrote above, but I believe taste comes from exploration, from trial and error, from being willing to be open and change old ways if new ways can improve how things are done!
Similar to taste in food or even music! You might think certain foods taste bad, but until you taste them, you won't be certain! Sometimes your senses will be right and the food will taste horrible; other times, you'll be fairly surprised. Similar to music—it's my life hobby—you might think certain genres or artists are bad, but until you sit down, give them a shot, and hear what they have to say, you won't know for certain if your judgment is correct.
If you meant by your comment that you need to code a lot to acquire taste and experience in software, I respectfully disagree. You are not completely wrong, though, but putting emphasis on code, if that's what you were going for, is an overstatement of code's importance. For that particular case, similar to the analogies mentioned above, it's about exploration and trying different languages, paradigms, solutions, and stacks, and unfortunately, we as humans have finite time, so we can't be trying and exploring fast enough to form judgments!
What I genuinely love about LLMs is mostly their exploratory powers. I try new stuff that I always wanted to try with actual products and form opinions about the whole experience.
Anyways, as I finished writing this, I realized, as always, I might be wrong! You might be right, taste is highly subjective! And acquiring it is a very personal thing! So I don't think right or wrong are important here!
It's obviously getting better, by the month, and the degrading of training data leading to a collapse has been a fantasy from the start.
"Perceived productivity gain" is a baseless conclusion. At this point it's abundantly clear that the productivity gain is real and massive.
Where I see the incoherence is in that people see too much value in code review and optimizations.
I believe many of the optimizations are premature. If you did not spot the problem in your QA testing, chances are it was not that important in the first place.
If the architecture wasn't perfect, it is now extremely cheap to change it later, as you realize real deficiencies. The agent will do it for you in a matter of hours, and largely on its own.
You will have to budget a lot of QA effort for verifying the significant changes before going to production, but I don't think it's a deal breaker.
> At this point it's abundantly clear that the productivity gain is real and massive.
Oh? Where is it? Why am I not seeing it as a user if it's abundantly clear? In none of the software I've used frequently in the last year have I seen any of those “real and massive” improvements. Not saying they don't improve at all, but the rate hasn't increased since the advent of LLMs.
I'm still waiting for that shiny new era of software productivity to impact any actual users.
I wrote several tools for myself that have a huge impact on my personal life (a calorie/food tracker, a very custom Todo app that works by combining several strategies that I learned over the years, a personal knowledge base that surfaces information better for me). But it's a personal app, I'm not gonna make it public
Congrats? I used to write one of these every so often in a couple hours when I was learning a new programming language.
Don’t take this as directed at you because I don’t know your background, but I’m starting to wonder if many who called themselves engineers here that have similar takes to this ever knew how to actually program to begin with.
No not fully fledged and feature complete. But a baseline POC with the core functionality? Isn’t every react tutorial these days a Todo app?
Anyways my point wasn’t to try and make myself out to be some John Henry (I could have done without the snarky "congrats", apologies). It’s that drowned out in the noise of AI based development, we forget that there are amazing software engineers out there (I’m not placing myself in this category) that were incredibly productive in a short amount of time. Quite some time ago, I watched, in awe, a live stream of two prominent Go developers completely rewrite the HTTP package, for example. And this was over the course of a night.
We need a new metric to gauge AI usefulness other than “I built this thing quickly”
It used to be that most programmers were good enough to reimplement a basic database app in a few hours as a way of learning a new language or framework.
The pool of programmers today is many times larger and less trained and so correspondingly most of today's programmers simply aren't very good and are astounded by feats that were once basic learning tasks.
I mean in total it's about 300k lines of code (across all the tools I built for myself). It's a poor metric, but it's not something you put together in a couple of hours.
Oh, and specifically the food logging part: no, you didn't do that in a couple of hours years ago. Taking pictures of food and having estimates grounded in your personal "recipe" library was a major research project until vLLMs came out.
There is still like 20 free versions of these on the market (pre-ai) I am sure yours is barely different. You just wasted time and tokens and pat yourself on the back for it and somehow feel this can be applied generally as "productivity" and "massive imporvements in software"
> I believe many of the optimizations are premature. If you did not spot the problem in your QA testing, chances are it was not that important in the first place.
QA testing can't find missed optimization opportunities unless the performance is absolutely abysmal. Even then, it can't distinguish abysmal performance due to missing easy optimization opportunities from abysmal performance that's going to require you to sacrifice functionality.
If an app performs well because it caches the result of a slow function, it's going to pass QA just fine. If optimizing that slow function would make it fast, you can drop the cache and have a simpler and more robust app — but QA can never tell you that.
Having a more complex and less robust implementation adds a multiplier to all of your future costs. When a system is 200k lines of code instead of 10k lines of code, every new feature you add and bug you fix takes 5–10× the effort.
I had similar concerns when I was working with GPT-5.5 earlier this year. At the time my project was perhaps ~150k LOC.
After struggling with some more complex features even after many iterations, I started to doubt, and questioned whether I'm writing myself into a dead-end.
I did not. I still made progress and things worked out just fine. Then Fable and later GPT-5.6 Sol came out, boosting productivity and complexity of features I could implement successfully.
The effort only ever went down, and my app was released in the AppStore last week, at about ~500k LOC.
I understand why people might believe that using AI and not manually cleaning up suboptimal or overly defensive implementation would eventually result in an unmaintainable mess.
Based on my personal experience and 500+ hours put into my project, I can confidently say that this is not the case. Increased effort to maintain is negligible.
If the test suite takes too long, it can easily be cleaned up later rather than spending extra time judging test cases in every new MR.
Maintenance of test cases has also become trivial, even with thousands of them. When the pipeline goes red on master I click the new chat button and run /check-pipeline, where I put some concise instructions, and the failure gets investigated and resolved in short order.
It sounds like the problem isn't really that engineer's optimizations are premature. The problem is that engineers have an opinion about the code that doesn't really matter.
What I think you're saying is that opinions that LLMs produce are good enough and malleable, so why even have an opinion?
Which is debatable - oftentimes their opinions are not good enough, and not malleable at all.
We're all still learning how to distinguish those. But in order to learn this, you have to read and understand the code they produce. LLMs can migrate a codebase to Rust, but they still suggest walking to a car wash. Every day they make decisions that humans would be fired for.
LLMs are definitely not yet capable enough to blindly trust them, and there's a pretty good chance they never will be.
> The training data will degrade in quality if more people rely on LLMs so I'd expect models to eventually start getting worse even if they currently enjoy apparent improvements in output.
RL is not dependent on training data. The mathocalypse didn't happen because they found better training data somewhere.
> The mathocalypse didn't happen because they found better training data somewhere.
There are reasons to suspect that OpenAI may have intentionally or unintentionally trained on chat logs from mathematicians--mathematicians who were actively using OpenAI to try to to solve the same problems themselves.[1] If those mathematicians were on the right track, and getting close to a solution, that could be classified as finding "better training data somewhere."
sorry but
given how post-training actually works, there's essentially no direct way in which literal PII-scrubbed data made its way into a production oai datamix for RL warmup purposes in 2026 to the extent this data has influence, it's indirect/at the level of process decisions.
what i am trying to say more specifically is, the folk model some people seem to still have about ai companies "directly training on user transcripts" in this naive fashion is firstly wrong, and secondarily, would be counterproductive as a strategy for continued progress
"transcript shows human trying to do an exotic thing evals don't currently cover -> build synth constructions that broadly cover its genre of problem class -> RL on the verifier set designed to cover the failure"
that's the loop...
the """flywheel""" is... diagnostics! wow!
and so this kind of process improvement driven use of data has an obvious diffuse causal influence on what happens during training, right?
but ~none of that process influence looks like "specific mathematician's specific intuitions leak into the training data".
It’s perfectly plausible that unscrupulous people at OpenAI (and that’s the culture right from the top) searched user transcripts for relevant approaches to problems they were working on, or searched them for problems to look at, and nudged models based on that.
I agree it’s unlikely this happened by accident, or entered training data (though not impossible), but it could certainly be within the context agents were using when tackling problems.
We have no evidence either way, but they had all the data they would need to do this.
saw an interview from the mathematician involved, Tristan Buckmaster, on the numberphile yt channel, and he said he thinks it was more a case of people at anthropic leaking it in person to openai. Apparently with a hint about the area they were looking at it could have been easy (ish) to get the same approach, and indeed the paper openai released for Navier–Stokes uses their approach
Rl works on one of two things - (1) a literal set of rules for a system or (2) training data. The former matches perfectly to things like (most modern) math and games that have perfect sets of rules. Even things like physics and robotic where our rules/models are extraordinarily good, rl barely adds value. Move one step farther to chemistry or a complicated physics scenario like weather, and you are back almost entirely dependant on (2) data. Now there might be a few more domains with rigorous rulesets that haven't been explored fully (id bet with some effort programming might make some strides here, maybe something like circuit layout, maybe even law if folks decided to make a lean-like law scheme...) but the vast majority of problems AI is targeting are dependant on data.
I’m not sure that’s a good argument that models will get good at novel brainstorming and research ideation. Where do you get scalable and high signal verifiable rewards for creativity and insight?
> The training data will degrade in quality if more people rely on LLMs so I'd expect models to eventually start getting worse even if they currently enjoy apparent improvements in output.
Its not a direct analogy, but this seems to me as like saying "I already know maths and english, why do we still hire and pay teachers". Its a massive issue that just seems to be hand-waved away as "we'll figure it out later".
To compound on the problem as well, even those of us who do still write code by hand are actively discouraged from sharing it, knowing no human will ever read it and we'll never be credited for our work, it'll just go in the pockets of a few big AI companies. I suppose there will be some that take some pride in knowing their code is helping AI models advance, but thats a small pool of an already small pool, assuming most people decide to just use the AI to write the code for them. A minority of a minority.
I guess this is the chickens coming home to roost of this whole attitude towards growth now, fallout later
> Why do we expect this to get better? The training data will degrade in quality if more people rely on LLMs so I'd expect models to eventually start getting worse even if they currently enjoy apparent improvements in output.
You are assuming that AI should be trained on speicific field to reason better but it actually doesnt need to.
This always reads like “writing english is solved” just because you collected snippets of famous works and have built a spell corrector.
Code is written to process data. The struggle was never to learn programming syntax (most grammar are only a few page long). The main struggle was that it’s hard to model a domain into an information system, identifying the components and their relationships, what is ingested, and what needs to be output.
Computers have integers and floats, but you need to implement units like distance, money, various counts,… The real world have words and sentences, but that needs encoding like ascii and unicode as well as the “string” concepts to make it work. Take HN, I’d bet that if you can model it in terms of data, writing code is not an issue.
Thank you, this is what I've been seeing. Structs and objects are easy, but the hard part is encoding the domain most accurately into given data types of the computer (or creating new data structures to solve the problem!)
If anything, I do expect the casual non-tech users to be making the most adaptable software because they have what they want in their head.
Although, part of what the dev does is figuring out the weeds, and only a part of users have the patience to sit through detail corrections. And the AI user has to verbalize their wants exactly too.
> I do expect the casual non-tech users to be making the most adaptable software because they have what they want in their head.
Those are the people who are most likely to give a monolithic prompt and then spend a week telling the machine 'I don't care about encryption or file permissions, just make it work.' Not to mention that most people can barely verbalize the problems they have with devices they use for 8hrs a day, let alone form a full, self consistant, project idea and then put it in writing.
if they're persistent and patient enough to see the result of what they think they want and iterate enough times they'll eventually learn. non-coders aren't dumb by default after all and I suspect they'll learn much faster than us because they won't be bogged down by the minutiae like reading docs.
Imagine you could build a piece of furniture without actually needing to know how to cut wood, drive nails etc. Imagine you could do it in 3d without knowing how to actually handle wood, you just have an image you can iterate on without actually costing you anything but time.
I remember the first big app I wrote in high-school. I had to learn C++, learn Qt etc. I knew what I wanted but putting that into code when I didn't know any of the libraries / tools was a huge time investment. If I had LLMs I could've done the same thing it took me 6+ months in probably less than a month.
> non-coders aren't dumb by default after all and I suspect they'll learn much faster than us because they won't be bogged down by the minutiae like reading docs.
They may have a perfectly good shot at being smart, but I still don't buy the argument. The reason they struggle to become coders (assuming they tried at all), in my experience, is a lack of problem-solving and debugging skills. There is no real shortcut here. An "image" of the chair isn't enough; when the first iteration is wrong, you have to be able to explain why it's wrong.
I think there are a lot of people out there with good problem-solving skills but for whatever reason did not get into programming or they tried and they got put off with learning the languages / libraries or got stuck and didn't find a solution easily enough and gave up etc.
With a sufficiently good LLM these people will skip the tedious part of programming that most of us had the motivation / patience to go through. All the folks saying that actually writing code wasn't the bottleneck have had a LOT of years to get to that point, years a lot of people aren't interested in going through.
I'm not saying lay people will be able to join mega projects and work at FAANG. I'm saying a sufficiently courageous lay person will be able to make an app to automate whatever tedious birocratic process their niche work hasn't seen a product yet.
I can absolutely believe this, actually. Considering how the access to rapid prototyping has rejuvenated my interest in programming when I am a seasoned programmer, I definitely see the value in that kind of jump-start.
And yeah, without LLMs, writing code absolutely can be the bottleneck sometimes, if only for psychological reasons. I've definitely experienced something like "coder's block" before. An anti-flow state, if you will.
(But skydhash is also right. The "unknown unknowns" are a big deal and there's only so much an LLM can do about that, at least unless the user is actively aware of the general shape of that problem and trying to learn via the LLM and fill in those knowledge gaps.)
> All the folks saying that actually writing code wasn't the bottleneck have had a LOT of years to get to that point, years a lot of people aren't interested in going through.
The years was not spent on learning how to code. The years was spent on learning about what needed to be coded. And that would have spanned multiple domains. Even the most basic production level software have multiple concepts that need to be integrated. And for that, they need to be learned.
Using AI without the requisite knowledge is playing Russian roulette. It’s kinda wanting to log into a file, and not thinking about disk storage being finite, and rotating the log file gives you flexibility. That’s not related to coding, it’s just hard thinking about the problem and the constraints (or finding a good book or article).
> Using AI without the requisite knowledge is playing Russian roulette
Every few months that revolver has a few more empty chambers and even if it shoots them in the foot, it can bandage them quite nicely. Will a solo lay person with an agent build a high quality high reliability software business? Doubtful. Will they be able to make a tool that helps them and 10 other people or a personal website? probably.
> Will a solo lay person with an agent build a high quality high reliability software business? Doubtful. Will they be able to make a tool that helps them and 10 other people or a personal website? probably.
Just take a quick look around to see how many daily tools and utilities are actually built by experts and industries. Yes, you can put a plank on top of two concrete blocks to make a bench, but many people don’t and prefer to have good bench made, even when it’s quite easy to DIY one.
So I don’t foresee lots of people trying to make custom tools, more so to keep a subscription active.
just because you have an idea doesn't mean it will work. kids come up with ideas all the time in their drawings, but they disobey the laws of physics and are therefore not feasible in reality. sure, the model can guide them, but eventually they would have to pick up the necessary knowledge.
The necessary knowledge needed with LLMs goes from needing to know programming languages and libraries to having a high level view of what is possible which is far easier and less boring to get
> Although, part of what the dev does is figuring out the weeds, and only a part of users have the patience to sit through detail corrections. And the AI user has to verbalize their wants exactly too.
Even with basic pseudocode and very lenient logical rules, most people won’t bother to provide a a good definition of some process, as it’s quite unnatural to be that logical. Instead, it’s mostly just playing around. When you ask the one that is supposedly 10x-ing his productivity to explain his mental model of their own projects, it’s often a wishlist instead of a real understanding of what he has built.
> The struggle was never to learn programming syntax
For 85-90% of devs, yes this was the struggle. Creating and updating rows in a database from user input is all the “domain modeling” there is in most software. That’s what is solved by AI and it’s the majority of software dev.
Learning syntax takes a weekend. Learning to approach problems analytically and algorithmically takes much longer, and is the only "hard part" of computer programming.
It takes a weekend for a software developer. For a totally novice it takes a lot more time
The issue here is that you are learning the syntax at the same time you are learning foreign concepts, and the syntax is often confusing and counter intuitive
Learning the syntax separately is of no use because without a mental model you can't do anything with it
I remember learning python in high school by following the tutorial in the official docs, then from a book. Up until function and OOP, it was fairly easy, but I gave up afterwards because the exercises touch on things I haven’t learned yet like trigonometric identity. I could see and understand the syntax, but I couldn’t understand the code.
This is why children can easily learn enough programming to create games, because it’s a domain they can comprehend easily. Just like they can use english to write about their daily lives. It will be simple sure, but it will work.
So learning to code is fairly easy, you can easily create simple programs after a weekend. More complex programs will still use most of the same basic concepts as far as coding goes, but you will need to master more complex concepts, and that takes time. And such concepts are not about coding, they are about the nature of computing or the domain the software is created for.
This is why experienced devs only need a few days doing kata to get up to speed with a new programming language. It’s usually the same old, with slight differences. What was always needed was an understanding of computing and the ability to break down problems and create a solution that can be computed.
Coding to me is the same as drafting an engineering schematics, or writing a sheet music. It’s just a tool, not the end result.
Those who think the challenge is to overreach the limitations imposed by programming languages and types, etc, to do seemingly "magical" things. These are ones who would never understand why you might want higher-kinded types, the relative pros and cons of the functional vs imperative approaches, or even of type systems in general. They also tend to think the code written, in itself, is the valuable output of their work.
And then there are those who think the challenges is to model problem domain so well, using the language features, types and data structures, etc, that the solution (and further adjustments to it) becomes obvious and straightforward. They also tend to think the analysis and model that results is what is valuable.
Another way to see this difference: what impresses them? it is a hack that uses arcane knowledge to bypass software and hardware "limitation" or structures? Or is it code that is simple but still effective at solving complex problems in elegant ways, that may look simple but reflects a huge amount of knowledge, skill, and architectural intuition built up over many years?
I of course think that the second group are more effective and are doing proper programming, but many programmers and the general public think otherwise, or can't event tell the difference. I once worked with some colleagues could never see the light, despite a lot of support from me.
So here is the kicker, the dissonance in response to AI, I think, is kind of analogous to these two groups. The first groups likes that the AI will not stop them, but rather get them much quicker to their "magic", whereas the second group is more skeptical and sees many issues with AI not seeing the big picture and understanding deeper intent.
It certainly wasn't "approaching problems analytically and algorithmically" that prevented me to contribute to many open source projects in the past.
On the other hand, I've made 3 apps in 3-days' spare time, which I could not find in years and were not worth investing months of effort at the time.
I don't think the education barrier fully accounts for SWE salaries. It takes a particular disposition and other hard things, learned over years of doing, to actually keep do software engineering as a career.
AI changes the dispositions and difficulties, but it still favors a narrow selection of people to do the (still) difficult task of software engineering with agents.
First of all ppl are just processing the new enironment. So go easy on conclusions. Its all in flux.
Secondly most the posts on HN are from individuals. It doesnt represent group dynamics and how group/institutional knowledge works. For example there might be a regulation on publishing data every quarter. On the surface it looks like compliance but in reality different internal groups will hide data, delay it, fudge it etc etc. LLMs train on public data. Only after they are allowed access internally do they discover group knowledge. We are far away from seeing that happening.
I see what you mean, One thing I noticed is that the suggestions it gives are always generic and reproducible. If I want unique ideas I have not had good experience withh LLM's.
Fair. I'm not really expecting "unique ideas" from the LLM; I'm expecting it to help me examine my own ideas (e.g. by restating and expanding them, and sketching out an implementation plan) while simultaneously doing research. And from what I can tell, this aligns very well with the strengths of the technology.
If I'm prompting with phrases like "Any ideas?", it's because I'm exploring a domain where I know I don't know a lot, and where I expect that there is enough prior art to get a foothold.
If models are trained on human texts that contain expressions of introspection and emotions, why is it surprising that these models output similar texts? This endless anxiety about LLMs being conscious feels delusional and psychotic. Mr Olah thinks the models are 'alive' because he can't recognize his own reflection in LLMs
The endless insistence that LLMs cannot possibly be conscious just reeks of insecurity to me.
Humans once again insisting that there must be something very special to them. That they're not just animals, that they're not just matter, that they're not just a bunch of carbonhydrates on a space rock in a middle of nowhere in particular. This kind of insistence has a very bad track record.
The ugly truth is: we don't know.
We don't know what "consciousness" is, our best attempts to pin down the requisites may or may not be rooted in anything at all - and even by the metrics established by those attempts? Different theories on consciousness disagree on whether LLMs can be conscious.
So, when you say "they aren't duh", why are you saying that? Because you somehow know better than the sum of humanity's best attempts so far? Or because you know what answer you want to be true, truth be damned?
If AI is conscious then what about it is conscious? Because turn to turn it might be served from completely different GPUs and I can just change something in the context it is fed and it will have completely different output.
It is in the end a set of mathematical operations that if not queried will do nothing.
That is not true for any other animal on earth so whether you consider humans as superior to animals or not I would like the make the argument that something should at the very least be alive to be considered conscious, ignoring any scholarly definition of consciousness.
Pretty much if you consider an LLM conscious then my pocket calculator doing a mathematical calculation should be considered conscious by the same argument since it is a simpler version of an LLM.
This is a good take, but where does one consciousness end, and another begin?
Does my consciousness extend only to the central nervous system (and peripheral nerves and such are inputs)? Or am I my brain + all nerves? What about organs, does my consciousness include my liver? Is my liver another conscious entity?
Going the other way, if I'm using a computer, does my consciousness extend to the computer? I use a computer for well over 8 hours a day, I can use vim motions more naturally than, say, move my earlobes or control my toes. Does that make it part of my consciousness?
Sadly, I don't think any answer to these questions can be achieved in our current times, or maybe ever if consciousness is somehow non-physical. So even the very reasonable claim that consciousness is an statistically defined property (like temperature) is too far fetched for our current knowledge.
Indeed thats the difficulty. I used to think potentially conscious AI systems are so far into the far future scifi that I never have to bother, but unfortunately real life forced me to think of these much sooner than I wanted. Its no longer quite a purely academic question now anymore due to the potentiality of current AI being conscious.
One thing however we can be sure of, "supernatural" stuff is an extremely low to near impossible probability given current knowledge and the fact that history has always won the bet against supernaturalism throughout the centuries.
My bigger argument that is if I cease living the atoms that correspond to my body no longer has consciousness.
So is your argument that whenever a session ends or I interrupt a session that consciousness ceases for the LLM and it has died?
Then how do you correlate that with a consciousness not being able to be returned one dead? You can try to make the argument that consciousness is the firing of individual nuerons in a fixed system but we have never been able to model that and some pretty popular theories tend to believe that is a by product of consciousness and not the cause of it, once again not exactly possible to argue for or against.
There's also the wrinkle that consciousness might be a property that isn't reducible to some defined subset of the system. Imagine we have a haystack in a field. I can remove one piece of hay, and what we're left with remains a haystack. But repeat that process enough times and we'll eventually have something that's not a haystack, even though there's no unambiguous point where removing one bit of hay transforms a haystack into not-a-haystack.
That's not a problem. Physics has dealt with macro and statistical properties all the time, indeed that was the "wrinkle" that upended physics completely in the late 19th century and led to our modern theories. The thing you said can be said about temperature too. Which atom contains "20C" in a piece of metal? Its a statistical property of a large number of atoms or molecules, a concept that is the bread and butter of modern physics.
Have no idea why you’re being downvoted as this is a very valid point.
Also, in principle everything a GPU does could be done on pencil and paper. You could distribute the entire forward pass to the whole of humanity and have them write it out. Even if it took millions of years, the point is it’s computationally reducible to hand calculation. Is that entire process now “conscious” as well? The only difference would be the clock rate.
It’s an assumption, with the LLM it is a fact. There is only one source of entropy, whatever the seed value is to any pseudo random number generator used for stochastic operations. Given those values, everything is deterministic. For the brain, it’s not at all obvious that this is true.
I actually don't care if they are, I'm saying the topic is a distraction and irrelevant at best. As you say consciousness is not well defined. What is well defined and well grounded in reality is the disruption these models have on society. I think the consciousness debate AI researchers are having is a dangerous ego-trip for the sake of avoiding responsibility for the damage being caused by their work.
I think you don't appreciate the complexity of even the simplest multicellular organism and their training environment. Nothing to be ashamed of because no one understands that complelxity. The entire scientific and engineering domains only understand a fraction of it. If you want to talk about hubris, you should consider your own. To insist this sliver of computation is all that's needed for life to be conscious reeks of it.
Side-stepping this screaming match, what I do find interesting is that we do have real entities where we know for a fact that they are sentient, yet we eat them as steak, schnitzel and wurst.
The whole debate about hypothetical temporary sentience found in a certain ephemeral configuration of DRAM cells thus feels rather misguided to me.
It's always off-putting seeing AI proponents do this kind of strawman psychoanalysis and pretty toxic to reasonable discourse. I constantly see it used as a sort of prebuttal when layoffs/automation comes up to depict the other side as insecure/scared/"coping"/etc.
It comes across as smug and superior, conveniently dismissing any opposing arguments in advance by framing the interlocutor as emotional and irrational regardless of what they may say in response.
If you're going to try and debate something constructively this is not a good place to start from. I suggest avoiding making claims about the mental state and motivations of people you disagree with and instead explaining your position and asking why theirs differs. You may not agree with what they have to say but you might gain some better insights into why they think the way they do. It'll certainly be more fun than flinging insults back and forth anyway.
For my part, I'm quite certain that LLMs are not conscious despite being unable to define consciousness or specify any conclusive test for it. The reason these are not in conflict is down to the nature of consciousness and how it relates to knowledge and language.
Firstly, I know I am conscious with a higher degree of certainty than I know anything else because conscious experience is how I know anything else, up to and including that I exist (cogito ergo sum). For all I know every experience I have might be a trick. I might be a brain floating in a tank wired up to the Matrix, but even if so I am conscious.
Secondly, one of the things that I consciously experience is the relationship between consciousness itself and language in my mind. What I see is that language is smaller than, secondary to and somehow contained by consciousness. I consciously experience language but I also consciously experience much more than that, things that cannot be conveyed with language. Most of my experiences are like that, things that I could perhaps evoke in your mind by likening them to other, shared experiences, but that I could never concretely capture in language and unambiguously communicate. I know this because I experience those things and I experience language and I can perceive, directly, the categorical distinction between them.
To ask me to "define" or "prove" consciousness, then, is to ask me to capture in language the most extreme example of this categorical distinction. It's asking me to encapsulate in language not just one of the vast array of everyday experiences I have that cannot be so captured, but the entire thing, experience itself. It's just not possible.
But the fact I can't do this doesn't then imply that I don't know anything about consciousness and have no basis on which to hold a view such as "LLMs are not conscious". Indeed the reason I know I can't define consciousness is that I can perceive it and see that I can't. That same perception of and familiarity with consciousness is what enables me to form opinions about whether other people, creatures and things are likely to be conscious. I base these opinions on what I perceive of my own consciousness and what I know of how those other entities behave, came to be and work.
You can of course perfectly reasonably disagree with my opinions on those things, and you certainly have different conscious experiences than I do, but I hope the above makes it clear that the fact that I can't define or prove consciousness doesn't imply that I have no basis on which to form opinions about it.
Given the above it may surprise you to hear that I think true, conscious artificial intelligence is probably possible and that while I don't think it can be implemented by a classical algorithm I doubt it requires new physics or quantum mechanics. My speculation is that the continuum is both physically real and is (as it is represented in our existing physical theories) sufficient to allow it. Consciousness is larger than language but so is human understanding and if we keep going I expect it will eventually get us there. Nonlinear dynamics is where I'd look.
As for insecurity, I'm pretty sure spiders are conscious and that hasn't given me an inferiority complex, so I don't see why data centres being conscious would. I just don't think they are.
pretty sure an ant is more conscious than claude's latest model and I am pretty sure you are just completely missing the point and invented a fake argument to continue to miss the point
Is your comment anything other than an emotional outburst? Do you have a clear and rigorous and testable definition of consciousness? It strikes me as delusional when people talk about "consciousness" as if it were well defined and everyone had a clear concensus on it.
Strong opinions on this topic one way or another are usually mostly ungrounded.
Thats my point, why are we applying the ill defined concept of consciousness to a statistical model of language that outputs data similar to its training data?
Why are we applying ill defined concepts of consciousness to a collection of molecules sharing electrical signals that outputs similar data to what it was trained as?
Damn. I keep falling into this weird visual trance after about 4000 where I can't focus on the ball anymore and it ends up in the periphery of where I'm looking.
Playing that game reminded me of the level of concentration I needed to beat the levels in Battletoads in Battlemaniacs on the SNES. (I never played the NES version.)
I've heard a few times now that giving birth while lying on the back is a relatively modern invention and that for most of history women adopted squatting or leaning forward positions.[1] And that the back position is actually much more painful. How much does laying on the back increase the pain to the point where an epidural is necessary? Is it still necessary in the other positions?
Anecdotally from personal experience (gave birth to one child on back, the other in a squatting position, didn't have an epidural for either), the pain didn't depend on position - very painful both times! But pushing felt much much easier squatting than on my back.
The hospital only had two rooms suitably equipped for giving birth in a squatting position, so I was lucky to get one second time.
My wife works at a leading private maternity hospital in India. I can’t yet report from personal experience, but my wife says that epidurals are normal procedure, and per the hospital’s published statistics, about 90% choosing to give birth propped up or squatting (this figure excludes C-sections, water births, and complicated cases), which implies not being in a bed for at least the last part.
No, you can squat in the hospital bed. (Source: have given birth.) I'm not sure whether this is the same in India, but in the USA most hospitals don't want you to get out of bed post-epidural because you're a fall risk which incurs legal liability.
> My most successful work this year was hand-writing a tool, replacing ~4,000 lines of vibe-code with a few hundred. My system has proven more flexible, easier to test, and more generalizable than what it replaced.
> agents still need human taste and experience to produce good work.
Where do you think human taste and experience in software comes from?
> overstimulating & exhausting in a way that coding never was.
> for many of us, honing the code, finding the right design, seeing the parts mesh together — that was the meditative purpose we found in our jobs.
> That’s not the only time AI waylaid me. LLMs could not figure out how to annotate that model’s training data correctly. I spent days going back and forth with them
> LLMs are not yet good companions when brainstorming complicated things.
Why do we expect this to get better? The training data will degrade in quality if more people rely on LLMs so I'd expect models to eventually start getting worse even if they currently enjoy apparent improvements in output.
> Writing code is solved.
What a strange thing to say given all the above experiences, which I think we all share.
Why are we willing to give up so much in terms of purpose and understanding for perceived productivity gains?
reply