Nim is stack allocated by default, and you annotate types explicitly with `ref` to use the GC. Even then, the compiler elides reference counting using borrow checks, you can choose different GC options, and the GC isn't stop the world like most. You don't need to touch reference counting if you don't want to, and Nim fits embedded dev pretty well for that reason.
LLMs don't produce text at all, they produce probabilities of tokens. Tokens aren't text, they're high dimenensional coordinates in a latent "concept space".
These are displayed to us as text, but this distinction is important when you think about what they're actually doing, which is closer to building and transforming concept geometries.
No language models are programmed, they are "grown" or evolved from data.
There's no print statements or human entered logic involved in the raw model expression at all.
The only thing that humans have programmed is efficient parallel dot product pipelines that "animate" (for lack of a better word) the models.
Everything these models do is emergent from their backpropgation guided evolution. This even includes in context learning itself, which was not an expected outcome.
They aren't grown/evolved from data, they are fit to the data. The fitting process can be fully deterministic although its fairly easy to screw things up such that it isn't deterministic, but that just a defect not some fundamental shift.
You have completely misunderstood what I was saying so badly I can't even formulate a response other than to suggest you read my reply again. I was not suggesting that LLMs are programmed with print statements, for fuck's sake.
This perspective that consciousness cannot be programmed can only make sense if you're a dualist. We don't know how consciousness arises. If you're a naturalist it can't be ruled out based on the simplicity of the algorithm.
> When you write a program to predict tokens based on context, seeding its context with something that makes it predict "self-reflecting" text is trivial. Program does what it is programmed to do. Would observing the output of the following program inspire doubt as to its sentience?
Then you follow it up with print statements as if that is a good analogy.
As I said, they are not programmed, so your question above is not relevant to your argument.
You say they're programs that are stochastically jiggled, but that's simply not accurate either. All LLM abilities are emergent, even when the training corpus is well defined.
I didn't think you literally thought they were made of print statements, but you are implying they're software that's been "fuzzed". Hopefully you don't literally that either and you're just using it as a bad analogy.
You could have argued from the stance of neural networks being universal functions, which might at least be closer to the truth, but instead your example is print statements!
I get you're trying to say that something trained to say a thing doesn't mean it has arrived at the thing like a mind would, and perhaps that would have been closer for GPT 2.
These days though, we just have so much more awareness of what they're actually doing internally that it's bizarre to even compare them to stochastic parrots of the training corpus, if that is closer to what you're implying.
First you run a program (training framework) to generate a database of values.
Then you run a program (inference engine) which performs calculations against the database of values.
To put it in ELI5 terms: run a program against a book, counting how many times "I love <x>" appears in the book. Note "dogs" 4 times, "cats" 5 times, "you" 1 time into a database. Then run a program against that database. When inputting "I love" as the preceding text, the second program determines the most likely result is "cats" and returns "I love cats" (or returns "I love cats" 50% of the time, or dogs 40% of the time, or you 10% of the time, or some variation by different methods of weighting).
Yes, this is an extreme simplification. Yes, the model is not technically a database either. But this is fundamentally the process followed. You would consider it a single program if the training framework and inference engine were part of the same software and stored the computed training values to memory instead of disk, taking an input dataset and an input context as params and returning "I love cats" as the output. There's all kinds of incredibly sophisticated techniques applied on top of this foundation to vastly improve the statistical modeling and efficiency, but the underlying basics have not fundamentally changed.
> Then you follow it up with print statements as if that is a good analogy.
The print statements were not an analogy. They were pointing out the ridiculousness of doubting whether software is conscious because it generated self-referential text. Gettting software to generate self-referential text is as easy as `print(self_referential_text)`. So the only question is how the self-referential text is generated. For self-referential text generation to be more interesting than passing it as a literal print value, there would have to be some really wondrous "how" going on. But, it turns out, the "how" of an inference engine isn't that much more interesting than literally doing a `print`.
What if we can interact with this video, and it reacts in the same ways the source organism does?
Then we put it in new situations that weren't in the source video, and it interacts in a similar way to the original organism in these situations, too.
What do we make of reactions of pain or joy? Where's the line between simulation and enaction?
This is closer to the reality of these models.
I'm not suggesting I know where that line is - if indeed it is a line at all - it could well be a gradient.
LLMs are deterministic, though. Much like the video.
AFAIK using the same input tokens, weights, and numerical operations will lead to the same probability distribution for the next token. It uses pseudo-randomness to enable temperature, etc. Like a fuzzy video.
"Markers that would indicate consciousness if observed in a biological organism" just does not mean very much. A PR phrase used to hype the IPO.
No, that is not at all something we can just state as a fact. Whether the brain is deterministic is an open question that just inherits the good old, probably unsolvable determinism debate.
The LLM pseudo-randomness from above is engineered by us humans and fully understood, much like an algorithm playing a video frame sequence.
You could theoretically record a full register of all states of an LLM setup with all the possible inputs and environment parameters, and it would fully describe everything you would ever get from a given LLM setup. It would be a very large, convoluted book.
I understand that Anthropics PR department wants to see truth or reason behind every "I'm alive" the LLM generates. Even the term "self-report" is anthropomorphizing, as an LLM does not do anything on its own at all. (It also does not hack any company on its own.) That is just one of the narratives they spin probably at least until the IPO.
Nonsense. There was one proposal for relevant quantum effects in brain dynamics, and that turned out to be not relevant. Even if they were, you could substitute all quantum randomness with pseudo randomness and obtain an absolutely indistinguishable object.
But even if this were a debate, its absolutely absurd to claim that the question of determinism in the brain has any bearing on our moral standing. If we discover tomorrow that quantum collapse is deterministic and can be derived from an underlying theory, and thus all of physics is deterministic in the good old fashioned Newtonian sense, this would not affect our moral standing in the least.
You seem to have conceded the "is it deterministic" argument only to sidestep by declaring determinism irrelevant. Your original claim was that LLMs are deterministic "in the same sense" as brains.
We can write down an LLMs full register, and that register/book contains the whole output universe of the text generator. That book does not act, it is morally neutral. That the brain has such a register at all is just restating the determinism axiom, which you treat as fact.
A text is not conscious, and we can not wish it into consciousness, no matter how many human-like patterns we find in the book / the generated text. It has not been shown that running the text adds anything over the text written out. Researchers are super motivated to find machine consciousness but cannot find it, while a company months from its IPO keeps pitching shadows of consciousness all day. It really is a PR strategy.
It does have bearing if your definition of consciousness rests on free will and you think that's incompatible with determinism. Now I don't think a lot of people seriously believe that [1] but it's not some logical nonsense.
[1] Off topic: I think most people are really compatibilist but a lot of them (like me) also believe in non determinism. Not believing in free will is really rare.
I have never come across this position argued seriously. I would also consider it absurd, as then the question whether we have consciousness depends on unknown properties of fundamental physics (which is not incompatible with determinism, see e.g. Bohms theory). It would therefore be unknown whether humans are conscious. That is at the very least a notion of consciousness that is utterly distinct from any established meaning of the word.
That's not to say this project isn't cool, though. As usual with Rust projects, it's a bit breathy with hype (eg "sophisticated conditional compilation patterns" for cfg(feature)), but it seems well developed, focused, and most importantly, well documented.
It also shows some positive signs of being dog-fooded, and the author(s) clearly intend to use it.
Unifying GPU back ends is a noble goal, and I wish the author(s) luck.
Unfortunately, while OOP promises code reuse, it usually makes it worse by introducing boundaries as static architecture.
OOP's core tenet of "speciating" processing via inheritance in the hope of sharing subprocesses does precisely the opposite; defining "is-a" relationships, by definition, excludes sharing similar processing in a different context, and subclassing only makes it worse by further increasing specialisation. So we have adapters, factories, dependency injection, and so on to cope with the coupling of data and code. A big enough OOP system inevitably converges towards "God objects" where all potential states are superimposed.
On top of this, OOP requires you to carefully consider ontological categories to group your processing in the guise of "organising" your solution. Sometimes this is harder than actually solving the problem, as this static architecture has to somehow be both flexible yet predict potential future requirements without being overengineered. That's necessary because the cost to change OOP architectures is proportional to the amount of it you have.
Of course, these days most people say not to use deep inheritance stacks. So, what is OOP left with? Organising code in classes? Sounds good in theory, but again this is another artificial constraint that bakes present and future assumptions into the code. A simple parsing rule like UFCS does the job better IMHO without imposing structural assumptions.
Data wants to be pure, and code should be able to act on this free-form data independently, not architecturally chained to it.
Separating code and data lets you take advantage of compositional patterns much more easily, whilst also reducing structural coupling and thus allowing design flexibility going forward.
That's not to say we should throw out typing - quite the opposite, typing is important for data integrity. You can have strong typing without coupled relationships.
Personally, I think that grouping code and data types together as a "thing" is the issue.
> Data wants to be pure, and code should be able to act on this freeform data independently, not architecturally chained to it.
If behaviors are decoupled from the data they operate on, you risk a procedural programming style lacking the benefits of encapsulation. This can increase the risk of data corruption and reduce data integrity...
Behaviours don't have to be decoupled from the data they operate on. If I write a procedure that takes a particular data type as a parameter, it's a form of coupling.
However, there's no need to fuse data and code together as a single "unit" conceptually as OOP does, where you must have particular data structures to use particular behaviours.
For example, let's say I have a "movement" process that adds a velocity type to a position type. This process is one line of code. I can also use the same position type independently for, say, UI.
To do this in an OOP style, you end up with an "Entity" superclass that subclasses to "Positional" with X and Y, and another subclass for "Moves" with velocity data. These data types are now strongly coupled and everything that uses them must know about this hierarchy.
UI in this case would likely have a "UIElement" superclass and different subclass structures with different couplings. Now UI needs a separate type to represent the same position data. If you want a UI element to track your entity, you'd need adapter code to "convert" the position data to the right container to be used for UI. More code, more complexity, less code sharing.
Alternatively, maybe I could add position data to "Entity" and base UI from the "Positional" type.
Now throw in a "Render" class. Does that have its own position data? Does it inherit from "Entity", or "Positional"? So how do we share the code for rendering a graphic with "Entity" and "UIElement"?
Thus begins the inevitable march to God objects. You want a banana, you get a gorilla holding a banana and the entire jungle.
Meanwhile, I could have just written a render procedure that takes a position type and graphic type, used it in both scenarios, and moved on.
What do I gain by doing this? I've increased the complexity and made everything worse. Are you thinking about better hierarchies that could solve this particular issue? How can you future proof this for unexpected changes? This thinking process becomes a huge burden to make brittle code.
> you risk a procedural programming style lacking the benefits of encapsulation. This can increase the risk of data corruption and reduce data integrity...
You can use data encapsulation fine without taking on the mantle of OOP. I'm not sure why you think this would introduce data corruption/affect integrity.
There's plenty of compositional and/or functional patterns beyond OOP to use beyond procedural programming, but I'd hardly consider using procedural programming a "risk". Badly written code is bad regardless of the pattern you use.
That's not to say procedural programming is all you need, but at the end of the day, the computer only sees procedural code. Wrapping things in objects doesn't make the code better, just more baroque.
OOP and especially post-OOP languages don't encourage the "Dog is-a Animal" type of inheritance that you describe. Sadly, education has not caught up to industry and so it is still often taught the wrong way. Composition-over-inheritance has been the dominant methodology of practical OOP for a long time, so much so that most post-OOP languages (Swift, Rust, Go, ...) have dropped inheritance entirely, while still preserving the other aspects of OOP, like encapsulation, polymorphism, and limited visibility.
Although it's not as extensive as Rust's lifetime management, Nim manages to infer lifetimes without specific syntax, so is it really a syntax issue?
As you say, though, C++ template magic definitely has its limits.
Nim is stack allocated unless you specifically mark a type as a reference, and "does not use classical GC algorithms anymore but is based on destructors and move semantics": https://nim-lang.org/docs/destructors.html
Where Rust won't compile when a lifetime can't be determined, IIRC Nim's static analysis will make a copy (and tell you), so it's more as a performance optimisation than for correctness.
Regardless of the details and extent of the borrow checking, however, it shows that it's possible in principle to infer lifetimes without explicit annotation. So, perhaps C++ could support it.
As you say, it's the semantics of the syntax that matter. I'm not familiar with C++'s compiler internals though so it could be impractical.
I did not hear that Nim made ORC the default, thanks for that!
I still think that my overall point stands: sure, you can treat this as an optimization pass, but that kind of overhead isn't acceptable in the C++/Rust world. And syntax is how you communicate programmer intent, to resolve the sorts of ambiguous cases described in some other comments here.
> Where Rust won't compile when a lifetime can't be determined, IIRC Nim's static analysis will make a copy (and tell you), so it's more as a performance optimisation than for correctness.
Wait, how does that work? For example, take the following Rust function with insufficient lifetime specifiers:
pub fn lt(x: &i32, y: &i32) -> &i32 {
if x < y { x } else { y }
}
You're saying Nim will change one/all of those references to copies and will also emit warnings saying it did that?
It will not emit warnings saying it did that. The static analysis is not very transparent. (If you can get the right incantation of flags working to do so and it works, let me know! The last time I did that it was quite bugged.)
Writing an equivalent program is a bit weird because: 1) Nim does not distinguish between owned and borrowed types in the parameters (except wrt. lent which is bugged and only for optimizations), 2) Nim copies all structures smaller than $THRESHOLD regardless (the threshold is only slightly larger than a pointer but definitely includes all integer types - it's somewhere in the manual) and 3) similarly, not having a way to explicitly return borrows cuts out much of the complexity of lifetimes regardless, since it'll just fall back on reference counting. The TL;DR here though is no, unless I'm mistaken, Nim will fall back on reference counting here (were points 1 and 2 changed).
For clarity as to Nim's memory model: it can be thought of as ownership-optimized reference counting. It's basically the same model as Koka (a research language from Microsoft). If you want to learn more about it, because it is very neat and an exceptionally good tradeoff between performance/ease of use/determinism IMO, I would suggest reading the papers on Perseus as the Nim implementation is not very well-documented. (IIRC the main difference between Koka and Nim's implementation is that Nim frees at the end of scope while Koka frees at the point of last use.)
Oh, that's interesting. I think not distinguishing between owned and borrowed types clears things up for me; it makes a lot more sense for copying to be an optimization here if reference-ness is not (directly?) exposed to the programmer.
Thanks for the explanation and the reading suggestions! I'll see about taking a look.
To be fair, you've posted a toy example. Real games are often chains of dependent systems, and as complexity increases, clean threading opportunities decrease.
So, while yes it's nice in theory, in practice it often doesn't add as much performance as you'd expect.
reply