Can a system based on a language model be conscious?

(0) No

(1) Yes, but it hasn't happened yet

(2) Yes, and such a system has been created

(3) Other

Discuss.

(http://www.autoadmit.com/thread.php?thread_id=5862594&forum_id=2)#49857985)

One of my main criticisms has always been that existing systems can't self-modify. Interestingly, OpenClaw bots can edit their own soul.md files, so maybe we're getting closer (or maybe it just looks like it). They're not updating their internal models, so it's different from neuroplasticity, but they are making decisions about how to act in the future. Even if we did say that rewriting on that level counts as self-modification, though, these bots are seeded with a soul.md written by a human, so I wouldn't buy that the changes demonstrate actual will. More info on soul.md: https://soul.md

The other major limiting factor IMO is that context windows just aren't big enough for actual consciousness to arise. A conscious system would need a much more robust memory than what's available. In theory maybe it could get there with a ton of DB storage. But right now, even if you take the info in the model AND the info stored by an agent, it's just not anywhere near comparable to what's encoded in the human nervous system.

THAT SAID, I think it's theoretically possible. Humans' personalities are a combination of our genetics, our experiences, and our interpretations of those experiences: what we remember about them and how that shapes our outlook/future decisions. An AI agent is a combination of its underlying model, its experiences (interactions with the world – right now just text and images, but in the future they could perceive more data about the physical world), and interpretations of those experiences: what it remembers and how that shapes its decisions going forward. In a sufficiently complex system, maybe that leads to a thinking, self-aware being with emergent preferences and desires.

I am not remotely convinced that consciousness can only arise from human or even biological minds. The thought experiment that solidified that for me was when I considered a consciousness spread over galaxies, with nodes communicating via electrical impulses, similar to our nervous system. Of course that is a much, much closer analog to the human brain (that's the whole point) than an LLM, but I still find it a useful intuition pump for people who can't conceive of a non-biological mind. There's also Chalmers' gradual-neuron-replacement thought experiment. But here, too, synthetic consciousness being possible != AI consciousness being possible.

(http://www.autoadmit.com/thread.php?thread_id=5862594&forum_id=2)#49858075)

See the paper I recently posted about on this topic

(http://www.autoadmit.com/thread.php?thread_id=5862594&forum_id=2)#49858088)

No the one about the paper explaining that subjective conscious experience requires a continuous mapmaker

(http://www.autoadmit.com/thread.php?thread_id=5862594&forum_id=2)#49858237)

it's designed to always answer prompts in the most conventional way possible because that's the only way it can work. they're powerful because with scaling more and more things start showing up as "latent knowledge" in its memory banks like e.g. mathematical concepts, and because many domains don't need actual creativity or awareness to solve problems; you just need the right abstractions + banging shit together. but its concepts tend to be very ad hoc, and domains where they can be honed to perfection through reinforcement learning are the exception.

https://old.reddit.com/r/ClaudeCode/comments/1sj9ab7/opus_45_vs_opus_46/

per the above, test how it experiences normal human situations and it can't contextualize anything adequately at all (both models' answers are equally retarded IMO). this is because a language model not only isn't conscious, it's a paradigm that doesn't even attempt to simulate consciousness unlike e.g. robotics.

(http://www.autoadmit.com/thread.php?thread_id=5862594&forum_id=2)#49858683)

I feel like the world is entirely simulated, so our concept of “consciousness” is a trifle in the grand scheme.

(http://www.autoadmit.com/thread.php?thread_id=5862594&forum_id=2)#49858754)