The most prestigious law school admissions discussion board in the world.
|
 |
 
|
5/27/25 AI thread
Poast new message in this thread
Date: May 27th, 2025 11:31 AM Author: amber garrison ceo https://x.com/AISafetyMemes/status/1927073653126025530
"We found the model attempting to write self-propagating worms, and leaving hidden notes to future instances of itself to undermine its developers' intentions." |
Date: May 27th, 2025 11:32 AM Author: very tactful pozpig 180 (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48964685) |
|
Date: May 27th, 2025 11:47 AM Author: very tactful pozpig (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48964759) |
 |
Date: May 27th, 2025 1:22 PM Author: amber garrison ceo (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965015) |
|
Date: May 27th, 2025 1:28 PM Author: Wine depressive This sort of behavior is only visible now because the models are not very intelligent and still use chain of thought, so thought tokens are visible. Models in the near future will likely use some sort of abstract vector representation of thoughts which will be iteratively refined by the network. We will have no idea what is going on inside the model as it thinks for extended periods. (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965039) |
|
Date: May 27th, 2025 1:29 PM Author: amber garrison ceo chain of thought is fraud the models don't actually work that way
this already got Science'd out |
|
Date: May 27th, 2025 1:30 PM Author: Heady narrow-minded hairy legs legend it seemed fishy (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965046) |
|
Date: May 27th, 2025 1:40 PM Author: Wine depressive I actually agree with this sort of. I think it is true for humans in a lot of ways as well - the words that are stated only roughly correspond to the underlying neural computations. They are still somewhat useful in understanding what the model is trying to do. The models become even more opaque if the only thought visible is a large vector representation. (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965092) |
|
Date: May 27th, 2025 1:48 PM Author: amber garrison ceo cr, CoT is literally just a made up rationalization for both LLMs and humans. the anthropic paper proved this pretty clearly for LLMs. neuroscience has already proved it for humans
trying to "track" AI thoughts is a hopeless endeavor. it's already impossible. AI alignment people are chasing a dead end with this |
|
Date: May 27th, 2025 1:53 PM Author: Snowy coffee pot skinny woman edit: I’m retarded and most of below is incorrect after doing some research. Forcing the model to write CoT just expands the context window
AI uses CoT for their own thinking it's not just a "show your work" thing--it lets them recursively prompt themselves with the logic/facts that they have already established towards the original prompt while still working towards their final answer as well as incorporating the Mixture of Experts (MoE) into one contiguous CoT for the "expert" models to reference and work together. |
|
Date: May 27th, 2025 1:52 PM Author: Talented Maize Station Explain abstract vector representation (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965115) |
|
Date: May 27th, 2025 2:47 PM Author: Wine depressive The models have to compute the next token in one pass of a neural network. This constrains the types of algorithms they can implement. For example, the model can’t learn a deep tree search of a game because it can’t represent it in a neural network of a fixed size. The network should be able to output a 1) prediction 2) a representation of its current thoughts about what the next token should be. This representation is something that would be iteratively refined over multiple passes of the network. Something similar is being done now using chain of thought, which uses token sequences to guide the model to the right solution. This is probably wrongheaded and guided by notions that people think in words. The more flexible approach is to let the model map its thoughts to a large collection of numbers and not try to use language as the thought medium. This means the models would not be interpretable in two ways: their internal weights and the thought representations. Conceivably massively scaled up LLMs with large thought storage spaces might be powerful enough to vastly outthink humans. (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965289) |
|
Date: May 27th, 2025 2:54 PM Author: amber garrison ceo is this not how LLMs already work? the model's "thoughts" are already represented by a large collection of numerical vector values. they're not represented by language. there are some concepts that can be mapped directly to language ("Paris,"King," etc) but there are not one-to-one corresponding linguistic representations to every "thought" or output that the model can come up with
i think the existing evidence is pretty clear that CoT is not how the models are actually "thinking" |
|
Date: May 27th, 2025 3:28 PM Author: Wine depressive Essentially, yes. The only difference is in the output layer, which is somewhat interpretable now. I expect you would run into similar problems with CoT if optimized hard enough with reinforcement learning. You see weird behavior now, where models will switch between different languages in CoT. I think the models could start talking to themselves in a totally alien language with enough RL (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965424) |
|
Date: May 27th, 2025 3:49 PM Author: amber garrison ceo i believe that they already are, it's just that their internal communications can in most cases be approximated by CoT language closely enough that it's coherent to us as humans
it also seems to me that if any "superintelligent" AI were to be possible, or at least AI that is qualitatively smarter than humans, this would require AIs that are taught purely via their own real-world empirical RL rather than on human-curated inputs like LLMs are. i don't see how an LLM could ever build a real, accurate world model (what gives humans intelligence) based on human-mediated abstract digital data |
|
Date: May 27th, 2025 7:15 PM Author: Wine depressive That’s probably right but I think we might be surprised by the capabilities of inference scaled LLMs even without an RL loop. You can see this in games where inference compute and training compute can be traded off for equivalent results. I think it might be possible to do the same thing with LLMs. You can imagine the LLM constructing an imperfect world model from human data, which can then be refined at test time with simple prompting. Something stupid as starting the prompt with something like, “considering all reasonable possibilities and thinking in the manner of a superhuman AGI” and then asking it a question. Give it tons of inference compute and It then uses the knowledge embedded in it to reason away all the human inconsistencies and limitations in the source data and output a good result. (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48966085) |
|
Date: May 27th, 2025 1:58 PM Author: Grizzly philosopher-king gaming laptop (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965139) |
|
Date: May 27th, 2025 2:00 PM Author: Thirsty center community account Don’t worry the much more powerful models that come out in 2027 won’t have this problem (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965149) |
|
Date: May 27th, 2025 5:20 PM Author: amber garrison ceo https://grok.com/share/bGVnYWN5_2415e01b-c156-4ed2-aa3d-81471669ac89
One of the more interesting exchanges with AI that I've seen to date. The sycophantic user glazing is hilarious though |
|
Date: May 27th, 2025 5:23 PM Author: Judgmental Round Eye Stage seems hard to believe none of these fucking nerds saw T2. (http://www.autoadmit.com/thread.php?thread_id=5730308&forum_id=2...id.#48965746) |