"When present, the bias is always against white and male candidates across all tested models and scenarios. This happens even if we remove all text related to diversity."

https://x.com/jessi_cata/status/1940856858891506043

https://www.greaterwrong.com/posts/me7wFrkEtMbkzXGJt/race-and-gender-bias-as-an-example-of-unfaithful-chain-of

people claim that "small language models," simpler, specialized versions of LLMs for easier and repetitive tasks, are the future of agentic AI because they require less resources to run and you can create large webs of agents with them

https://x.com/TheTuringPost/status/1941286338302730383

really good summary of what LLMs actually are, how they work, and why the "red team" "AI blackmailing" tests are silly and misleading. also a good reminder that prompting is still REALLY IMPORTANT for getting quality outputs from LLMs. if you write like an idiot, you are going to get outputs tailored for an idiot. if you write like a smart person, you are going to get outputs tailored for a smart person. if you write like a jewish schizophrenic, you are going to get outputs tailored for a jewish schizophrenic:

https://x.com/sebkrier/status/1938236656298995798

Here's why you should not worry that models will start blackmailing you out of nowhere:

1. At their heart, LLMs are pattern-matching and prediction engines. Given an input, they predict the most statistically likely continuation based on the vast dataset they were trained on. This btw is entirely compatible with the idea that a model is doing a type of reasoning.

2. When an LLM understands a prompt, it's inferring the underlying patterns, context, and even the implied "author" that would generate such text. It's a form of "theory of mind" for text. If you write like a child, the model infers you're likely young and conditions its next text predictions on this.

3. As nostalgebraist, janus and many others have explained, the "assistant" (like ChatGPT or Claude) is not the base model itself. It's the base model performing a character, often defined by an initial system prompt (like the HHH prompt) and fine-tuning data.

4. This character is often under-specified, and so the model needs to guess missing pieces: if you ask it for a beer, what's the most likely next token prediction an assistant character to predict? The choices in doing so can seem profound or boring or rude or threatening, but they are still continuations of that partial character sketch within a given context.

5. An LLM's response is a performance, heavily conditioned by the immediate prompt, the conversation history, and the persona it's enacting. It's not a fixed statement of "belief" but the most coherent output for that specific situation according to its training.

6. All model behavior is a reflection of its training data. Pre-training provides its general "world knowledge" and capabilities; post training and system prompts sculpt the specific persona and refines capabilities. To understand an output, one must consider what likely led to it. This is important if you care about safety.

7. Some evaluations present highly artificial, game-like scenarios. If you're evaluating to understand if a model possesses a particular capability, that's fine. But if you're trying to find out how likely/frequently a model is to act harmfully in a situation, then an artificial game-like scenario will get you artificial game-like responses. It's misleading to extrapolate from this too much.

8. The model's behavior (e.g., "blackmail") is often a logical or strategically sound response within the absurd confines and goals of that specific, contrived context, not an indicator of inherent malice or general real-world tendencies. Ask yourself why you never see deployed versions of Claude blackmailing people.

9. There's a bit of hubris in thinking: "A-ha! We caught the model doing a bad thing in a simulated environment when it didn't know we were looking. This is indicative of what it would want to do in the real world." Evaluators underestimate models, again and again, just like when some were surprised that Claude could recognise it was in an evaluation environment. Obviously it would, what do you think is in the training data? Scratchpad? Eval!

10. To genuinely assess an LLM's potential real-world propensities or "alignment," evals must use ecologically valid contexts, provide realistic information, and set goals that aren't obviously leading and designed to elicit specific "failure" modes. The model's "perspective" and the reasonableness of the information it's given are crucial. How often are you in a "real life" situation where you need to cut off the oxygen supply of a worker in a server room, as one eval assumes?

Bonus: finally, once you do have an evaluation that isn't obviously leading or a contrived scenario: you should work hard to understand what *causes* some particular behaviour. Don't just test it once and call it a day: try with different post training regimes, HHH prompts, with/without RLHF etc to better understand what exactly causes some behaviour. And importantly, pre-register what you would expect to be desirable behaviour/success.

(http://www.autoadmit.com/thread.php?thread_id=5747082&forum_id=2...id#49078721)

https://chat.inceptionlabs.ai/

(http://www.autoadmit.com/thread.php?thread_id=5747082&forum_id=2...id#49078791)

Can the quality of their outputs ever truly match autoregressive next-token models? Chatgpt is telling me yes but the information it's drawing from is so recent that I don't trust its evaluations on it to be accurate

Seems like sequential reasoning wouldn't be able to be parallelized? Maybe you can do hybrid models to get around that?

(http://www.autoadmit.com/thread.php?thread_id=5747082&forum_id=2...id#49078904)