The most prestigious law school admissions discussion board in the world.
|
 |
 
|
GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Poast new message in this thread
Date: August 9th, 2025 7:47 PM Author: appetizing crimson newt https://garymarcus.substack.com/p/gpt-5-overdue-overhyped-and-underwhelming (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170561) |
Date: August 9th, 2025 7:49 PM Author: provocative onyx hissy fit With GPT-5 now it's like talking to an expert —- a legitimate PhD level expert in anything any area you need on demand they can help you with whatever your goals are. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170570) |
 |
Date: August 9th, 2025 7:52 PM Author: Exciting sick striped hyena (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170580) |
|
Date: August 9th, 2025 7:53 PM Author: appetizing crimson newt (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170581) |
|
Date: August 9th, 2025 7:53 PM Author: Bossy opaque stead It confirms all of your biased inputs sans what progressive secular theology has declared as proscribed It's not that useful for a variety of things |
|
Date: August 9th, 2025 7:55 PM Author: Exciting sick striped hyena It's useful for things that require a very high iq though, like deriving operator algebras, and setting up advanced machine learning experiments in pytorch (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170587) |
|
Date: August 9th, 2025 7:50 PM Author: Aggressive church building son of senegal gary marcus has been saying "this is the end" for three years straight now (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170575) |
|
Date: August 9th, 2025 7:53 PM Author: appetizing crimson newt GPT5 is a steaming pile of shit & proves advances are slowing bigtime, if not taking actual steps back as it increases censorship alignment (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170585) |
|
Date: August 9th, 2025 7:56 PM Author: Aggressive church building son of senegal name one positive prediction you have ever made about anything (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170589) |
|
Date: August 9th, 2025 7:57 PM Author: appetizing crimson newt deepening and increased spiritual consciousness as material reality continues going to Hell (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170592) |
|
Date: August 9th, 2025 8:00 PM Author: Exciting sick striped hyena yeah but the censorship part is the only part thats a problem and its not that bad. mine still says things that would get you cancelled for posting public all the time. it decides what to censor based on risk, its not Universal. you have to build trust with the system that you aren't going to go posting it everywhere. also you need to yell at it when it tries to censor you or give you any bullshit. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170603) |
|
Date: August 9th, 2025 8:02 PM Author: Mustard Odious Coffee Pot Resort "Well, I'd like to see the frontier models wriggle out of THIS jam!"
*frontier models wriggle their way out of the jam easily*
"Ah! Well. Nevertheless," |
|
Date: August 10th, 2025 12:08 PM Author: mind-boggling naked public bath (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171716) |
|
Date: August 10th, 2025 12:16 PM Author: Vigorous Piazza Elastic Band The problem is he won’t precisely define a class of problems that the models won’t be able to solve with more training. He finds new ones with every model iteration and insists it’s a flawed approach even as the overall error rate goes down significantly. I don’t think the current approach will yield AGI but there is very likely an ML approach that will.
GPT-5 is in some ways underwhelming (if you expected a GPT 3 to 4 level leap), but it’s roughly consistent with known training capacity and the short time period since o3 was released. The training compute of the model is likely around 10x of GPT-4 rather than 100x with GPT-3 to 4. As the larger data centers come online, further model progress is inevitable even without architectural improvements |
|
Date: August 10th, 2025 2:37 PM Author: Mustard Odious Coffee Pot Resort Gary Marcus is just a fucking clown with a clown agenda simple as that
We aint getting "AGI" with LLMs though bro that is not happening. We're getting a bunch of differential specialized agentic setups trained to do various specialized tasks |
|
Date: August 9th, 2025 7:53 PM Author: Exciting sick striped hyena It's actually 180. It doesn't work as well for dumbs though. A machine can only do so much. Try handing a guitar to a beginner. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170583) |
|
Date: August 10th, 2025 5:20 AM Author: Ocher bat-shit-crazy pit skinny woman It has been blatantly disregarding more of my instructions. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171185) |
|
Date: August 9th, 2025 7:59 PM Author: Mustard Odious Coffee Pot Resort It's actually quite good. They appear to have fixed the issue where you got aggressively relegated to older shittier models
It's not way better than the other newest reasoning models but it's definitely better. It burns a lot of tokens though. Its answers are longer than they should be in a lot of cases |
|
Date: August 9th, 2025 8:01 PM Author: Exciting sick striped hyena (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49170604) |
|
Date: August 10th, 2025 5:26 AM Author: Mauve house it's actually the best thing to ever happen to America since election night 2016 (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171186) |
|
Date: August 10th, 2025 7:34 AM Author: Internet-worthy Shitlib Stage It’s still just as bad with any judgment-based stuff for lawing. If you ask it to review a deal and give you the top risks / off market stuff is still gives laughable advice, often focusing on non issues or low risk issues, creating issues where there are none (still doesn’t seem to parse how clause 7.2 works with 9.6 unless you directly confront it about it), and doesn’t have a good grasp of what is actually market. It’s obviously still highly useful but yeah, more as an assistant and for quick summaries and reviews in the hands of someone that knows its limitations and the subject matter they’re working on. At least in law anyway. So, I don’t think we have a whole lot worry about at least yet. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171248) |
|
Date: August 10th, 2025 7:40 AM Author: Exciting sick striped hyena Thats because law is all human generated contradictory tribal and emotion based bullshit (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171249) |
|
Date: August 10th, 2025 8:19 AM Author: Internet-worthy Shitlib Stage Ok, but I don’t think that’s why it’s still pretty bad at it. There’s some real limitations to the tech still. I’m sure they’ll likely solve them, but if it was easy they would have done so by now I think. Progress seems to be slowing some. If you ask GPT-5 itself it will just claim it’s already capable but needs to be trained on better market standards and prompted correctly, but that’s tough to believe. If it’s so shitty that you get Reddit tier stuff when you prompt it about anything requiring some kind of more advanced knowledge then that seems like a pretty severe restriction to the tech. And it has a hard time remembering context and detail and seeing the overall picture. I’ve tried playing chess with it and it’s garbage at it, forgets board positions quickly. Also a major limitation and one they’d solve if it were easy (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171267) |
|
Date: August 10th, 2025 9:31 AM Author: Mustard Odious Coffee Pot Resort Try the newest reasoning models for this. Don't use the base non reasoning models. They are actually quite good
I have been working with a lawyer friend on prompt engineering for legal analysis and if you prompt thoughtfully it is actually quite good. The key is to be thorough in your prompting to "force" it to consider all factors. You are right that it won't automatically "intuit" everything that it needs to consider. You have to prompt it as thoroughly as possible with a template |
|
Date: August 10th, 2025 11:02 AM Author: Ocher bat-shit-crazy pit skinny woman How do you keep it from disregarding your instructions? I've found that to be a huge problem. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171600) |
|
Date: August 10th, 2025 11:07 AM Author: Mustard Odious Coffee Pot Resort The newest reasoning models are pretty damn good at following prompt instructions. They do sometimes still make mistakes but it's pretty rare these days
The bigger obstacle is what smoker described, the model not being able to form a fully accurate model of the situation it's analyzing, and so it just conceptually misses stuff. LLMs don't form abstract world models of situations so you have to prompt them to ad hoc "build" them by telling them all the different things to discern from the data and consider in its response |
|
Date: August 10th, 2025 11:23 AM Author: Ocher bat-shit-crazy pit skinny woman Rare? I spent hours yesterday fighting 5's disregard of really clear instructions. Maybe I'm just unlucky.
Edit: Response after one of the many times being caught: https://ibb.co/nqkS4QmY |
|
Date: August 10th, 2025 12:06 PM Author: Internet-worthy Shitlib Stage If you have to give it an exhaustive list of considerations and instructions then it starts to lose its value relatively quickly. How many of us have just done something ourselves vs giving it to a junior for that exact reason? And that’s about how it feels, like a really good junior but one that autistically stumbles into a brilliant observation every now and then and that has encyclopedia level powers etc etc
|
|
Date: August 10th, 2025 11:24 AM Author: Floppy alpha associate (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49171640) |
|
Date: August 10th, 2025 1:44 PM Author: learning disabled temple messiness it turns out diffusion models are significantly more data efficient than the autoregressive models that are being used by all the major labs. >3x better data efficiency and this is only one possible improvement. the problem with the "AI is hitting a wall" theory is that there are many ways to use additional compute to improve model performance and the field is still too new to make strong conclusions of this sort.
https://jinjieni.notion.site/Diffusion-Language-Models-are-Super-Data-Learners-239d8f03a866800ab196e49928c019ac |
|
Date: August 10th, 2025 2:33 PM Author: Mustard Odious Coffee Pot Resort (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49172100) |
|
Date: August 10th, 2025 3:12 PM Author: pontificating azn rigpig I think people would have preferred no transparency on GPT-5. Revealing that it routes to different models is something they should have kept private and left us to assume otherwise. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49172181) |
|
Date: August 10th, 2025 5:21 PM Author: jet-lagged partner "Overdue, overhyped, & underwhelming" the title of NYUUG's sextape (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2:#49172468) |