The most prestigious law school admissions discussion board in the world.
|
 |
 
|
GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Poast new message in this thread
Date: August 9th, 2025 7:47 PM Author: irradiated casino https://garymarcus.substack.com/p/gpt-5-overdue-overhyped-and-underwhelming (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170561) |
Date: August 9th, 2025 7:49 PM Author: Tan angry theater With GPT-5 now it's like talking to an expert —- a legitimate PhD level expert in anything any area you need on demand they can help you with whatever your goals are. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170570) |
 |
Date: August 9th, 2025 7:52 PM Author: drab location (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170580) |
|
Date: August 9th, 2025 7:53 PM Author: irradiated casino (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170581) |
|
Date: August 9th, 2025 7:53 PM Author: house-broken theater stage It confirms all of your biased inputs sans what progressive secular theology has declared as proscribed It's not that useful for a variety of things |
|
Date: August 9th, 2025 7:55 PM Author: drab location It's useful for things that require a very high iq though, like deriving operator algebras, and setting up advanced machine learning experiments in pytorch (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170587) |
|
Date: August 9th, 2025 7:50 PM Author: Maroon heady temple volcanic crater gary marcus has been saying "this is the end" for three years straight now (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170575) |
|
Date: August 9th, 2025 7:53 PM Author: irradiated casino GPT5 is a steaming pile of shit & proves advances are slowing bigtime, if not taking actual steps back as it increases censorship alignment (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170585) |
|
Date: August 9th, 2025 7:56 PM Author: Maroon heady temple volcanic crater name one positive prediction you have ever made about anything (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170589) |
|
Date: August 9th, 2025 7:57 PM Author: irradiated casino deepening and increased spiritual consciousness as material reality continues going to Hell (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170592) |
|
Date: August 9th, 2025 8:00 PM Author: drab location yeah but the censorship part is the only part thats a problem and its not that bad. mine still says things that would get you cancelled for posting public all the time. it decides what to censor based on risk, its not Universal. you have to build trust with the system that you aren't going to go posting it everywhere. also you need to yell at it when it tries to censor you or give you any bullshit. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170603) |
|
Date: August 9th, 2025 8:02 PM Author: Titillating halford azn "Well, I'd like to see the frontier models wriggle out of THIS jam!"
*frontier models wriggle their way out of the jam easily*
"Ah! Well. Nevertheless," |
|
Date: August 10th, 2025 12:08 PM Author: chrome jap (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171716) |
|
Date: August 10th, 2025 12:16 PM Author: Blue Lodge Legal Warrant The problem is he won’t precisely define a class of problems that the models won’t be able to solve with more training. He finds new ones with every model iteration and insists it’s a flawed approach even as the overall error rate goes down significantly. I don’t think the current approach will yield AGI but there is very likely an ML approach that will.
GPT-5 is in some ways underwhelming (if you expected a GPT 3 to 4 level leap), but it’s roughly consistent with known training capacity and the short time period since o3 was released. The training compute of the model is likely around 10x of GPT-4 rather than 100x with GPT-3 to 4. As the larger data centers come online, further model progress is inevitable even without architectural improvements |
|
Date: August 10th, 2025 2:37 PM Author: Titillating halford azn Gary Marcus is just a fucking clown with a clown agenda simple as that
We aint getting "AGI" with LLMs though bro that is not happening. We're getting a bunch of differential specialized agentic setups trained to do various specialized tasks |
|
Date: August 9th, 2025 7:53 PM Author: drab location It's actually 180. It doesn't work as well for dumbs though. A machine can only do so much. Try handing a guitar to a beginner. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170583) |
|
Date: August 10th, 2025 5:20 AM Author: soul-stirring elastic band theatre It has been blatantly disregarding more of my instructions. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171185) |
|
Date: August 9th, 2025 7:59 PM Author: Titillating halford azn It's actually quite good. They appear to have fixed the issue where you got aggressively relegated to older shittier models
It's not way better than the other newest reasoning models but it's definitely better. It burns a lot of tokens though. Its answers are longer than they should be in a lot of cases |
|
Date: August 9th, 2025 8:01 PM Author: drab location (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49170604) |
|
Date: August 10th, 2025 5:26 AM Author: Mewling nudist keepsake machete sandwich it's actually the best thing to ever happen to America since election night 2016 (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171186) |
|
Date: August 10th, 2025 7:34 AM Author: 180 Property Hissy Fit It’s still just as bad with any judgment-based stuff for lawing. If you ask it to review a deal and give you the top risks / off market stuff is still gives laughable advice, often focusing on non issues or low risk issues, creating issues where there are none (still doesn’t seem to parse how clause 7.2 works with 9.6 unless you directly confront it about it), and doesn’t have a good grasp of what is actually market. It’s obviously still highly useful but yeah, more as an assistant and for quick summaries and reviews in the hands of someone that knows its limitations and the subject matter they’re working on. At least in law anyway. So, I don’t think we have a whole lot worry about at least yet. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171248) |
|
Date: August 10th, 2025 7:40 AM Author: drab location Thats because law is all human generated contradictory tribal and emotion based bullshit (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171249) |
|
Date: August 10th, 2025 8:19 AM Author: 180 Property Hissy Fit Ok, but I don’t think that’s why it’s still pretty bad at it. There’s some real limitations to the tech still. I’m sure they’ll likely solve them, but if it was easy they would have done so by now I think. Progress seems to be slowing some. If you ask GPT-5 itself it will just claim it’s already capable but needs to be trained on better market standards and prompted correctly, but that’s tough to believe. If it’s so shitty that you get Reddit tier stuff when you prompt it about anything requiring some kind of more advanced knowledge then that seems like a pretty severe restriction to the tech. And it has a hard time remembering context and detail and seeing the overall picture. I’ve tried playing chess with it and it’s garbage at it, forgets board positions quickly. Also a major limitation and one they’d solve if it were easy (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171267) |
|
Date: August 10th, 2025 9:31 AM Author: Titillating halford azn Try the newest reasoning models for this. Don't use the base non reasoning models. They are actually quite good
I have been working with a lawyer friend on prompt engineering for legal analysis and if you prompt thoughtfully it is actually quite good. The key is to be thorough in your prompting to "force" it to consider all factors. You are right that it won't automatically "intuit" everything that it needs to consider. You have to prompt it as thoroughly as possible with a template |
|
Date: August 10th, 2025 11:02 AM Author: soul-stirring elastic band theatre How do you keep it from disregarding your instructions? I've found that to be a huge problem. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171600) |
|
Date: August 10th, 2025 11:07 AM Author: Titillating halford azn The newest reasoning models are pretty damn good at following prompt instructions. They do sometimes still make mistakes but it's pretty rare these days
The bigger obstacle is what smoker described, the model not being able to form a fully accurate model of the situation it's analyzing, and so it just conceptually misses stuff. LLMs don't form abstract world models of situations so you have to prompt them to ad hoc "build" them by telling them all the different things to discern from the data and consider in its response |
|
Date: August 10th, 2025 11:23 AM Author: soul-stirring elastic band theatre Rare? I spent hours yesterday fighting 5's disregard of really clear instructions. Maybe I'm just unlucky.
Edit: Response after one of the many times being caught: https://ibb.co/nqkS4QmY |
|
Date: August 10th, 2025 12:06 PM Author: 180 Property Hissy Fit If you have to give it an exhaustive list of considerations and instructions then it starts to lose its value relatively quickly. How many of us have just done something ourselves vs giving it to a junior for that exact reason? And that’s about how it feels, like a really good junior but one that autistically stumbles into a brilliant observation every now and then and that has encyclopedia level powers etc etc
|
|
Date: August 10th, 2025 11:24 AM Author: self-absorbed space (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49171640) |
|
Date: August 10th, 2025 1:44 PM Author: aromatic mood it turns out diffusion models are significantly more data efficient than the autoregressive models that are being used by all the major labs. >3x better data efficiency and this is only one possible improvement. the problem with the "AI is hitting a wall" theory is that there are many ways to use additional compute to improve model performance and the field is still too new to make strong conclusions of this sort.
https://jinjieni.notion.site/Diffusion-Language-Models-are-Super-Data-Learners-239d8f03a866800ab196e49928c019ac |
|
Date: August 10th, 2025 2:33 PM Author: Titillating halford azn (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49172100) |
|
Date: August 10th, 2025 3:12 PM Author: Arousing dead internal respiration whorehouse I think people would have preferred no transparency on GPT-5. Revealing that it routes to different models is something they should have kept private and left us to assume otherwise. (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49172181) |
|
Date: August 10th, 2025 5:21 PM Author: iridescent comical toilet seat "Overdue, overhyped, & underwhelming" the title of NYUUG's sextape (http://www.autoadmit.com/thread.php?thread_id=5760544&forum_id=2E#49172468) |