I think this plan is really quite well-balanced and sane. Also kudos to UChicago Law School for including an actual implementation plan here, not just vague thoughts.
Some bits I found interesting:
[For required courses in year 1]
> Across all 1L sections, we will prohibit the use of electronic devices such as laptops, tablets, and phones in the classroom.
> Additionally, examinations will be in-class without access to the internet, electronic files, or apps. And most of all, we will continue our longstanding tradition of emphasizing the Socratic Method as part of these courses.
[For Legal Research & Writing year 1]
> Throughout the year, students will write without AI, while also using AI for research, revision, iterating on drafts, and preparation for oral argument. Students and their instructors will review together both their writing and their use of AI.
[Electives]
> For these courses, the use of the Socratic Method, no-device policies, and in-class, no-access exams will remain, but as default rules rather than required policies.
> In all elective courses, we will encourage instructors to experiment with pedagogy.
[Upper level writing]
> We will be adding one additional requirement, which is that all students will be required to engage in an oral discussion of their SRP with their supervising professor, in an in-person setting.
w10-1 5 hours ago [-]
These are the right bits to call out.
But the effect is to make success more dependent on networks. (If they are using AI as an excuse to make networks more relevant than skills, it relieves competitive pressure.)
First, a main draw of law journals is the filing cabinet of past exams as remembered by past students, available only to journal members i.e., friends of connected people. Reducing one's ability to remember what was in a lecture only increases the advantage insiders have.
Second, the "Socratic" Method for teaching law is nothing like it is in philosophy; it's more of a weird combination of hide-the-ball and distraction that again ensures people who can see through the clouds are using out-of-band information, and share it only with those they seek to enlist or join.
> all students will be required to engage in an oral discussion of their SRP with their supervising professor, in an in-person setting
I'm surprised anyone could get through otherwise. When did it become legitimate to dump your paper and not answer for it?
mag7269 5 hours ago [-]
All nice and all, but their grads will be wrecked by a street-wise guy with AI glasses (and eventually an implanted chip). There’s no competition in a machines vs humans match. It’s over. Of course these elite centers of knowledge want to keep the status quo, they base their entire existence on gate-keeping useful shit from the riff-raff. Well, fuck them.
j4yav 5 hours ago [-]
A street wise moron used as a sort of meat puppet to carry a computer around that answers questions for him? What does that even mean practically? Won’t the graduates who learned how to think also have access, but then also be able to think for themselves too?
kozzion 5 hours ago [-]
As a CTO of an Ai first law firm I would like to add that most human lawyers are absolutely terrible. We review 100s of old contracts every week and most of them have 10-20 serious drafting errors on them. The most common ones are of the order of defining the same concept like 'a working day' in various conflicting ways without every specifying when which of the conflicting definitions should be used. Lawyers still have something to add to process but for every time they catch AI out on them, the machine is usually already 10-1 ahead.
matwood 2 hours ago [-]
> As a CTO of an Ai first law firm I would like to add that most human lawyers are absolutely terrible
This is many people at many jobs. We continue to see this weird thing when AI comes into an industry that suddenly the product being produced by the people was perfect/amazing/whatever. Maybe that's true for the majority of HN who are lucky enough to work with experts in every field, but in the general population that's simply not the case.
Many doctors, lawyers, and programmers are not very good at their jobs - I've seen it first hand. AI gives the general population a way to steer around these people a bit, and maybe ask the right questions. Is it perfect? Nope, but it's often better than the people someone has access to.
dgellow 30 minutes ago [-]
The problem is that a layman has no way to know if what the llm told them is better than experts around them, or even just remotely correct. A wrong response and a correct, informative, helpful response look exactly the same, the LLM will express the same level of confidence and will defend them in both cases.
So, on one hand you can get an expert take that might be wrong, but is linked to an actual person, with a reputation and some level of ownership. On the other hand you have an over confident LLM that might be wrong and has no reputation, no ownership. How does that actually improve things compared to the older status quo?
tossandthrow 4 hours ago [-]
> most human lawyers are absolutely terrible.
I think this extends to the other areas that AI is also disrupting.
Software engineers are also absolutely terrible: In being sloppy with handling errors, doing coercions, etc. All because always engineering after best practice simply takes too much time and mental load.
The real disruption from AI is likely that it does not get mentally fatigue. It can keep on, launch adversarial audits, make sure that everything is up to all standards.
ShinyLeftPad 2 hours ago [-]
> The real disruption from AI is likely that it does not get mentally fatigue
It's worse than mental fatigue. A friend had GLM 5.2 suddenly become dumb for hours. It started forgetting instructions, randomly inserted some bits in simplified chinese... Apparently throttled due to demand. there was tons of tokens left. (btw when your llm runs out of tokens is another example that's worse than fatigue)
A tired programmer usually knows he's tired (and can take a break) and fucks up in more expected ways
tossandthrow 2 hours ago [-]
> A tired programmer usually knows he's tired
6 months ago everybody talked about how AI was not able to plan their work. They could only do minor changes.
The introspective property of knowing when you are "tired" and acting appropriately will likely be trivially observed in 6 months.
ShinyLeftPad 2 hours ago [-]
human fatigue is detected by sensory experience and there's no sensory experience for llms. so it can just guess.
operators will never expose this guess to the user (who wants to pay for a tool that complains and refuses work?) so they probably will use it to load balance differently. and if they are out of resources then they can't load balance so they'll just hope you won't notice, so we're back to where we started. maybe it's already roughly how it works now...
what's most likely in 6 months is a big correction in pricing, once they get people hooked up on this enough. maybe there will be a model that manages to detect when it's stupid but it won't be available outside FAANG and DOD
tossandthrow 57 minutes ago [-]
Ahh sorry, I see that you explicitly wrote that deteriorated performance was due to throttling.
Naturally, LLMs do not get fatigue. As such they also don't need to detect it. That is correct. But LLMs can see if their performance have fatigue-like deterioration and correct for it.
That case is even more trivial and has nothing fundamentally to do with LLMs. Capacity is added everyday.
I would not base my long term projects on what happens in the market based on the fact that a single provider needs to throttle to satisfy demand now.
Your assumption that the current pricing is not sustainable might be fair. Personally I believe the opposite, and I havde not seen any indication that inference should not continue to decline in price.
In particular, LLMs appear to be hyper commoditisable. So if Anthropic or OpenAI is doing pricing shenanigans, people will likely move on.
ShinyLeftPad 55 minutes ago [-]
> Naturally, LLMs do not get fatigue
they are not human so human sensations don't apply. but I heard when you run out of tokens it's also kind of like fatigue, just less predictable
zer00eyz 4 minutes ago [-]
> most human lawyers are absolutely terrible. We review 100s of old contracts every week
I hope the irony is lost on no one that "Attention Is All You Need" was the paper that kicked the LLM boom off.
Part of the distinction between a how good one lawyer will be vs another is exactly that: attention to detail. And, as a trade, a practice, it seems like a skill that they try to hammer into the people who pursue this career (to varying degrees of success clearly).
It makes the last 20 years of leet code hiring look downright wrong. Your ability to recall what sorting algo to use, or solve some brain teaser conveys nothing about your ability to read (massive amounts) of code, and think critically about it.
bonzini 5 hours ago [-]
But the real question is 1) whether other human lawyers would catch the error 2) in how much time 3) which of the errors introduced by AI vs. human would have bigger consequences. On 1 and 2 the answers are very likely to be "yes" and "it would take humans more time", but the question that really matters is 3.
cognitiveinline 4 hours ago [-]
Adversarial agents and reviews means 3 is very clearly giving the benefit to AI.
Let's face it, only thing lawyers are there for right now is to take liability. And even that may not be needed 5 years from now.
Middlemen that can be automated will be automated.
nradov 1 hours ago [-]
Lawyers do a lot more than accepting liability. In my field they spend a lot of time on contract negotiation, and on interpreting what federal and state regulations actually require us to do.
dozerly 3 hours ago [-]
Most people are terrible at what they do, period, regardless of profession.
ButlerianJihad 5 hours ago [-]
I have the impression that most law firms consume and produce documentation by means of a cadre of paralegals who are minimally trained, minimally experienced, and maximally stressed by the pressure of deadlines and case loads. These paralegals are probably working to read hundreds of pages of legal filings and make sense out of them, and on the other end, they are working to produce as much documentation as possible and that's going to be by cut-and-paste assembly. So no, they are not going to understand stuff very well, and no, their output is not really going to be consistent or intelligible. But the managing attorneys depend on these paralegals to get anything done, just like Congresspeople rely on their staffers to actually read bills and draft legislation.
At the end of the day, I think these paralegals have been quite weak links, and nearly nobody will notice a difference in quality when it's all handed over to LLMs. But the paralegals will lose their jobs, and probably have nothing better to do than purchase angle grinders and cut down Flock camera poles, or crown the front hood of Waymos with a traffic pylon.
nradov 1 hours ago [-]
It's really the junior associates doing most of that grunt work. Paralegals in most firms have more limited responsibilities. Some work has already been handed over to LLMs but the job market for graduates of first-tier law schools remains strong.
agentprose 6 hours ago [-]
I recent beat an actual lawyer as a pro se plaintiff in a civil suit. It was all codex and claude, using the cli agents to write the "import docx" scripts that wrote all the filings, and scripts to assemble exhibits. I don't know how long the bars will be able to hold onto their monopoly powers when confronted with LLMs that are measurably superior.
bryanrasmussen 6 hours ago [-]
I've seen lawyers lose to non-lawyers who didn't use all that, just because the lawyers didn't do their work, thinking, no doubt, "I'm a lawyer I will just win", at any rate there is a well known aphorism that a lawyer who represents themselves has a fool for a client.
finally, was this thing you beat the actual lawyer at in their specific specialty, since lawyers are often focused on particular aspects of the law?
agentprose 2 hours ago [-]
It was a condominium law specialized attorney representing a condominium association on a matter of condominium law.
applfanboysbgon 5 hours ago [-]
Is it perhaps possible that you won your case because you didn't break the law, or conversely because the other party did break the law? Unless ChatGPT persuaded a judge to side in your favor even though you were legally in the wrong, this really doesn't tell us anything at all. (And even in that circumstance, it might speak more to the complete incompetence of the other party's legal representation.)
agentprose 2 hours ago [-]
In my case, the defendant was indeed breaking the law, but it was a "new" law untested by case law, so it was somewhat risky how a judge would interpret it. The counsel of record for the defendant was bringing "recycled briefs" energy, possibly out of overconfidence.
slopinthebag 6 hours ago [-]
[flagged]
agentprose 2 hours ago [-]
I'm sorry that I created a new account to not doxx myself. The number of bits of entropy of court cases in the US, in which a pro se plaintiff won, and has obviously written-by-codex legal filings, is not one I would sacrifice to identify my main account.
sublinear 3 hours ago [-]
You're not being downvoted by real people either. This happens a lot on HN, but I'm not even sure they have an audience to sway anymore. I don't get it either.
Everyone has seen the limits of this wave of "AI" for years now. There is no use case for them in general knowledge/work. They require teams of expert humans to do a ton of heavy lifting behind the scenes. It's only ever worth it for automation where failure is cheap. Most work is not like this. Anyone with a job today earns their pay by improvising during one-off tasks with more depth than breadth. LLMs are clearly not the missing piece to any automation, which is why nobody asked for them in the first place. Automation that can be trusted requires deterministic rules vetted by humans.
If LLMs are going to stay relevant in the coming years, it's time for them to mature into the niche tools that they will inevitably become. They are good at search and summary. They are good at natural language processing. Of course they are. The only people who think otherwise have been ripped off and now they have no choice but to try to rip everyone else off to make up for it.
Kaibeezy 3 hours ago [-]
The posted article is about law school, but the comments are focusing on practice. These overlap but are not the same.
“Lawyer” is one of many possible career paths. Law graduates are uniquely qualified as premium law-aware generalists. In my opinion, there is no better education to prepare for future business uncertainty.
JSR_FDED 6 hours ago [-]
I think more universities will follow. Every prof I speak to has noticed shorter attention spans, low effort LLM writing, and inability to assess competence.
keiferski 5 hours ago [-]
It’s interesting to think that the “public speaking skills” aspect of some legal jobs will probably continue to be AI-proof for the indefinite future. No one is going to present ChatGPT on an iPad as their defense attorney.
On the other hand, the paperwork-first side of law is probably in for a tough future. Especially if your clients are not big corporations or rich celebrities.
In general I think my advice to young people is: focus on social skills, public speaking, etc. These are not going to be replaced by robots in your lifetime.
dgellow 23 minutes ago [-]
> No one is going to present ChatGPT on an iPad as their defense attorney.
Even small corporations have a huge need for paperwork in terms of contract negotiation and redlining agreements. Some of this can be partially automated but the real value an experienced attorney brings is in knowing how those issues fit into the larger business context, and deciding what to press on or concede.
shevy-java 5 hours ago [-]
> our graduates will be expected to be prepared to use them in legal practice.
But then why do I need a lawyer?
I do watch Travis's videos. Ever since he integrated AI,
the video quality dropped immensely and it became really
stupid, compared to his older videos. AI degraded the
quality and discussions he used to have with law enforcement
too, it's quite sad actually.
This is exactly why all these people talking about a “bubble” I think are misguided.
Are expectations way too high in the near term for tech companies and all of the crap that they’re talking? 100% totally agree. But this is “just another an AI cycle” and if you’re a practitioner it does look different than the previous ones.
However this kind of statement was not released for crypto, or for the Internet. Those were massively impactful technologies.
What you are actively seeing happening now are institutions saying “we have to change the way that we practice” “the way that we did business before is no longer as applicable as it’s going to be changing very rapidly in the future and we’re going to have to deal with that.”
That is not coming from investors or technology companies. That is coming from people who are utilizing this stuff and being impacted by it
So it is absolutely the case that there’s going to be some kind of financial changes but just like every other AI cycle we are left with a aggressively more capable technology that is more pervasive.
Everybody needs to be taking that seriously.
rayiner 7 hours ago [-]
AI is a real thing and it's the most significant development in the legal sector since the Internet. Saturday morning, I spent a couple of hours getting my arms around some discovery, which before would've taken me the whole day. Even if it never gets any better it's a huge step forward. It's not only going to change legal education, it's going to change what the pipeline looks like in law firms. Similarly to the changes I expect to see in any kind of white collar job where AI replaces a lot of the data analysis/data transformation work, leaving people to do more human interaction.
ElProlactin 7 hours ago [-]
> Similarly to the changes I expect to see in any kind of white collar job where AI replaces a lot of the data analysis/data transformation work, leaving people to do more human interaction.
Taking law as the example, I'm not sure I agree.
Legal is largely a distress purchase. Very few clients actually want to spend time interacting with outside corporate counsel, a divorce lawyer, a personal injury lawyer, etc.
The real value of an attorney is accountability. A person with a license and reputation who is on the hook for the advice they provide. But accountability is only worth buying if the judgment behind it is sound, and that judgment takes years to build.
The problem is that much of what makes a "good" lawyer comes from years of doing unglamorous work as a junior, financed by clients. If AI reduces the ability of firms to subsidize training the next generation of lawyers, you get fewer "good" lawyers and the whole market keeps eroding.
From what I've seen, the top AI models can now produce contracts that are on par with what I would have had to pay thousands of dollars for a decade ago, and the agents/chatbots are capable of collaborating with you to address fairly complex issues. If you have the money, you still run everything by a real attorney of course for the accountability piece, but the future is not bright for contract attorneys because people know even they're probably using AI to do half their work.
rayiner 6 hours ago [-]
> The problem is that much of what makes a "good" lawyer comes from years of doing unglamorous work as a junior, financed by clients. If AI reduces the ability of firms to subsidize training the next generation of lawyers, you get fewer "good" lawyers and the whole market keeps eroding.
That is what AI will do, and firms will figure out how to adapt. To be fair, this trend has been going on for some time now already. Clients want attorneys with experience and judgment to be on the hook in real trials, mergers, etc. There’s only so many of those folks. The trend of the last several decades has been pretty steady overall headcount at firms despite large increases in the amount of legal work, and ever increasing demand for the limited set of seasoned attorneys.
Firms have not done a great job cultivating the next generation against the backdrop of those trends. But that’s true in a lot of white collar jobs. And AI will make that problem more acute.
But—lawyers are a transaction cost and it’s good for the economy to automate as much of it as we can!
ElProlactin 5 hours ago [-]
> But—lawyers are a transaction cost and it’s good for the economy to automate as much of it as we can!
I guess it depends on how you look at it.
I don't think anyone would argue that spending money on lawyers is an ideal use of capital. But if AI-driven automation reduces overall demand for legal services (as measured by billable hours) and/or erodes the pricing power of lawyers/law firms, it doesn't mean that the "savings" will benefit all participants in the economy.
The biggest beneficiary will be corporations, as reduced legal expenditure can boost corporate profits. But that is a boon for the wealthy shareholder class, not the worker class. "Lawyer" has been one of the highest-paying white collar jobs available so the question becomes: as all of these high-paying white collar jobs become less plentiful, how do young people not born into wealth develop opportunities to get ahead economically?
anon373839 6 hours ago [-]
> From what I've seen, the top AI models can now produce contracts that are on par with what I would have had to pay thousands of dollars for a decade ago
How would you know? Unless you have expertise in this area, you can't really know if you have a good contract until you have to litigate it.
> you still run everything by a real attorney of course for the accountability piece
Your attorney can make better use of AI than you can. A growing time sink for lawyers (and I assume, physicians) is having to explain to clients that "yes, I understand this is what Claude told you," and "no, that's not actually how X works in practice."
ElProlactin 6 hours ago [-]
> How would you know? Unless you have expertise in this area, you can't really know if you have a good contract until you have to litigate it.
I ran multiple businesses for over 20 years, and during that time had to deal with countless contracts involving commercial transactions, leases, funding events and employment. I worked closely with outside counsel, and had to deal with numerous disputes and lawsuits.
Many contracts are boilerplate to begin with, which is why firms like Cooley and Orrick provide free document libraries and generators. They know that a huge segment of the people who might retain them aren't going to pay $10,000 for an NDA or a simple commercial contract. So they give documents away as a marketing tool, hoping that it might get them more complex work in the future.
Attorneys can add value when you have a unique wrinkle and of course when there's a dispute, but, as an example, if you need an employment contract for California, you don't need to pay someone $500/hour to draft one from scratch and frankly for many contract categories, you'd probably be taking on more risk by having a bespoke contract drafted than modifying a solid, widely-used template.
> Your attorney can make better use of AI than you can. A growing time sink for lawyers (and I assume, physicians) is having to explain to clients that "yes, I understand this is what Claude told you," and "no, that's not actually how X works in practice."
You're not wrong, but I think you're misunderstanding what I'm saying.
First, as far as contracts are concerned, the leading AIs are more than capable of producing pretty decent ones for many categories because they've been trained against all the real contracts that are out there. Your "yes, I understand this is what Claude told you but..." scenario is real, but it applies less and less than attorneys want to believe, especially when the subject is mundane.
Second, the bigger issue is that AI is fundamentally hindering the development of the next generation of lawyers. It's great that senior associates and partners can use AI to be more efficient, but the economic pressures AI creates mean that it's far harder for law firms to have clients subsidize the training of the next generation of lawyers.
So the question isn't whether an experienced attorney today can make better use of AI than you can, it's whether a new attorney today will be able to become a "good" lawyer when AI is eroding their firms' ability to have clients subsidize their training.
anon373839 4 hours ago [-]
Good points.
For contracts, I was thinking of the more complex deals where a battle-tested template is unavailable. Though even where one appears to fit, it doesn't always. One of the difficulties in professional/client relations is that there are often details that one party sees as minor or completely unremarkable, and the other party sees as changing the whole frame of analysis. I haven't been too impressed with LLMs being able to identify things like this -- they have reams of knowledge, but no practical sense.
The issue about halting the training pipeline is an interesting one. In law firms, a lot of "training" of young lawyers is really glorified bates-stamping charged out at professional rates. I think we could find that the work just gets more substantive and interesting. And yes, pyramid-shaped firms do rely on this type of stuff for revenue. But if all parties can do more sophisticated work, then AI may only nudge forward an arms race without fundamentally changing much (at least for work that has adversarial backpressure).
matwood 2 hours ago [-]
I don't know why you're downvoted. People act like lawyers do something magical, but if you have been around business contracts for 2+ decades you can certainly draft one and then just have a lawyer review if you want.
Terr_ 5 hours ago [-]
> Saturday morning, I spent a couple of hours getting my arms around some discovery
It might be worth distinguishing between LLMs as tools for fuzzy-searching hard facts, versus using them to craft logical arguments. [0] The risk-profile and verification difficulty are rather different.
[0] Or, more-precisely, using them to craft a pattern-fitting text artifact, which hopefully maps onto a sane concept when reinterpreted by a human.
vouaobrasil 7 hours ago [-]
Indeed - it helps people become much more efficient. When it comes to law, that includes criminals as well. Whatever gains in efficiency happen on one side, they happen on the other too. Yes, everyone loves to say how they coded a cool side project in a day instead of a week, but what of the situation where AI is placed in a situation with two sides going against each other. The same old fight, but with better weapons.
dgellow 21 minutes ago [-]
> This is exactly why all these people talking about a “bubble” I think are misguided.
A bubble doesn’t say anything about how useful the underlying technology is. We are in an AI bubble, in the sense that the allocated resources is completely out of line with the underlying businesses. AI companies would literally need the entire world as their TAM, using their services non stop to justify the valuations and level of expenditures we are seeing. What you describe is literally the symptoms of a bubble
KaiserPro 4 hours ago [-]
I am at a company that currently has a whole bunch of lawyers, because we are dealing with multi-year >100s million infra projects.
One of the things that AI has helped with is automatic breech notifications. They are largely proforma, but depending on which generation of contract they are notifying from, subtle changes are needed.
Law is mostly highly structured text, and you know what loves highly structured and documented text? LLMs.
Legal LLMs do share the same issue with coding LLMs, they tend to default to lazy/insecure. Its not the case where you as a non-lawyer can prompt it to make a "water tight" contract from scratch.
KPGv2 7 hours ago [-]
> However this kind of statement was not released for crypto, or for the Internet. Those were massively impactful technologies.
How has crypto been massively impactful?
cyberax 6 hours ago [-]
Crytobros gave us ransomware, industrialized pig butchering scams, billion-dollar heists, and arguably the second Trump presidency.
I'd say it's pretty impactful.
TurdF3rguson 7 hours ago [-]
The only way crypto touches lawyers that I can think of is smart contracts, and that's been.. minimally impactful at most.
Ampersander 5 hours ago [-]
> Are expectations way too high in the near term for tech companies and all of the crap that they’re talking? 100% totally agree.
This is what people mean when they say it's a bubble.
hatefulheart 7 hours ago [-]
The bubble is about the market you wally not about whether statistical models are useful.
I wish people would stop being so dense.
delusional 5 hours ago [-]
> However this kind of statement was not released for crypto, or for the Internet.
Yes they were. Educators have been talking about how access to the internet has affected their students learning since forever. We were cautioned and had dedicated classes to teach us to be critical of different sources, teaching us what Wikipedia could and could not be used for. The internet was deliberately turned off for our exams.
Crypto too was seen as culturally important, papers were written by otherwise well respected authors. We were expected to read some whitepapers about crypto. Interestingly, reading those papers only made us more sure it was a scam.
You're rewriting history to make this moment seem different, it is not.
nttylock 6 hours ago [-]
[flagged]
avaer 3 hours ago [-]
This is a more reasoned take than most reactions to AI. But nobody wants to consider the possibility that maybe we won't actually need what was traditionally called lawyers in the mid/near term future; certainly before the new grad's careers are up.
I would never expect Harvard Law school to admit this, their hands are tied by many interests, but IMO they still aren't being pragmatic enough to evolve.
I would make the courses 50% AI based immediately, the other 50% being based on validating and applying the AI's work with traditional rigorous work. Anyone who rejects this should drop out, because they won't have a career anyway. And if Harvard Law disagrees then they can find out how meaningful their degrees are going to be in 10 years.
Rendered at 12:19:38 GMT+0000 (Coordinated Universal Time) with Vercel.
Some bits I found interesting:
[For required courses in year 1]
[For Legal Research & Writing year 1] [Electives] [Upper level writing]But the effect is to make success more dependent on networks. (If they are using AI as an excuse to make networks more relevant than skills, it relieves competitive pressure.)
First, a main draw of law journals is the filing cabinet of past exams as remembered by past students, available only to journal members i.e., friends of connected people. Reducing one's ability to remember what was in a lecture only increases the advantage insiders have.
Second, the "Socratic" Method for teaching law is nothing like it is in philosophy; it's more of a weird combination of hide-the-ball and distraction that again ensures people who can see through the clouds are using out-of-band information, and share it only with those they seek to enlist or join.
> all students will be required to engage in an oral discussion of their SRP with their supervising professor, in an in-person setting
I'm surprised anyone could get through otherwise. When did it become legitimate to dump your paper and not answer for it?
This is many people at many jobs. We continue to see this weird thing when AI comes into an industry that suddenly the product being produced by the people was perfect/amazing/whatever. Maybe that's true for the majority of HN who are lucky enough to work with experts in every field, but in the general population that's simply not the case.
Many doctors, lawyers, and programmers are not very good at their jobs - I've seen it first hand. AI gives the general population a way to steer around these people a bit, and maybe ask the right questions. Is it perfect? Nope, but it's often better than the people someone has access to.
So, on one hand you can get an expert take that might be wrong, but is linked to an actual person, with a reputation and some level of ownership. On the other hand you have an over confident LLM that might be wrong and has no reputation, no ownership. How does that actually improve things compared to the older status quo?
I think this extends to the other areas that AI is also disrupting.
Software engineers are also absolutely terrible: In being sloppy with handling errors, doing coercions, etc. All because always engineering after best practice simply takes too much time and mental load.
The real disruption from AI is likely that it does not get mentally fatigue. It can keep on, launch adversarial audits, make sure that everything is up to all standards.
It's worse than mental fatigue. A friend had GLM 5.2 suddenly become dumb for hours. It started forgetting instructions, randomly inserted some bits in simplified chinese... Apparently throttled due to demand. there was tons of tokens left. (btw when your llm runs out of tokens is another example that's worse than fatigue)
A tired programmer usually knows he's tired (and can take a break) and fucks up in more expected ways
6 months ago everybody talked about how AI was not able to plan their work. They could only do minor changes.
The introspective property of knowing when you are "tired" and acting appropriately will likely be trivially observed in 6 months.
operators will never expose this guess to the user (who wants to pay for a tool that complains and refuses work?) so they probably will use it to load balance differently. and if they are out of resources then they can't load balance so they'll just hope you won't notice, so we're back to where we started. maybe it's already roughly how it works now...
what's most likely in 6 months is a big correction in pricing, once they get people hooked up on this enough. maybe there will be a model that manages to detect when it's stupid but it won't be available outside FAANG and DOD
Naturally, LLMs do not get fatigue. As such they also don't need to detect it. That is correct. But LLMs can see if their performance have fatigue-like deterioration and correct for it.
That case is even more trivial and has nothing fundamentally to do with LLMs. Capacity is added everyday.
I would not base my long term projects on what happens in the market based on the fact that a single provider needs to throttle to satisfy demand now.
Your assumption that the current pricing is not sustainable might be fair. Personally I believe the opposite, and I havde not seen any indication that inference should not continue to decline in price.
In particular, LLMs appear to be hyper commoditisable. So if Anthropic or OpenAI is doing pricing shenanigans, people will likely move on.
they are not human so human sensations don't apply. but I heard when you run out of tokens it's also kind of like fatigue, just less predictable
I hope the irony is lost on no one that "Attention Is All You Need" was the paper that kicked the LLM boom off.
Part of the distinction between a how good one lawyer will be vs another is exactly that: attention to detail. And, as a trade, a practice, it seems like a skill that they try to hammer into the people who pursue this career (to varying degrees of success clearly).
It makes the last 20 years of leet code hiring look downright wrong. Your ability to recall what sorting algo to use, or solve some brain teaser conveys nothing about your ability to read (massive amounts) of code, and think critically about it.
Let's face it, only thing lawyers are there for right now is to take liability. And even that may not be needed 5 years from now.
Middlemen that can be automated will be automated.
At the end of the day, I think these paralegals have been quite weak links, and nearly nobody will notice a difference in quality when it's all handed over to LLMs. But the paralegals will lose their jobs, and probably have nothing better to do than purchase angle grinders and cut down Flock camera poles, or crown the front hood of Waymos with a traffic pylon.
finally, was this thing you beat the actual lawyer at in their specific specialty, since lawyers are often focused on particular aspects of the law?
Everyone has seen the limits of this wave of "AI" for years now. There is no use case for them in general knowledge/work. They require teams of expert humans to do a ton of heavy lifting behind the scenes. It's only ever worth it for automation where failure is cheap. Most work is not like this. Anyone with a job today earns their pay by improvising during one-off tasks with more depth than breadth. LLMs are clearly not the missing piece to any automation, which is why nobody asked for them in the first place. Automation that can be trusted requires deterministic rules vetted by humans.
If LLMs are going to stay relevant in the coming years, it's time for them to mature into the niche tools that they will inevitably become. They are good at search and summary. They are good at natural language processing. Of course they are. The only people who think otherwise have been ripped off and now they have no choice but to try to rip everyone else off to make up for it.
“Lawyer” is one of many possible career paths. Law graduates are uniquely qualified as premium law-aware generalists. In my opinion, there is no better education to prepare for future business uncertainty.
On the other hand, the paperwork-first side of law is probably in for a tough future. Especially if your clients are not big corporations or rich celebrities.
In general I think my advice to young people is: focus on social skills, public speaking, etc. These are not going to be replaced by robots in your lifetime.
That already happened! https://youtu.be/MkmfZPt-gaw?si=gk1k3sGsFq9npp0X
But then why do I need a lawyer?
I do watch Travis's videos. Ever since he integrated AI, the video quality dropped immensely and it became really stupid, compared to his older videos. AI degraded the quality and discussions he used to have with law enforcement too, it's quite sad actually.
https://www.youtube.com/@Exchange_IDs/videos
Are expectations way too high in the near term for tech companies and all of the crap that they’re talking? 100% totally agree. But this is “just another an AI cycle” and if you’re a practitioner it does look different than the previous ones.
However this kind of statement was not released for crypto, or for the Internet. Those were massively impactful technologies.
What you are actively seeing happening now are institutions saying “we have to change the way that we practice” “the way that we did business before is no longer as applicable as it’s going to be changing very rapidly in the future and we’re going to have to deal with that.”
That is not coming from investors or technology companies. That is coming from people who are utilizing this stuff and being impacted by it
So it is absolutely the case that there’s going to be some kind of financial changes but just like every other AI cycle we are left with a aggressively more capable technology that is more pervasive.
Everybody needs to be taking that seriously.
Taking law as the example, I'm not sure I agree.
Legal is largely a distress purchase. Very few clients actually want to spend time interacting with outside corporate counsel, a divorce lawyer, a personal injury lawyer, etc.
The real value of an attorney is accountability. A person with a license and reputation who is on the hook for the advice they provide. But accountability is only worth buying if the judgment behind it is sound, and that judgment takes years to build.
The problem is that much of what makes a "good" lawyer comes from years of doing unglamorous work as a junior, financed by clients. If AI reduces the ability of firms to subsidize training the next generation of lawyers, you get fewer "good" lawyers and the whole market keeps eroding.
From what I've seen, the top AI models can now produce contracts that are on par with what I would have had to pay thousands of dollars for a decade ago, and the agents/chatbots are capable of collaborating with you to address fairly complex issues. If you have the money, you still run everything by a real attorney of course for the accountability piece, but the future is not bright for contract attorneys because people know even they're probably using AI to do half their work.
That is what AI will do, and firms will figure out how to adapt. To be fair, this trend has been going on for some time now already. Clients want attorneys with experience and judgment to be on the hook in real trials, mergers, etc. There’s only so many of those folks. The trend of the last several decades has been pretty steady overall headcount at firms despite large increases in the amount of legal work, and ever increasing demand for the limited set of seasoned attorneys.
Firms have not done a great job cultivating the next generation against the backdrop of those trends. But that’s true in a lot of white collar jobs. And AI will make that problem more acute.
But—lawyers are a transaction cost and it’s good for the economy to automate as much of it as we can!
I guess it depends on how you look at it.
I don't think anyone would argue that spending money on lawyers is an ideal use of capital. But if AI-driven automation reduces overall demand for legal services (as measured by billable hours) and/or erodes the pricing power of lawyers/law firms, it doesn't mean that the "savings" will benefit all participants in the economy.
The biggest beneficiary will be corporations, as reduced legal expenditure can boost corporate profits. But that is a boon for the wealthy shareholder class, not the worker class. "Lawyer" has been one of the highest-paying white collar jobs available so the question becomes: as all of these high-paying white collar jobs become less plentiful, how do young people not born into wealth develop opportunities to get ahead economically?
How would you know? Unless you have expertise in this area, you can't really know if you have a good contract until you have to litigate it.
> you still run everything by a real attorney of course for the accountability piece
Your attorney can make better use of AI than you can. A growing time sink for lawyers (and I assume, physicians) is having to explain to clients that "yes, I understand this is what Claude told you," and "no, that's not actually how X works in practice."
I ran multiple businesses for over 20 years, and during that time had to deal with countless contracts involving commercial transactions, leases, funding events and employment. I worked closely with outside counsel, and had to deal with numerous disputes and lawsuits.
Many contracts are boilerplate to begin with, which is why firms like Cooley and Orrick provide free document libraries and generators. They know that a huge segment of the people who might retain them aren't going to pay $10,000 for an NDA or a simple commercial contract. So they give documents away as a marketing tool, hoping that it might get them more complex work in the future.
Attorneys can add value when you have a unique wrinkle and of course when there's a dispute, but, as an example, if you need an employment contract for California, you don't need to pay someone $500/hour to draft one from scratch and frankly for many contract categories, you'd probably be taking on more risk by having a bespoke contract drafted than modifying a solid, widely-used template.
> Your attorney can make better use of AI than you can. A growing time sink for lawyers (and I assume, physicians) is having to explain to clients that "yes, I understand this is what Claude told you," and "no, that's not actually how X works in practice."
You're not wrong, but I think you're misunderstanding what I'm saying.
First, as far as contracts are concerned, the leading AIs are more than capable of producing pretty decent ones for many categories because they've been trained against all the real contracts that are out there. Your "yes, I understand this is what Claude told you but..." scenario is real, but it applies less and less than attorneys want to believe, especially when the subject is mundane.
Second, the bigger issue is that AI is fundamentally hindering the development of the next generation of lawyers. It's great that senior associates and partners can use AI to be more efficient, but the economic pressures AI creates mean that it's far harder for law firms to have clients subsidize the training of the next generation of lawyers.
So the question isn't whether an experienced attorney today can make better use of AI than you can, it's whether a new attorney today will be able to become a "good" lawyer when AI is eroding their firms' ability to have clients subsidize their training.
For contracts, I was thinking of the more complex deals where a battle-tested template is unavailable. Though even where one appears to fit, it doesn't always. One of the difficulties in professional/client relations is that there are often details that one party sees as minor or completely unremarkable, and the other party sees as changing the whole frame of analysis. I haven't been too impressed with LLMs being able to identify things like this -- they have reams of knowledge, but no practical sense.
The issue about halting the training pipeline is an interesting one. In law firms, a lot of "training" of young lawyers is really glorified bates-stamping charged out at professional rates. I think we could find that the work just gets more substantive and interesting. And yes, pyramid-shaped firms do rely on this type of stuff for revenue. But if all parties can do more sophisticated work, then AI may only nudge forward an arms race without fundamentally changing much (at least for work that has adversarial backpressure).
It might be worth distinguishing between LLMs as tools for fuzzy-searching hard facts, versus using them to craft logical arguments. [0] The risk-profile and verification difficulty are rather different.
[0] Or, more-precisely, using them to craft a pattern-fitting text artifact, which hopefully maps onto a sane concept when reinterpreted by a human.
A bubble doesn’t say anything about how useful the underlying technology is. We are in an AI bubble, in the sense that the allocated resources is completely out of line with the underlying businesses. AI companies would literally need the entire world as their TAM, using their services non stop to justify the valuations and level of expenditures we are seeing. What you describe is literally the symptoms of a bubble
One of the things that AI has helped with is automatic breech notifications. They are largely proforma, but depending on which generation of contract they are notifying from, subtle changes are needed.
Law is mostly highly structured text, and you know what loves highly structured and documented text? LLMs.
Legal LLMs do share the same issue with coding LLMs, they tend to default to lazy/insecure. Its not the case where you as a non-lawyer can prompt it to make a "water tight" contract from scratch.
How has crypto been massively impactful?
I'd say it's pretty impactful.
This is what people mean when they say it's a bubble.
I wish people would stop being so dense.
Yes they were. Educators have been talking about how access to the internet has affected their students learning since forever. We were cautioned and had dedicated classes to teach us to be critical of different sources, teaching us what Wikipedia could and could not be used for. The internet was deliberately turned off for our exams.
Crypto too was seen as culturally important, papers were written by otherwise well respected authors. We were expected to read some whitepapers about crypto. Interestingly, reading those papers only made us more sure it was a scam.
You're rewriting history to make this moment seem different, it is not.
I would never expect Harvard Law school to admit this, their hands are tied by many interests, but IMO they still aren't being pragmatic enough to evolve.
I would make the courses 50% AI based immediately, the other 50% being based on validating and applying the AI's work with traditional rigorous work. Anyone who rejects this should drop out, because they won't have a career anyway. And if Harvard Law disagrees then they can find out how meaningful their degrees are going to be in 10 years.