As the Opening Arguments podcast (one of the two hosts is a lawyer) said: If as a lawyer you do what was asked - just parrot what an AI tells you to parrot, you're going to get sanctioned and possibly disbarred. As a lawyer you are responsible for what you say and argue, and if you argue something that you know to be false, you're in violation of the ethics standards; just about every bar association lists that as sanctionable, or even disbarrable, offense *.
Thus, effectively, the only thing you could do is a watered down concept of the idea: A lawyer that will parrot the ChatGPT answer, but only if said answer is something they would plausibly argue themselves. They'd have to rewrite or disregard anything ChatGPT told them to say that they don't think is solid argument.
They also run a segment where a non-lawyer takes a bar exam. Recently they've also asked the bar exam question to ChatGPT as well. So far ChatGPT got it wrong every time. For example, it doesn't take into account that multiple answers can be correct, in which case you have to pick the most specific answer available. Leading to a somewhat hilarious scenario where ChatGPT picks an answer and then defends its choice in a way that seems to indicate why the answer it picked is obviously the wrong answer.
*) Of course, Alan Dershowitz is now arguing in court that the seditious horse manure he signed his name under and which is now leading to him being sanctioned or possibly disbarred, is not appropriate because he's old and didn't know what he was doing. It's Dershowitz so who knows, but I'm guessing the court is not going to take his argument seriously. In the odd chance that they do, I guess you can just say whatever you want and not be responsible for it, which... would be weird.
The first wave of ChatGPT stories were all "amazing, wonderful, humanity is basically obsolete". But now that people have had a little time, I'm seeing a ton of examples where people are realizing that ChatGPT sounds like it knows what it's talking about, but actually doesn't know anything.
We know that humans are easily fooled by glib confidence; we can all think of politicians who have succeeded that way. But it sounds like ChatGPT's real innovation is something that produces synthetic bullshit. And here I'm using Frankfurt's definition, "speech intended to persuade without regard for truth": https://en.wikipedia.org/wiki/On_Bullshit
ChatGPT does some really impressive stuff! As a creative tool or a code generator it can give you some good material to work with.
But the hype has been insane, totally untethered from reality. I think it's the instinct to assume that something which can mimic human speech quite well must also have some mechanism for comprehending the meaning of the words it's generating, when it really doesn't.
True. And the really interesting question for me is how much those people are also failing to notice this distinction not just with ChatGPT, but with other people. Which sounds absurd at first, but long ago Oliver Sacks gave an example that sticks with me: https://www.junkfoodforthought.com/long/Sacks_Reagan.htm
Your footnote: A judge should take that argument seriously, and therefore disbar him because he's old and self-admittedly doesn't know what he's doing.
Thus, effectively, the only thing you could do is a watered down concept of the idea: A lawyer that will parrot the ChatGPT answer, but only if said answer is something they would plausibly argue themselves. They'd have to rewrite or disregard anything ChatGPT told them to say that they don't think is solid argument.
They also run a segment where a non-lawyer takes a bar exam. Recently they've also asked the bar exam question to ChatGPT as well. So far ChatGPT got it wrong every time. For example, it doesn't take into account that multiple answers can be correct, in which case you have to pick the most specific answer available. Leading to a somewhat hilarious scenario where ChatGPT picks an answer and then defends its choice in a way that seems to indicate why the answer it picked is obviously the wrong answer.
*) Of course, Alan Dershowitz is now arguing in court that the seditious horse manure he signed his name under and which is now leading to him being sanctioned or possibly disbarred, is not appropriate because he's old and didn't know what he was doing. It's Dershowitz so who knows, but I'm guessing the court is not going to take his argument seriously. In the odd chance that they do, I guess you can just say whatever you want and not be responsible for it, which... would be weird.