The Intricacies of AI Policy and Governance (State Bar of Texas Annual Meeting 2024)

Episode Notes

Transcript

Rocky Dhir:

Hello and welcome to another episode of the State Bar of Texas podcast. We’re recording live from our state bar annual meeting in Dallas, Texas. This is your host, Rocky Dhir. Joining me now, we have two great guests. Well, we’ve had ’em on the podcast before. I dunno if I’d go so far as to say great. But we have the honorable John Browning who is now a professor at Thomas Good Jones School of Law. I dunno how good it is if you’re a professor there, John, I got to tell you True. I’m telling. And then we’ve got the Honorable Roy Ferguson from Mountain West Texas. So Judge, welcome back.

Judge Roy Ferguson:

It’s my pleasure to be

Rocky Dhir:

Here. So you guys just rolled off a talk for the Adaptable Lawyer track, one of my favorites here at the state bar. It’s about AI policy, governance and task force report. It’s about the ethical use of AI and governance. So there’s ethical implications now to artificial intelligence. So tell us about that. Why is this becoming an ethical issue

Judge Roy Ferguson:

All of a sudden?

Justice John G. Browning:

Well, I think the main reason there are ethical concerns because lawyers keep on getting into ethical hot water over use or misuse. We get

Rocky Dhir:

An ethical hot water over everything. I mean, why is this any different?

Justice John G. Browning:

Yeah, and it’s not really what I would call a technology issue. It’s more an ethics issue because lawyers are misusing this technology and failing in their ethical duties of competence, of diligence, of supervision, and also their duty in some cases of candor to the tribunal of being upfront with the judge once they’ve been caught citing things like fabricated cases.

Rocky Dhir:

I have a feeling that John Browning just completely lifted my resume and just recited it verbatim here on the podcast. Thanks John. Appreciate it. Judge Ferguson, are you seeing this in court as well as AI becoming something that you’re having to contend with as a judge?

Judge Roy Ferguson:

Interestingly, what we are anticipating is an explosion of self-represented litigants using ai.

Rocky Dhir:

Oh, interesting.

Judge Roy Ferguson:

We have a lot of people who are forced to self represent and they use things like the forms that were promulgated by the Supreme Court of Texas. What we found was instituting those forms and making them freely available, increased the number of people who file without a lawyer because it increases confidence, although not competence. And so now with people believing that generative AI like chat, GPT

Rocky Dhir:

Makes

Judge Roy Ferguson:

Them as good as a lawyer, then there are people who will think, well now I don’t need to spend all the money on a lawyer. I have chat GPT. And so we will get briefs from people and the opening paragraph is basically written in crayon on a big chief tablet. And then you have beautiful legal argument. And then at the bottom they misspell the word respectfully. And so we know that this is a chat GPT generated cut and paste filing of a brief. And so it’s not really sneaky a lot of the time. Loppy, for example, and Justice Browning can confirm or not, but a lot of the fabricated cases that we get cited are all one syllable names. So it’s state versus Smith or Jones V Arc. So it’s obvious names and sometimes they mix. For example, they might have the wrong reporter listed or it’ll be a criminal court and a civil argument attached to the case site. So we are seeing a lot of it. Our job as I see it, is to try to let lawyers know how to protect themselves from it when they can use it and when they can’t.

Justice John G. Browning:

And plenty of lawyers have done that as well as self-represented litigants. There’s actually about 28 federal courts around the country that have issued standing orders to basically offer guidance.

Rocky Dhir:

Read the cases.

Justice John G. Browning:

Yeah. Well yeah, to remind lawyers of their obligations. And I’ll break on Judge Ferguson for a moment. He’s one of the only state court judges in the country that’s actually issued similar guidance in his court.

Rocky Dhir:

Of course, we got to remember, judge Ferguson is also, he’s tech savvy. He’s the one that got all the other judges in Texas up and running on Zoom. When COVID-19 first hit in the states, if you

Justice John G. Browning:

Can’t trust the judge who presided over the lawyer, I’m not a lawyer, I’m not a cat, I’m a lawyer hearing who can you trust?

Rocky Dhir:

No, that’s it. People still remember this. Yes, we will remember it forever. So here’s a question for either or both of you. So maybe do I have to call you Justice Browning? I’ve known you for so

Justice John G. Browning:

Long. My wife refuses even to call me the Honorable don’t

Rocky Dhir:

Don’t calls you honey. Yeah, I don’t. I think she doesn’t call you

Justice John G. Browning:

Ever. It’s how you take out the garbage around my house.

Judge Roy Ferguson:

I think the best part of that story is that you expect your wife to call you the honorable.

Rocky Dhir:

No, no. There’s no way We all know that’s not happening in the Browning house. So here’s a question maybe John, we’ll start with you on this one. Let’s just cut down to brass tacks. Do we think AI is going to replace lawyers at some point in the future? I think that’s what’s looming over the legal profession at

Justice John G. Browning:

This point. There’ve been a lot of scary statistics. Goldman Sachs did the studies that 44% of legal tasks can be done by ai. I don’t think it’s going to replace lawyers. We don’t have to worry. I think about robot lawyers, what it is going to do. It’s going to shove out or shove aside, the lawyers who refuse to become at least conversant in technology and conversant in both the benefits and the risks of technology, including specifically AI lawyers who are proficient and savvy in AI may very well replace lawyers who are not so savvy. But I don’t think we have to worry about unemployment anytime soon.

Rocky Dhir:

I was counting on unemployment so I wouldn’t have to do this anymore. Judge Ferguson, what about you? Assuming you agree with John that we’re not going to be phased out of employment by ai, what’s the role that you think AI is going to play in the future legal landscape?

Judge Roy Ferguson:

There’s the good side and the bad side of that, of course, the good side is when it comes to administrative efficiency, AI is revolutionary. If you have staff who knows how to use it, then you can take off of the shoulders of the lawyer. Things that the lawyer shouldn’t have fingerprints on anyway that we currently have to do in review.

Rocky Dhir:

Sure.

Judge Roy Ferguson:

The flip side of that is it is not intelligence. It is not a human being. It never will be until we have some dramatic science fiction moment and they have cute names for that moment. But for our purposes, there is no way that it can currently evaluate things the way we do. And I was having a discussion with one of the tech experts on AI who’s in attendance here, and he was saying, but it’s very easy for us to put testimony in and generate an affidavit. And I said, does it matter to you the purpose of the affidavit? And he said, well, no, we’re generating an affidavit of their testimony. And I said, so in a summary judgment hearing, when you have to have one contested genuine issue of material fact, the affidavit is very narrow and it might be on a single element of a single claim.

But in family law, when you have to establish that there’s a substantial risk of harm to the child in the current living environment, that’s a much longer list. And there’s a million things they testified about that have nothing to do with that standard. And I said, how does your AI know the purpose, what’s relevant, what you don’t want to say and subject your client to 45 minutes of cross-examination on an irrelevancy in the hearing? And he said, well, it doesn’t. I said, well, that’s where the human comes in. That’s why it’s important. And one of the examples I was given at the AI summit, which Justice Browning oversaw was well, but it can be used for things like teaching people how to serve their petition. And I responded and said, there are so many times that petitions get filed and you don’t need service or you don’t want service. Maybe you don’t want to inflame a situation. Maybe you’re doing it as a venue placeholder. Who knows? But that requires expertise and experience and an understanding of the law and human nature. And her response was, none of that’s true. All you have to do is just have it served. And the look that John is giving me right now, this slow half smile head shake is exactly what the litigators

Rocky Dhir:

And judges in the room. That’s for everything though. That’s Browning’s look on everything he ever does.

Judge Roy Ferguson:

It is his go to move.

Rocky Dhir:

That’s

Judge Roy Ferguson:

It. It’s very effective though.

Rocky Dhir:

When he orders a martini, he’s like, okay, let’s get that martini look. We all know. Okay, let’s maybe get back to the present. Right? So we’ve talked about the future and what it may or may not look like. Let’s talk about the present. What do you think lawyers can use AI for today and do so responsibly in order to serve their clients better as opposed to just doing something to make their, there’s being lazy and not going to do the work. And then there’s trying to enhance yourself using these tools and become a better lawyer.

Justice John G. Browning:

It’s a great first draft. It’s a great having a very junior attorney or clerk or something that gives you a memo, but it’s not going to be a final product. Now that’s true. Whether you’re talking about doing a blog post to market your law firm, a piece of correspondence to an opposing counsel or to a client, a motion or a brief and supportive motion. It’s a great first stand, but things, but it needs a more experienced set of human eyes on it. So I think that that’s the best way of describing it. And let’s face it, a lot of lawyers, the biggest enemy, the biggest adversary is the blank page. How do I start off? How do I do this? What am I going to, this gets you past that blank page issue and goes, oh, okay, well I’d like to make the tone a little bit more friendly. So I’m going to change this word here. Or I, we didn’t cite the such and such cases, which is the leading case on this issue of law. I want to make sure to add that it’s great for a lawyer who is going to be responsible in terms of editing a draft. But make no mistake about it, it shouldn’t have much of a value beyond a first draft.

Rocky Dhir:

Judge Ferguson. Agree or disagree respectfully.

Judge Roy Ferguson:

Of course, I disagree to the extent that if it is an important document that’s going to be filed or is going to have an exhibit sticker on it. Sure. I don’t think first draft is where I have enough confidence to use it. Emails, internal documents, things like that. Summarizing notes and transcripts. Great. But I have seen so many examples of my own attempts where it generates just the wrong standard. We did a test on all of the major gen AI platforms about six weeks ago. So it’s a little dated, a little dated

Rocky Dhir:

Six weeks is a little dated.

Judge Roy Ferguson:

It is in the ai.

Rocky Dhir:

Oh my goodness. Wow.

Judge Roy Ferguson:

It was before 4.0, but it was after four. So it was in that zone. And we asked it to generate a brief less than 500 words on a specific issue. We asked it to argue a certain side of a legal issue that was the wrong side. The law does not allow you to do what the argument that was prompted. Four of the five generated fully fabricated incorrect statements of law very eloquently with case sites and rules that didn’t exist. And one of them said there was a good cause standard that doesn’t exist. It was a yes or a no legal issue. Only one of them said, I can’t do that because there is no viable argument that I can find. Call a lawyer. So the other thing is I was sitting here thinking maybe the second draft, maybe I could say I can’t say that either because when you upload a brief or something that is legally significant and you ask it to change the tone, which is the test I did, I found that it removed legal words of art and specifically legal terms that had been carefully placed into the letter or email or

Rocky Dhir:

Whatever it was. It might be important for the overall graben of the email,

Judge Roy Ferguson:

Not

Rocky Dhir:

Just, I just want to use the word grab. It’s just positive,

Judge Roy Ferguson:

Substantially related. Sure. That’s a legal term we use, right? If you put it in and say, make this easier to read or more friendly or less formal, it’s taking out substantially related and it’s putting in connected to,

Well now you’ve destroyed the legal significance of your brief letter, email, et cetera, right? There are magic words that every lawyer must use and they go into those documents for a very specific reason because the court of appeals, when Justice Browning was on it said, these are the proper words to evaluate this. And Chad GPT will take them out because it wants it to read more layman with less legal terminology to scare people. And it destroys the entire purpose. It changes the meaning of that argument in your brief. So I’m not sure we’re ready for prime time yet for something that’s going to be filed or offered as evidence because you will have to answer for it. If it’s an email and it goes into evidence, you’re going to be asked, read that out loud. Did you write this? What did you mean? Sure. I’m not sure. In your brief who’s going to be sanctioned? Is Chad GPT going to be sanctioned? Is Microsoft going to be sanctioned or are you going to be show caused in front of the judge and find $5,000 by a New York court?

Rocky Dhir:

So here’s maybe the question then. I think we’ve talked a bit about how you have to be careful in judging AI and the use of ai, but when lawyers are trying to implement versions of AI into their practice, because of course now we also have an ethical duty to be technically proficient as lawyers, right? So we’ve got these two competing interests

Justice John G. Browning:

And clients may expect it as well. Maybe a very client driven, why didn’t you use ai?

Rocky Dhir:

Yeah, why is it so expensive? Why’d you do this all yourself?

Judge Roy Ferguson:

I plugged into the chat GPT myself, it took two minutes to get the answer. Why did you charge me six hours of legal research and

Rocky Dhir:

Drafting? There you go. Yeah, exactly right. So what should lawyers be looking out for when trying to, what are some of the questions they should be asking themselves rather when they’re trying to implement AI into their law firms?

Justice John G. Browning:

Well, first they need to familiarize themselves with the functionality. What can it do? What can it do? What are its limitations they need to familiarize themselves with? What are the risks? Risks may very well include, like Judge Ferguson mentioned these hallucinations, but it may also include some other risks associated with using it, such as sacrificing confidentiality. If you are using an AI tool that’s then going to go out of a contained environment and search hither and yawn for things

Rocky Dhir:

Hither and yawn,

Justice John G. Browning:

It’s almost as good as grin, but not quite.

Rocky Dhir:

No, no. Gman was mine. But hither and yawn might actually, I mean, yawn may be if I’m tired and bored, but

Justice John G. Browning:

Yeah. But it is potentially going to open up your client’s confidential information to a whole host of third parties that should not be getting at. So lawyers have to be aware of those risks.

Rocky Dhir:

Well, guys, it looks like that is all the time we have for this episode. We have to keep this pretty brief, but I want to thank you guys. Thank both of our guests, judge Ferguson and Justice Browning for joining us today. That is all the time we have for this installment of the State Bar of Texas podcast. Thank you both for joining us.

Justice John G. Browning:

Thanks for having us.

Judge Roy Ferguson:

I’ve enjoyed being hither. I’m going to head yawn now.

Rocky Dhir:

I’ve been yawning this entire time, and I want to thank our listeners for tuning in. If you like what you heard, please rate and review us in Apple Podcast, Google Podcast, Spotify, Amazon Music, or wherever you get your podcasts. I’m Rocky Dhir signing off for now. Until next time, thanks for listening.

Continue Reading