Can generative AI tools make it easier for lawyers to offer pro bono services?

Episode Notes

Transcript

Announcer:

Welcome to the ABA Journal Legal Rebels podcast where we talk to men and women who are remaking the legal profession, changing the way the law is practiced, and setting standards that will guide us into the future.

Victor Li:

If you’re a regular listener of the show, well thank you. But also, you know that we talk about a wide range of topics, but two things that we end up talking about a lot it seems are generative artificial intelligence and the access to justice gap. So I figured let’s talk about them both in the same show. When it comes to access to justice, it is long been estimated that low-income Americans did not receive any or enough legal help for over 90% of their civil legal problems. Maybe they can’t afford a lawyer or they don’t think they need one, or they don’t understand enough about their problems and potential consequences. Either way, there’s a vast portion of the population that is not being served by the lawyers out there. One commonly cited solution to helping bridge this access to justice gap is for lawyers to just provide more pro bono work, and while it won’t come close to helping all the people in need, it’s a good first step.

In that regard, have generative artificial intelligence tools made it easier for lawyers to provide pro bono services. My name is Victor Li and I’m assistant managing editor of the A BA Journal. My guests on today’s episode of the Legal Rebels Podcast are Kristen Sonday and Laura Safdie. Kristen is Co-founder and COO of Paladin, a legal technology platform for pro bono programs. Laura is co-founder of Case Texts, a legal research service that was recently acquired by Thomson Reuters. They’re here today to talk about how generative AI is changing the face of pro bono services for lawyers and the clients they serve. Welcome to the show, Kristen and Laura,

Kristen Sonday:

Thanks for having us.

Laura Safdie:

Thank you. We’re happy to be here.

Victor Li:

So obviously I just gave the very, very quick version of your bios. Can you tell us a little bit more about yourselves and your backgrounds? Let’s start with Kristen first and then Laura.

Kristen Sonday:

Sure. Well, thanks again for having us. So my background is with the US Justice Department doing international criminal work in Mexico and Central America, and it was really doing this work in Mexico City where I got to see firsthand how complicated our judicial system is to navigate. And I always say that it’s hard enough if you’re American, you speak English and well-educated, and if you’re not, it can feel really impossible. And so I got to see the value of having an advocate with you to help navigate our system and how it could really mean the difference between winning and losing your case. And that’s how the kernel for paling got started. After DOJI joined the founding team for tech company in New York that we took through Y Combinator, we scaled it internationally and it’s where I learned how to build a business from the ground up. And now Paladin is this great intersection of figuring out how do we leverage technology to help more people get access to lawyers and pro bono resources, specifically to help them navigate our legal system in the most efficient way possible and hopefully result in better outcomes.

Victor Li:

Alright, and now Laura?

Laura Safdie:

Yeah, I’m Laura Ty. I am a former litigator. My legal aid access to justice origin story probably happened when I was a clerk in federal court in Manhattan. Obviously saw a very broad range of matters in that role, but for the first time was exposed to what it looks like in a riddle practical sense when you have, for instance, a person who’s imprisoned and is trying to get access to the legal system without representation, looking at the handwritten pages on a notebook by a person who is desperately trying to get their story told to the right people without the tools or legal knowledge or experience that one needs to properly avail themselves of the justice system in America. And so that context along with just experience practicing led me to join up with a good friend of mine who was also an attorney, but a former engineer as well to found case text where our goal was to avail legal technology of the more advanced tactics of machine learning, natural language processing, and earlier generations of AI solutions to provide better, more accurate, more efficient solutions to attorneys and therefore the clients that they serve.

And so we built case texts for about 10 years and were, I think pretty well positioned when the large language model technology was beginning to be developed now like 20 17, 20 18, were really involved in that space, involved in working with a large ual large language model, developers testing it against professional use cases, had a really high threshold for what we considered appropriate, safe and reliable for professional use, specifically for extremely sensitive uses in law and kind of kept watching the space very closely. But things changed materially when we first had access to GT four, and we will talk more about that later, about very quickly, seeing that the kinds of solutions enabled and the way that we thought about how lawyers interact with machines, it was about to evolve in really significant and frankly transformative ways. And so we really leaned in to leveraging AI to help provide more capacity and leverage for lawyers. And a little less than a year ago, we merged with Thomson Reuters. And so now I’m a TR and in this really interesting and exciting space of figuring out how we can best leverage AI to support the legal community. And I’m lucky enough to be in a position to really focus in on what that means specifically for pro bono and legal aid. Yeah,

Victor Li:

I think a lot of us learned about GPT through you guys at Case Text, so I guess thank you for that.

Laura Safdie:

It was a interesting moment, right where they actually launched GPT-4 the same day that we launched. And so it’s been exciting to be in this place where we’re just necessarily every day building something really new. It also puts us in this incredible position with law firms in-house legal departments, but really excitedly LSO and pro bono counsel to figure out what does this mean for access to justice? What does this mean in terms of use cases that we can deploy literally today to help us better serve clients in need? And so I know we’ll dig into that today.

Victor Li:

Yeah. So let’s talk about, I guess before those generative AI tools came about. Let’s talk about what it was like for lawyers providing pro bono services. And you guys, well both of you seem really good at finding pain points and addressing them within these processes and whatnot. So let’s start with lawyers working in legal organizations. What was it like for them? What were some of the pain points and what were some of the things they did well and some things they needed help with?

Kristen Sonday:

I think it’s first important to just level set on the magnitude of the justice gap in the US so that we have context for how important this work really is. So the justice gap is really the difference between the civil legal needs of low-income individuals and the resources that we have available to meet those needs. And as you referenced the Legal Services Corporation, which is the largest federal funder of legal aid in the country and the largest federal funder period, they put out a report recently that shared that 92% of low-income individuals civil legal needs are either not met at all or they’re inadequately met. So we have this huge gap despite attorneys having a professional responsibility to do 50 hours of pro bono work per year and hundreds of millions of dollars of funding flowing through the legal aid system. In addition to this huge gap, these legal services organizations, lso, as we call them, they have to turn away about 50% of low-income people who qualify for pro bono services.

So that’s meeting either 125% of the federal poverty level or below just because of a lack of capacity. So the funding and the resources are simply not enough, and we are in a civil legal aid crisis. That is why pro bono services are so important to helping solve this gap. The AmLaw 200, just as a group amongst themselves, they contribute about 5 million pro bono hours per year to help us make a dent. About 37 million people are affected by a civil legal issue. And again, we’re only able to solve about 8% of those fully. So that’s why it’s really important that we invest in this work. And traditionally, the legal aid ecosystem has been so underfunded and under-resourced, despite their amazing work on the front lines, that really creates a challenge of course, and it also creates a really big opportunity for us to build from the ground up and help implement some powerful technologies.

When I think about the areas where AI can help the legal aid ecosystem system most efficiently and in the near term most urgently, I think about it, the work in two different buckets. So the first is around practice management, and that’s building end-to-end solutions for managing a specific type of case specific practice area, and then also workflow management solutions. So that’s where Paladin sits. So thinking about how do we automate and improve the efficiency of these organizations’ workflows, there are two main areas within the workflow part of it that we think a lot about. The first is around intake. So traditionally, legal services orgs have been really great at meeting potential clients where they’re at and raising awareness of their services. So they partner with religious institutions, schools, government entities, and other community organizations to make sure that people are able to reach them via email, text, phone, video, and gain access to information materials and ultimately attorneys who can help them.

The challenge is that leads to a really high volume of clients interested in working with them, which creates a bottleneck for the system. I spoke with a really large legal services organization in North Carolina right after the pandemic, and they were saying that they had hundreds of people calling into their hotline every week, specifically around housing and unemployment issues, and they were just unable to service the majority of them. So in addition to that, everyone’s working off slightly different criteria among who qualifies the types of services they can provide. They’re using different tech stacks. Their funders might have different qualifications for reporting data and analytics and KPIs that they have to be attuned to. And so there’s no standardization around how we’re intaking clients and referring them out, which seems like a really big opportunity here. And then lastly, there haven’t been any new tech stacks with advanced technologies recently where folks can share this data and information workflow across the board, which is suboptimal and inefficient for both the clients themselves and these lso who are spending tremendous time and resources in taking clients.

So that’s a big opportunity I think, on the intake side. The second is around pro bono referral. So of course this is where a Paladin sits. And at Paladin, our whole mission is to increase access to justice by helping legal teams run more efficient pro bono services and programs specifically. And so we provide a free version of the tools to legal services organizations where they can input their clients once into the system with a click of a button they can send them out in real time to law firms, corporate teams, bar associations, US government attorneys to help create for the first time a one-stop shop for people to find pro bono cases that are going to be a good fit for them where they can really add value. And that’s never existed before. Prior to Paladin. Let’s say that you are a law firm with five offices and you typically work with five L SSOs in each of those cities.

You are fielding pretty regularly, mostly on a weekly basis, inbound emails of case lists of clients who need legal services. You’re having to copy and paste and curate and refer out those cases to different groups within your firm, which is incredibly manual and laborious. An attorney says that they’re interested by the time you go back to the LSO, the case has already been taken by another firm and it’s a bad user experience essentially for the lawyers who want to get involved and are interested in cases. But it’s another barrier, the manual process of it all. And so by automating the workflow, by being able to intelligently intake these clients, identify the right parties that would be a good fit to take on these cases, and then refer them out seamlessly in both the proactive and reactive way where attorneys can find them, that I think is going to be game-changing in leveraging advanced technologies and AI to help people get matched a lot sooner with clients that they know that they can help.

Laura Safdie:

And I can speak just very briefly to the practice management side. That’s a very broad set of needs. Everything from just mechanically, how do you manage all of your cases to the substantive work of representing clients, whether it’s legal research, drafting, document review and analysis. Historically, what we’ve seen is L SSOs specifically because pro bono lawyers tend to have access to the tools that their firms have access to, but L SSOs specifically don’t have the resources to subscribe to many of the more advanced tools. They also don’t often do not have dedicated tech or IT teams to actually implement them. And so oftentimes we’re finding LSO is doing quite a lot of work manually, not having access to the most advanced commercial grade tools within the law and also leaning even on relationships with law schools to access those tools. So really piecing things together based on limited access to resources and limited dedicated technical staff. And so that’s kind of the environment in which we’ve all walked into with this latest advent of AI supported tools.

Victor Li:

This might be two hypothetical or maybe even too off the wall or too much of a fantasy. So based on what you guys have been saying, it seems like the biggest obstacles here are funding. In addition to that, there’s also staffing and access to technology and things like that. So let’s say we live in a perfect world, Congress appropriates tons of money to the Legal Services Corporation into all these other various legal aid organizations out there and whatnot. Bar associations jack up the pro bono requirement from 50 hours to let’s say 200 or something because everyone now has so much free time and then they have all this access to all this great technology. Would that even come close to solving the problem or would there still be issues?

Kristen Sonday:

This is going to be a multi-prong front in order to help make a dent in the justice gap. Pro bono lawyers can definitely be a small part of it, and of course we advocate for as much pro bono as possible, but at the end of the day, it’s going to take a combination of technology funding, increased capacity from other legal professionals who can provide services that are not giving legal advice. It’s going to take regulatory reform, working with the courts to overhaul their processes, et cetera. So it’s really going to be a cross-functional effort in order to make the system more accessible to all. One thing that we talk about a lot in tech and at startups is the user journey and thinking about every touch point that a user might have across your product or your service. And I’d say we need to apply the same framework to the justice problem.

Thinking about the first moment that an individual encounters their legal issue, how do we help them realize that it is actually a legal issue? Most people will just see it as an issue. How do we help them find the right resources online and the right organizations to reach out to help them with it? How do we get them a pro bono lawyer or help them solve the problem themselves? And then how do we work with the court system and all of the other stakeholders who are going to be involved in that process for them, manage the communication and the experience in a more streamlined and frankly, non-legal jargon, natural language way so that everyone understands what’s happening across the entire journey.

Laura Safdie:

I totally agree with Kristen, and I’ll add just a couple follow ups on that. One is that every kind of vector of improvement that Kristen just described will of course make incremental improvements when there’s real dedication of resources, focus man and woman power to back this up. However, there are definitely some investments that we might expect to have more of a step change impact. And I think that’s one place where technology can play a really critical role. So for instance, when we’re talking to an LSO and they’re a hundred percent on paper or they have paper files, paper intake, when you get them onto a more scalable technology solution for case management, that is a step function improvement in their ability to cut out 20%, 25% of their time spent on admin tasks, lawyers spending that time on admin tasks, what they should be spending representing their clients.

And another version of this, of course, which we’ll talk about is the kinds of solutions enabled by ai. The amount of time that lawyers across all practice areas, but especially in pro bono SSOs that are spending on the kind of work that actually now can be competently delegated to a machine. When you can incrementally increase the number of use cases and the amount of your day-to-day client representation work that can be competently, safely and reliably delegated to an AI solution, then you’re spending more of your time as the attorney representing clients, making sure that their client user journey, so to speak, is seamless. That you are optimizing across all the ways in which you’re representing clients and ultimately representing more clients across more needs. So totally aligned with the concept that this is multi-pronged, and as we’re thinking about where to make these kinds of investments, it’s how do we dedicate more resources into the system as it exists today to incrementally make improvements today, but how do we make investments in doing things differently and changing operating models so that we can get those step function type of changes to drive real headway into what otherwise has always seemed to be this insurmountable gap between the need and the representation allotted.

Victor Li:

Great. Alright, before we continue, let’s take a quick break for a word from our sponsor. And we’re back. So now let’s talk about these generative AI tools. What are some of the ways in which pro bono lawyers have used them to help them better represent people in need?

Laura Safdie:

That’s a really important question. I wonder if I might take a second to level set on what the it is here. What is gen AI and what kind of difference is it making because it’s not new that technology solutions can assist pro bono lawyers in serving their clients? What’s new is how powerful this new generation of AI metals are and the new kinds of solutions that this technology enables. So in just over 18 months since the launch of Chay UBT, there really has been a fundamental and profound shift in the types of tasks that we can expect machines to competently do for us. It’s allowed legal professionals the opportunity not just to get tasks done more quickly, but to really think about how we’re transforming what’s possible to accomplish in that work. And I really believe that there’s nowhere where the impact of this breakthrough is more needed, but also nowhere where it has the potential to drive more social good than in pro bono and legal aid.

It’s because this technology has made possible really for the first time ever, a reliable professional solution that can competently handle a broad range of very substantive tasks. The kinds of tasks that we as attorneys do all the time, research, drafting, document review and analysis, and it’s not just fast. It really can read, analyze, and write at what we think of as a postgraduate level, which means it understands context and complexity in a request and in what’s required to fulfill that request, allowing it to complete different types of work. And so when we think about what that enables, it can look like rather than spending four hours reviewing a hundred pages of documents in detail, you can put it into an AI system and ask really complex questions against it. The kinds of things that you couldn’t traditionally search that would really require a human to review in full and use their own synthesis and knowledge and interpretation to answer those questions can be done in minutes.

You can look at a database of a litigation record and ask really complex questions about witness testimony or inconsistent with this testimony, things that you couldn’t ask before. And so that can actually manifest in a very, very broad range of types of tasks across in the same framework that Kristen laid out earlier across intake, across workflow management and across actual legal practice. One kind of caveat to all that is that I think when most hear that description, they might think, yeah, that’s chat GPT. But a really important constraint that certainly I’m thinking a lot about and attempt to do as much as we can to educate users on is the difference between consumer grade AI solutions and professional grade AI solutions. And so just to give the very high level on that, AI provides extremely powerful processing engines, but you can point that engine at a bunch of different places including different sources of data.

So when we see lawyers leveraging chat GPT for legal work, it creates a lot of risk because they’re relying on chat GPT as a source of knowledge. And that’s where we get some of the scarier examples that we’ve seen like chat GPT lawyer filing briefs with made up cases and made up citations. Because what large language models do really well is they can lie to you very convincingly. And so we’re very, very concerned with lawyers thinking AI is a monolith, like it’s just one set of things. The large language model is an engine, and then it really matters what car you put that engine into or set another way, what safety mechanisms you build around that engine. So legal AI solutions or those that are built specifically for legal practice will take the power of that engine, but they’ll surround it with the kind of substantive reliability and accuracy controls as well as data privacy and confidentiality controls that are required for law. So we can talk about that in more detail now across those different types of use cases. But that’s one thing I did want to flag is as we’re thinking about this issue and the opportunity to leverage this technology, it’s also thinking both opportunistically but also in a sense making sure that we mitigate the risk of misunderstanding the appropriate applications of certain types of tools to legal solutions.

Kristen Sonday:

What I think is really exciting is that we are just now starting to see some different tests and initial applications of this technology to figure out where it can add the greatest value to the Lee laid community. And I’d say we’re really in the bottom of the first inning here on understanding what’s possible. And there are a couple really cool examples that I think are going to be indicative of where we can apply it in the future. So one of them is that we’re starting to see more LSO partnering with universities and law schools to build out smarter intake forms that include smart navigation and make it easier for people to understand what their issues are and where they might get routed. They’re incorporating natural language processing. And that’s really powerful because NLPs can help us analyze the language that clients are using to help us identify their key issues.

They can help us categorize the cases and the type of pro bono matter that it might end up being. And it can also detect the urgency of potential matters to make sure that folks are giving it the right attention that it needs. Another example is Joseph, which is a great legal tech company. They’re working on a project with a court system in New York where they’re training AI chatbots to respond to users’ hotline questions when they call in real time. But what’s really cool about this example is that they are working with legal aid lawyers who are acting as intermediaries who are able to essentially QC the answer and make tweaks to it before it goes out back to the recipient. And the idea is that over time, as lawyers continue tweaking the answers and understanding the types of questions and types of answers that the GPT are giving these clients, the organizations are going to gain a lot more confidence in the answer and the information that’s going out that they can ultimately open source more broadly. So I do think that leveraging this technology with a really hands-on approach from lawyers who can help control the information flow, train the data over time, that’s going to really empower the technology to help serve clients in the most efficient and accurate way possible to Laura’s point to help account for some of the risk management parts of it. And Laura, I know you see a ton in your day-to-day, I would love to hear more about what you all are working on too.

Laura Safdie:

Yes, absolutely. So we’re focused more as you know on the actual substantive work of legal representation and what it means to leverage this technology to help support and provide leverage and capacity through a broader range of these tasks. And I’ll say, just to reinforce your point about it being the first inning right now, it is clear that this technology is incredibly powerful and we are every day learning about and developing new use cases of ways to apply it depending on the practice area and depending on the needs of the clients and of course the SSOs themselves. But we have worked really closely with some early partners in the legal aid space to get a deeper understanding of their day-to-day needs and needs of their clients and how we can deploy this technology. And it has been just stunning the breadth and depth of the applications that we’ve seen.

So one example is we work closely with the California Innocence Project and of course they do exoneration work, and I came in with an assumption that a lot of the same litigation use cases that we see will be what they’re most interested in. And they were, of course you can upload an FBI file and automatically create a timeline of events in minutes that project would possibly take a week for someone focusing on really reading everything and parsing it out and structuring it literally takes minutes. You can review Innocence Commission reports and ask it to answer really complex reasoning questions like, has the defendant maintained innocence throughout the case? Was the defendant offered a plea deal and refused it? These are not searches that we could historically do. These were always uniquely human, but beyond the use cases that are really common within litigation and appellate work, we actually were stunned to see that one of their biggest pain points is just a huge backlog of applications for representation.

There were hundreds of applications from imprisoned people. The applications include the entirety of a litigation record and they just did not have people, they didn’t have the bodies with the experience to review that content and make the judgements as to whether there was a strong basis for appeal. And so we worked with them to create and very, very simply just upload the litigation record and ask really complex questions against that application. And they were able to produce, again, literally in 20 minutes as opposed to taking one or more weeks to actually review each application, a report of where you might see in the record evidence of inconsistent testimony, whether the argument for exoneration makes sense or whether there are gaps. Does the prosecution’s case make sense or are there gaps? And so that really showed us that when we dig deep with the experts in the field, the people who are actually doing this work, we are uncovering use cases.

They’re wildly compelling and also kind of beyond what we ever thought technology was capable of. We’ve seen the same thing in immigration, the broad range of use cases from searching an administrative record to answer a question and receiving a memo with citations to supporting files literally in minutes summarizing long complicated documents like country reports or IJ decisions, the immigration court transcript. These are the types of activities that are extremely detailed and time consuming for humans to do, no matter how seasoned or experienced, how amazing of a lawyer they are. We are ultimately still human and exceptionally strong applications of ai. And it goes beyond just understanding what is in a provided set of documents. It goes all the way towards drafting. You can automatically draft a letter to DHS and enclose a FOIA request. You can automatically draft initial objections to discovery request these capabilities.

In short, they save attorneys an enormous amount of time. It allows them to conduct much more rigorous and comprehensive reviews of large data sets than they can do when they’re doing it manually. Because remember, the counterfactual is not usually that humans are perfect. Usually when humans are doing this work under kind of time and pressure constraints, you see errors, you see misinformation that’s common. You can do a very efficient, comprehensive review of information that also reveals more of the material information that the attorney often needs to build their case. And it lowers ultimately the amount of time required to serve an individual client or an individual legal need by just increasing capacity and leverage. And so that’s to our discussion earlier of what’s it going to take to really make a dent collectively? These use cases are cutting hours and hours, days and days, weeks and weeks of work that it takes to really competently represent a client and it allows the attorney to focus their effort more on the actual client service, on being there for this person in need in executing judgment and building their strategy.

And that time, by the way, is precious for the attorney and the LSO, but it cannot be more precious for the client themselves that’s mired in a legal conflict that is desperately seeking asylum, that’s sitting in jail. All of those days are just enormously costly and painful. And so that’s just a short list of examples that we’ve seen, but this is not just within immigration and exoneration work. We see it in employment discrimination places, housing, domestic violence. It’s the opportunity here cannot be overstated, and right now is only limited by our ability to make it simple, accessible, not intimidating, and really value oriented to roll out these solutions around specific use cases that help the lawyers at an LSO or the lawyers doing the pro bono work as well as their clients.

Kristen Sonday:

Another benefit of these enhanced tools for the practice of law in pro bono is that it opens up a greater breadth of opportunity for pro bono lawyers to get involved in types of pro bono work that they might not have gotten involved with before. So when we think about increasing capacity of pro bono lawyers, specifically in the private sector, the two pieces of pushback that we get are, number one, I don’t have time, and number two, that’s not my practice area. And so folks are intimidated or afraid to take on a new type of case if we’re decreasing the amount of time that it takes to resolve a pro bono matter, and we’re equipping the pro bono lawyer with a ton more resources and information and data and knowledge and training through these AI tools to make the process more efficient and informative, the more people I think we can get to engage in pro bono longer term.

Laura Safdie:

I totally agree. And there’s also an attorney development and education piece of this as well. If pro bono matters become less about doing the work of pulling together the building blocks, which can often be very, very time consuming and more about client representation, more about representing the client in a hearing in court or in front of an ij, it becomes more appealing for attorneys, especially younger and mid-level attorneys who really want their moment in court, who really want to learn more about client representation and client service to spend more of their pro bono hours actually working with clients than deep in documents

Kristen Sonday:

And also using AI and advanced technologies in pro bono cases translates well to their billable work. So it’s a win all around for professional development too. It’s a great point.

Laura Safdie:

Of course, lawyers are doing this work mostly out of the good of their heart, but it doesn’t hurt if we make it beneficial to them also in their own career development.

Victor Li:

What are some things that maybe, obviously we talked a lot about what great things that they’re capable of. What are some things that lawyers should be careful of when they’re using these tools?

Laura Safdie:

Yeah, so me get a little deeper in the conversation. We started earlier around the difference between consumer grade AI tools and professional grade tools. The real distinction is what are you using as your source of knowledge and is that source of knowledge safe, accurate, and verifiable? And that I think is the crux of it. The biggest drawback I’ve seen now, now we’re a little over 18 months with lawyers even having access to these kinds of tools is there is a learning curve to just understand the landscape and to understand that AI is not a monolith. Not every tool that someone slaps an AI label on is equally appropriate for the requirements of legal practice. And so if you are a lawyer that is leveraging chat GPT for substantive legal practice tasks, first of all, anywhere they could open AI on their website says, do not use this for legal representation because they themselves say that it isn’t a verifiable accurate source of data for legal information.

That’s the wrong way to use ai. The right way to use AI is to ensure that you’re using tools that point the power of a large language model, the processing power that allows you to ask these kinds of complex questions against large data sets, but that points it safely at reliable sources of information. So that could be primary law, that could be the litigation record, that could be a database of contracts, that could be your own firms if you’re on a pro bono team, your own firm’s historical database of pro bono briefs, leveraging that knowledge that you know is reliable. Now, it isn’t just saying, okay, that’s the information that you plug into a system and the system gives you an answer and it’s a black box and you just trust it. That’s another critical step is the actual verifying the source of the answer.

And so that’s where the second place where I’d say people can get tripped up is, oh, the AI said it’s true and therefore it’s true, and I will copy and paste it and submit it. Although, to be honest, I think other than a very, very small number of exceptions, lawyers really do understand that they hold the ethical obligation. These are tools, they’re exceptionally powerful tools, but they’re ultimately just tools. And it’s your job to review and validate the information that you’re submitting to a court or submitting to a client for advice. The last thing I’ll say on that point is I think the tools that are built properly for legal make it really easy to verify. And when we train lawyers across private practice to legal aid, we’re really emphasizing not just the fact that you need to check, but also the fact that you really should think about these delegating tasks to these tools in the same way that you would think of delegating tasks to a paralegal or a junior associate.

If I asked a junior associate to draft a first draft of a brief for me, I would never copy and paste that brief and file it right to the court. I would review it, I would review the sources, I would review the arguments, I would review the citations to the record, and I would validate that it makes the arguments that I think it should make and that it does it in a way that’s robust and inaccurate. And so it’s the same idea. No one who is developing AI tools is suggesting that you should treat AI as if it is a magic ball. You should treat AI like a really, really powerful tool that it’s your responsibility to use properly and to leverage. And so when we partner with pro bono teams and lso, but really any legal professional that’s beginning down their journey of understanding AI and beginning to deploy it in their work, we’re always emphasizing that fact that we’re doing so really intentionally and we’re delegating with enough skepticism like you would apply to another colleague on your team.

Victor Li:

So let’s take another quick break for a word from our sponsor and we’re back. So looking forward, I mean obviously both of you talked at length about these tools and how reliable they are and difference between commercial grade and professional grade. But for people who might not necessarily understand this distinction or follow these types of arguments and whatnot, what is the danger of these tools? Being out there and having all these unrepresented or underrepresented people just be like, okay, well, I’ll just rely on chat GPT or whatever and I’ll just go into court or go to my hearing or whatnot and I’ll be fine because I’ll just rely on this or I’ll just use this tool and I don’t need a lawyer. I’m fine.

Laura Safdie:

I think that as we see this technology develop and we’re just in a moment in time and all signs are pointing toward, technology will continue to develop very, very quickly and increase the rate of change more than we’ve ever seen before. I think what it means is that ultimately it’s hard to imagine a world where we don’t have solutions that will serve individuals in need directly. That said, it’s the responsibility of those who develop those tools to ensure that they are properly equipping the users to know what kinds of questions they can and should ask that they can get reliable results to and what is inappropriate. And that’s one reason why I personally approach direct to consumer tools within legal with a high degree of not skepticism, but I want to be extremely intentional with how we do this because they’re not lawyers. And so one really important point of the answer we had before, which was how should lawyers engage with this technology today and what are the risks?

Well, lawyers are really used to validating their sources. Lawyers are used to double and triple and quadruple checking things and having a degree of skepticism and really making sure they’ve turned over every corner in a way that consumers just usually aren’t. And so I think we need to be really cautious about how we step into offering tools directly to consumers and really have a group of people who are experienced within ai, experienced within the legal space working together to figure out how we want to approach that problem. I do think that this could ultimately be one of those step function type of changes that we talked about earlier when we can actually deploy AI tools directly to consumers. But right now we’re just not there. So I will continue to tell anyone who asks that leveraging chat, GPT by underrepresented or unrepresented people as a source of legal knowledge can be really quite dangerous.

And so I’d like to see us leaning more into how do we amplify the impact of the lawyers that are serving these clients so that they can do more for more people than having these tools go direct to consumer. If you came back to me in two years, I think we’ll probably be in a different place then. I think we’ll probably have enough maturity around this technology, enough maturity around the applications, even better safety controls that we might be in a place to really develop direct to consumer legal aid solutions. We’re just not there today.

Kristen Sonday:

So my take is that I agree with Laura and with millions of low income individuals in this country who have urgent civil legal needs, the reality is that they’re already using AI to solve their problems. They don’t necessarily know that their issue is a legal one. And without help legal spotting or issue spotting, they’re going to go to Google, they’re going to go to chat GBT, and they’re going to try to solve their problem on their own. If folks get turned away from SSOs, which is about half of people who qualify, they’re going to go online and try to find resources to help solve the issue themselves. They’re going to go ask their friends and family who might look it up and give them answers that they find online off of chat gt. So the reality is that these answers already exist online, and we need to be really collaborative and thoughtful as a community about how we can push out technology faster that at least helps them get more accurate and authoritative responses to their legal issues.

One thing that I think is critical to alleviating the risk that this open source model causes is just massive collaboration among the legal services community, legal technology companies, academia, law firms in the private sector. Because traditionally, one of my challenges with Justice Tech is that it tends to be built in silos. One LSO is working with a tech company, a law firms collaborating to build something in-house, and there’s little standardization or information and knowledge sharing across all of these projects. And so this is an opportunity, as Laura said, this is a moment in time where we can come together and share best practices and learnings and knowledge as these new projects are getting spun up to make sure that we’re building really responsible AI tools, not just for legal aid and pro bono lawyers, but ultimately the end users. I

Laura Safdie:

Totally agree. And there are some amazing examples where we’re essentially pointing the power of this technology at reliable sources of counsel, whether the example that you gave earlier with the court LSO is directly having resources and just making them easier to access. Also, just the translation services that you can get with AI is incredible. And so you have a list of materials, but you can easily translate it to Spanish, to Russian, to Cantonese, whatever you need. And so I totally agree that there is a moment now where the urgency should push us to do something, even if those of us in the space who understand legal ethics are just really aware of the risks associated with that. But that’s where that collaboration comes in, creating spaces within the legal community to talk about what we’ve seen, what works, what doesn’t. How do we build these tools so that they both can serve a need that’s urgent today, but do so in a way that we are really ensuring that we don’t send these people down a path where they’re not getting reliable counsel from these tools.

Kristen Sonday:

Exactly.

Victor Li:

And so looking forward, what is the potential for these tools and how do you see these tools working with or even independently from lawyers in a few years down the line?

Kristen Sonday:

So one of the challenges currently of the pro bono ecosystem is really one of resource allocation because there is such great need and such little funding and people available to help ’em. So I think one of the biggest opportunities here is to connect intake systems longer term with past outcome records and use predictive analytics to predict the likelihood of a successful outcome for a case based on that historical data, the specifics of the client’s situation, which we can understand better through natural language processing, we can better assess the resource requirements for these cases to determine how we should allocate time and people resources within the legal services organizations, within the private sector, pro bono teams or online resources if we think that this is something that they can manage themselves through self-help tools that are guided through AI or advanced technologies. And ultimately, at the end of the day, these efficiencies will lead to our ability to serve much higher volume of people in need.

And then specifically something that we’re thinking about a lot at Paladin is just around pro bono matching to make sure that we’re finding the best lawyers for each pro bono case to maximize the client’s outcome and chance for success, whatever that definition means to them. And so we are looking at AI on the law firm side to figure out how we match clients with the most suitable lawyers based on the lawyer’s experience, maybe their availability, their case history, their interests, and we can make sure that we’re getting them the right resources that they need based on their background to ensure that they’re properly trained, well staffed, they have the supervision they need and the technological resources at hand to train them really well on a specific area or type of case that they can then replicate at scale to help that many more

Laura Safdie:

People. I would say, at least in the next one to two years, I believe that all lawyers will have an AI legal assistant, one that they can rely on very similarly to a colleague, to delegate a broad range of tasks from substantive legal practice to formatting, to communication to admin work. And that’s true of all lawyers and I think will very quickly become true of pro bono lawyers and LSO lawyers more than anyone because it’s just too powerful to ignore. And that because AI can handle a broader and broader set of tasks, especially as the technology evolves, but even today, it will take fewer lawyer hours per task and it will take fewer lawyer hours per representation, which means that we can expect pro bono and legal services organizations to be able to serve more people in need and to serve a broader set of legal needs, which is just incredibly exciting.

And that’s just speaking about the technology as it exists today. I think right now we’re not in a moment of technological limitations. We’re actually in a moment of human limitations on communication and education change management. That’s what I’ve been focused on a lot recently. Kristen and I are talking at length pretty commonly about this. How do we create systems to make it easier to understand this technology, easier to understand how to deploy it in ways that provide real value right now, easier to develop the skillset of working with ai? Because it is a skill and it’s a skill that’s going to be really important for all lawyers for the future, but as you develop that skill and work with it more, you are better positioned as the expert in the space to identify new use cases that you can deploy it on. So I think that the opportunity is massive and it’s exceptionally exciting, and that’s just based on the technology that exists today.

We’re seeing a lot of signs that the next generation of large language model technology will be a leap forward over the current tech, similar to what we saw GPT-4 was over GPT-3 and 3.5. And just to level set on that, on why that’s so stunning, even to those of us who’ve been working in this space is, as you know, Victor, there were extensive studies on these large English models and how they performed against, for instance, theBar exam along with many other professional tests, and 3.5 performed at a 10th percentile level, and then GPT-4 shot up depending on the test from 70 to 90th percentile, so outperforming up to 90% of the humans who actually take theBar exam in real life. And that was the leap forward to G PT four. And we’re seeing signs that the next leap forward is going to be commensurate in size and impact, which is stunning, but also extraordinarily exciting about the opportunity to deploy this technology exactly in the ways that we’ve talked about direct to consumer to help solve legal needs directly in a way that is responsible, reliable, and informed by lawyers who understand the space. So I think we’re in this incredible moment of opportunity now where the impact we can have on pro bono legal aid and just access justice more broadly has never been this exciting and this significant. And so those of us who are working in this space just want to get to work and make it happen

Victor Li:

So we can expect the next GPT model to get a perfect score on theBar exam. Is that what you’re saying?

Laura Safdie:

I have no idea. I mean, it’s almost like it’s going to raise above our small human brain conceptions of what tests even are, right? Just the idea that it can, what it’s going to look like in practice is hard to imagine and is just exceptionally exciting,

Victor Li:

And it’ll probably make all of us feel stupid

Laura Safdie:

Probably.

Victor Li:

Yeah. To wrap up, if our listeners want to get in touch with you, what’s the best way to do so? If they want to ask you questions or they want to talk about this stuff?

Kristen Sonday:

Yes, exactly. So they can reach us both via email. I’m Kristen, K-R-I-S-T-E-N, at join paladin.com.

Laura Safdie:

I am laura dot [email protected].

Victor Li:

Excellent. Thanks again for joining us today, Kristen. Laura, I hope you enjoyed talking with me about this.

Laura Safdie:

We did. Thank you so much.

Kristen Sonday:

Thank you.

Victor Li:

If you enjoyed this podcast and would like to hear more, please go to your favorite app and check out some other titles from Legal Talk Network. In the meantime, I’m Victor Li, and I’ll see you next time on the ABA Journal Legal Rebels podcast.

Announcer:

If you’d like more information about today’s show, please visit legal rebels.com, legal talk network.com, subscribe via iTunes and RSS, find both the ABA, Journal and Legal Talk Network on Twitter, Facebook, and LinkedIn, or download the free apps from ABA, Journal and Legal Talk Network in Google Play and iTunes. The views expressed by the participants of this program are their own and do not represent the views of nor are they endorsed by Legal Talk Network is officers, directors, employees, agents, representatives, shareholders, and subsidiaries. None of the content should be considered legal advice. As always, consult a lawyer.

Continue Reading