From Hype to Practice: Using Microsoft Copilot in Your Law Firm, with Ben Schorr

Episode Notes

Transcript

Zack Glaser:

I’m Zack.

Chad Fox:

And I’m Chad, and this is episode 598 of the Lawyerist Podcast, part of the Legal Talk Network. Today, Zack is interviewing Ben Schorr about how you can use Copilot in your law firm today.

Zack Glaser:

Yeah. Chad, I’ve interviewed Ben a couple times now about his experience with AI, how you can use AI well, but this one’s going to be fun. This is very specific on literally what can you do today in your office because I think a lot of times we’re just told, “Go use AI and good luck.” So anyway. Yeah,

Chad Fox:

Ben’s great.

Zack Glaser:

Yeah. Yeah, yeah. He’s fantastic. We haven’t done a podcast intro in a while. I haven’t seen you in a while. IRL in real life, as the kids say. But I’ve seen you on Strava recently. And I’ve been- Strava stalking me. Yeah, I mean, definitely. I’ve been Strava stalking you. I know where you live now.

Chad Fox:

You know the trails I run.

Zack Glaser:

I know your roots. Yeah, all of that.

Chad Fox:

Yeah. You know which trees to hide behind now.

Zack Glaser:

I’m always concerned about that. I actually tell my athletes every year to go onto my Strava, because my Strava is public. Go into my Strava and see if they can figure out where I live. And I do that to show them how easy it is to figure that out so they won’t do their public Strava. I don’t want my high school kids making their Strava public. So now I’m going to have to go make mine private.

Chad Fox:

I never even thought about

Zack Glaser:

That. Right? I start my runs from pretty much the same place every single day. So what do you think that means?

Chad Fox:

That’s probably at your house.

Zack Glaser:

I mean, it’s somebody’s house. I’m going to start. Yeah.

Chad Fox:

I have two different paths. I’ll either drive out to a trail or if I’m pressed for time in the mornings when I run during the week, I run here in the neighborhood. But yeah, same thing. I start in front of my house.

Zack Glaser:

Yeah. Yeah. I mean, for all you out there, I don’t start in front of my house that’s somebody I don’t know. I go to a completely undisclosed location every time. So as much as Strava and things like that can be creepy, it’s also helpful because I wake up sometimes and I’m like, “Oh, Chad gave me kudos. Chad saw me running.” And so it’s just virtual community of people that are like, “Hey, man, good job.” Because it’s not always easy to get up in the morning and put on your running shoes or your biking shoes or your workout pants or whatever it is and do it. So how do you do it? How do you get up in the morning? How do you get up in the morning?

Chad Fox:

I need leverage. So I planned to do these Spartan races this year, going to do three. What are your Spartan races? So it’s like the obstacle course races where there’s distance, either a 3K, a 10K or a 21K with obstacles and it’s off-road, trails, mud, cow pasture oftentimes here in Florida.

Zack Glaser:

Oh, like what you think cross country should be.

Chad Fox:

Yeah. Yeah, much more mushy.

Zack Glaser:

Okay. Cross country in the United States is like you’re running on a golf course. It’s really well defined in everything. Cross country in Europe, they intentionally make things wet and intentionally put stuff out to jump over. So we want that and we just won’t do it in our cross country. So we’re like, “Hey, let’s make Spartan races and all that, ” which I think is super cool. How awesome is that? If you’re running as an 11-year-old, that’s what you want to do. I want to dive over fences and

Chad Fox:

Stuff. Yeah, right? So much more fun.

Zack Glaser:

So what do you mean by leverage? What do you mean by you need leverage to get up?

Chad Fox:

So I committed. I signed up for three. I’m going to do at least three this year, the sprint, the super and the beast. And so now we got the dates on the calendar and it’s real. You either get ready or it’s going to suck when you get out there. So those are kind of the choices. Because it’s happening one way or the other. It’s one way or another going to be there. Yeah,

Zack Glaser:

That’s fair.

Chad Fox:

So this morning it was 45, 46 degrees when I stepped out from my run and definitely didn’t feel like it. Oh man.

Zack Glaser:

But

Chad Fox:

Had to do it.

Zack Glaser:

Yeah. Okay.

Chad Fox:

And I did not start from my house.

Zack Glaser:

Yes. Yeah.

Chad Fox:

For anybody listening.

Zack Glaser:

Right, right. I did not start from there.

Chad Fox:

Walked to an undisclosed location and then started Strava.

Zack Glaser:

I flew to a different city and then yeah, it’s not even close to there. So leverage sounds like something that you could do in your business too, right. Yeah.

Chad Fox:

Yeah. Leverage helps. And one of the first things we do in lab is we have them identify what’s important to them and set some personal goals. And oftentimes that is the leverage that will keep people going. Why are you doing this? What kind of life do you want to live? How can you build a business that serves you not the other way around? And that’s the leverage if we get it right. And we usually try to do that. I mean, we always do that in the beginning of the lab journey with our labsters coming in.

Zack Glaser:

Yeah. Well, I remember when I was running my firm, I would often have days that I thought like, “What do I want to do? What do I want to be? How do I want to make it work?” And a lot of times the answers were, “I want to make more money or I want to work less hours or whatever.” But the thing I like about this leverage that you have here, and the thing I think that works is that you have a date certain. You have an impending thing that is going to happen. There’s an idea of like, well, I guess the difference between I want to make more money in my office or I want to take more time off and an actual thing that is leverage is kind of like me being like, “Well, I want to be faster. I want to be quote unquote healthier.” Okay.

Well, I mean, you can do that. Why? Yeah. Why? But why? And the Spartan races, yes, they’re important. They’re fun. They’re interesting. But at the end of the day, I don’t think you’re a professional Spartan racer. And so I don’t imagine that it in and of itself has much effect on your life any more than any other race that would’ve been there. And so it’s really just there to motivate you.

Chad Fox:

Yeah. Being a professional Spartan racer is not my aspiration, but I do want to do much better than I did when I ran the races two years ago. So I did the three races two years ago and I want to do better. I want to be able to do all the hanging obstacles because those are the ones that killed me last time is the-

Zack Glaser:

Hanging obstacles.

Chad Fox:

Like monkey bar across stuff and anything where you have to hold on and go across, those were tough on me. Oh man. Just couldn’t hold

Zack Glaser:

On. Yeah, that’s not talent we learn in cross country. That’s not what I learned running track in college. Yeah.

Chad Fox:

So that’s my goal on this round is to be able to do all the obstacles because if you fail on one, you have to do 30 burpees before you can move on.

Zack Glaser:

Oh

Chad Fox:

Yeah. And 30 burpees, just doing 30 burpees once- Just doing them right. Yeah. Yeah,

Zack Glaser:

If you’re doing

Chad Fox:

Them correctly. But just having to do that set of 30 the first time is brutal. Imagine having to do it six times throughout the race. So yeah, that’s the

Zack Glaser:

Goal. Yeah. Well, good. Well, I want to hear more about your races each time you do it. So I’m looking forward to hearing more about that. I

Chad Fox:

Mean, you want to sign up? Come do one with me.

Zack Glaser:

You’re going to put me on the spot here on-

Chad Fox:

Listen, I mentioned it to Stephanie. I had her thinking about it. Oh,

Zack Glaser:

Man. I’m going to tell you, you got me thinking about it. It sounds very, very cool. Sounds very, very cool. I’ll have to look into it. Another thing that sounds very, very cool is Ben talking about Copilot, and I don’t say that in Jest, it is always fun listening to talk about Copilot. So here’s my conversation with Ben.

Ben Schorr:

Hi, I’m Ben Shore. I’m an innovation strategist at Affinity Consulting Group, and I help people be more successful with Microsoft 365 and AI.

Zack Glaser:

I love that. I help people be more successful with Microsoft 365 and AI. And I know that you do. I want to dig a little bit further. For some people, and I think a lot of people on the Lawyers Podcast, you actually need no introduction, but I want to give you a little bit deeper introduction for those that haven’t been fully introduced or haven’t seen you at tech show or haven’t seen a CLE that you’ve put on. Before you were with Affinity Consulting, you were with Microsofta for nine years. Nine years. What was you did there? Yeah.

Ben Schorr:

So I worked in a group called Customer Success Engineering and my team created and managed all of the content that you find at support.microsoft.com, onlearn.microsoft.com. We also did quite a bit of the in- app content that you would see, whether it’s help in training, little tips, things you would find that would show up inside the apps. Sometimes we would do what we call UX design where a product team is adding a new button or a new feature and they want some text that helps to explain what that feature does. And so we would do some of that. And so that’s basically what we did is we created and maintained content that helped customers be successful with Microsoft 365 and

Zack Glaser:

Copilot.That’s exactly what I wanted to get to was that even before you were here with Affinity, you were not only in the Copilot environment, you were in the, how do I use Copilot environment? Absolutely. So I wanted to make sure that we brought that up because my question for you is, how do I use Copilot, Ben?That’s obviously a big question, but for this conversation, I want to get into some practical stuff of how attorneys can be using Copilot in their office today and just like a couple weeks from now, not way out. But before we do that, I want to differentiate Copilot from this big broad term of AI. Talk to me about what Copilot is specifically in this environment.

Ben Schorr:

Copilot is Microsoft’s productivity AI, and it’s built on the OpenAI platform. And so the same platform basically that ChatGPT is built on. So it’s a large language model. I know Microsoft adds a lot of its own little magic on top of what OpenAI creates, but under the hood, it’s basically OpenAI’s model. Now, recently, Microsoft has started adding some other models like Anthropic to Copilot that you can choose to use in certain circumstances. But yeah, for the most part, that’s what it is. It’s Microsoft’s productivity AI. And one of the things that makes it different from the other ones, and a little bit special for lawyers, the vast majority of lawyers are Microsoft 365 customers. We know

Zack Glaser:

This.

Ben Schorr:

There’s a small percentage that are Google Workspace, but for the vast majority of them, they’re Microsoft 365 customers. And that’s where Copilot thrives, right? Copilot is built in Microsoft 365.

And so Copilot can see all of your Microsoft 365 stuff that you have access to. And so it can see your email, your calendar, your Teams chats, your documents, and the documents that are in SharePoint and OneDrive. And so all those things that are inside your Microsoft 365, Copilot has access to. If you have access to it, Copilot has access to it. I should emphasize that because that’s a question I get all the time from attorneys is, are all of my documents suddenly exposed to Copilot? And the answer is no, Copilot has the same permissions that you do. So you only see what your Copilot can only see what you can see. But that’s a big part of it, is that Copilot’s built into Microsoft 365, so all that stuff is there already. Is there a way that ChatGPT that you could add a connector and see OneDrive with ChatGPT?

Yeah, you can, but it’s not built in. It’s something you have to add on and configure. And then the other part-

Zack Glaser:

But I can see

Ben Schorr:

SharePoint. Yeah.

Zack Glaser:

Okay.

Ben Schorr:

Okay. Yeah, definitely. You can see SharePoint, OnDrive, all of that with Copilot, and there is a way to make that work with ChatGPT. It’s just not

Zack Glaser:

Native. Sorry, I was getting into what we’re trying to avoid right now. I conflated right there, ChatGPT and Copilot. What you were saying is that yes, you could connect ChatGPT to your OneDrive or your SharePoint, but it’s not native. It has to do something else. Whereas Copilot, that’s what it does. Copilot natively

Ben Schorr:

Accesses all of that. Yeah.

Zack Glaser:

Okay.

Ben Schorr:

Yeah. And then the other part of it with Copilot that makes it special is that because Copilot’s inside your Microsoft 365, everything you do with Copilot stays inside your Microsoft 365. So from a security and privacy standpoint, if you have a client document, you absolutely should not be uploading that to free ChatGPT because that’s a huge privacy nightmare. You absolutely should not be updating that to free Grock or Claude or whatever. And not free Copilot either, by the way. I wouldn’t upload that to the free version of Copilot either, but with Microsoft 365 Copilot, you’re signed in with your Microsoft 365 credentials there, anything you upload to it, anything you ask it, anything you have Copilot draft for you, all of that stays inside your Microsoft 365 tenant, so it inherits all the same privacy and security that your Microsoft 365 does. That’s a big deal, especially for attorneys.

Zack Glaser:

Yes, because to me then that is, if you are comfortable with your documents being on SharePoint, then you should be comfortable essentially with using Copilot logged in within that Microsoft 365 environment on those documents. Absolutely. And I think that’s huge. To

Ben Schorr:

Be fair, if you have a ChatGPT enterprise license, then you also get very good privacy and security, but not very many firms do have that enterprise license, and that’s a separate license you have to have there.

Zack Glaser:

And one of the reasons I wanted to talk about Copilot specifically is because not only is that a separate thing that you have to do, it is also a separate hill you have to climb inside your firm to get that enterprise license. You have to go to your IT department, or I mean, you may be your IT department if you’re a small firm, but you still have to go through all these hoops. Whereas the first one, it is much more likely that you, as an attorney, have access to Copilot than the appropriate Copilot mechanism as opposed to the appropriate ChatGPT that you can put client files into. I mean, you can do marketing on pretty much anything, but we’re talking about helping you in your day-to-day being a lawyer.

Ben Schorr:

Yeah. And in fact, these days, if you have a Microsoft 365 license, you get a limited version of Copilot that already is going to be secured by your enterprise data protection. And so just having a Microsoft 365 license gets you a little taste of Copilot. Then you can also get the Microsoft 365 Copilot license on top of that, then you get the whole party.

Zack Glaser:

Okay.That’s how they get you. They get you the little taste and then you want the rest of it. They’re just dealing artificial intelligence to everybody. Okay. So Copilot lives in that Microsoft 365 environment. Most attorneys are on Copilot. Let’s talk about how to use it then in a law office. I know there are a lot of ways that the people can come up with, and I use ChatGPT in a lot of different ways, but again, we’re talking about this internal AI. How can lawyers be using that right now?

Ben Schorr:

So there’s a few different sort of big buckets I think about when I think about how people and attorneys can use it. So one of them would be in the create and edit mode. That’s the one that people think of a lot of times with generative AI. You tell it to draft something and it does.

And so you could, for example, in Word, ask it to draft a letter, a document, a brief, a memo. The more specific you can be with the instruction, the better the results you’re going to get. It is really important to remember that it’s not, in most cases, creating finished product for you. What Copilot’s doing is it’s getting you from a blank page to a plausible first draft in seconds. I have many times done demos for attorneys where I’ve asked them, “Tell me a kind of document you’ve created recently.” And I’ll say, “Oh, a motion for dissolution of a thing or an employment agreement for a tech executive or whatever the document happens to be. ” And I’ll open Copilot and Word and I’ll give it a relatively simple prompt and have it create that document. Obviously it doesn’t have any details because I haven’t gone that deep just for the demo, but it’ll create the document and it creates three, four, five, eight, however many pages.

Then I’ll go through with the attorney, we’ll scroll through it quickly. We don’t read it carefully, but we’ll scroll through it quickly and I’ll ask the attorney what they think. And invariably, the answer I get is, “That’s a pretty good start.”

Zack Glaser:

Okay.

Ben Schorr:

And that is what you got. It’s a blank page to a plausible first draft

Zack Glaser:

In

Ben Schorr:

Second, right? Yeah.

Zack Glaser:

Are you

Ben Schorr:

Going to file that? No. Are you going to send it to your client? No. But you’ve got a document now, you can edit, you can add your own document, your own info too, you can revise it. It gets you a good part of the way there.

Zack Glaser:

Well, yeah, that’s what I was going to ask, man, was, can I send something that comes directly out of one of these AI products directly to my client or directly to the court without reviewing it?

Ben Schorr:

Funny, you should ask. There is actually a rule that has somehow come to be named for me. Schorr’s Law, I think they’re calling it, which you are leading up to, of course, which says never ever show the output of an AI to a client or the court without reviewing it first. Yeah. We’ve heard way too many horror stories about people getting sanctioned for filing fake citations. And that’s just another example of please, please, please review the output before you send it to the client or the court.

Zack Glaser:

Right. And I think that I like that obviously because it’s very helpful, but too, because it leads into kind of what you’re saying here of you’re getting a good first draft on these things. Specifically in Word though, and I like to get very practical with this, specifically in Word, when you say open up Copilot and Word, is that … Because I can find Copilot when I go into Microsoft 365 and I can find it as like a little button that says Copilot. You’re talking about the button, the Copilot button on that upper ribbon inside of Word, and then it opens up a text box in a sense often. There’s a technical term for that box that opens up, isn’t

Ben Schorr:

There? It’s a side panel.

Zack Glaser:

Okay. Okay. So it opens up a side

Ben Schorr:

Panel. So you can do it that way. The other way you could do is when you first open Word, if you have a Copilot license, when you open Word to a blank document, you’ll get a little Copilot prompt box at the top of the screen where it’ll ask you, “What do you want to create?” And you can tell it from there. That’s another way you can do it. So yes, that is definitely a way to do it. Now, of course, we all know that attorneys rarely start from a blank page. A lot of times they’re reusing existing documents and that’s fine too. And Copilot can be helpful there too, because the other element of this is the edit part of it. So you can go through an existing document, for example, and you can ask Copilot to help you edit or revise. And so one of the ways I’ve done that, there’s a couple ways.

One thing I do is I’ll write an article and then I’ll ask Copilot, look at this article and carefully fact check it to make sure that I’m being technically accurate. And then Copilot will go through the article, because most of what I write is technical content, of course. And so it’ll look through there and it’ll find anything it thinks I’ve made a mistake on or been too vague about. And most of the time, thankfully, it comes back and says it looks pretty good, but occasionally it says, “Hey, on page two, you said this and that might be a little vague, you might want to clarify.” Or, “Hey, you said this, but actually that product got renamed last month.”

And

So now that product’s actually called this. So sometimes it catches little things like that. And that can be a really handy way to use it in your editing process as kind of a check. I’ve also occasionally had it look over something I’ve written and said, “Act as an attorney who wants to learn more about this topic. Is there anything in this article I’ve overlooked or been unclear about? ” And so then Copilot will go through it and it’ll look at it and it’ll come back and say, “Yeah, in this section you use this term. It’s not really clear what you mean by that term. A layperson might not understand it. ” So just kind of prompt me, “Okay, I should go clarify that. ” Or, “Yes, you mentioned A, B, and C, but you might also want to mention D because that could be relevant.” And so it gives me some feedback and suggestions.

I don’t always take it suggestions, but it can be a very helpful sort of co-author in that regard.

Zack Glaser:

Yeah. It’s a nice little co-author to use your words. It’s like sitting there right with you and it’s not annoyed that you’re making it wait.

Ben Schorr:

Definitely.

Zack Glaser:

And that you’re like-

Ben Schorr:

The other way I’ve seen it used, I’ve seen attorneys use it is they’ve written something and I’ve never written anything where I loved every paragraph I wrote. There’s always some part of it where I’m like, “Yeah, I like this article, but I’m just not feeling it in this part for some reason.” And so you can kind of see the same thing with attorneys. They’ve drafted something and they like all of this part, but man, this part, I don’t know about this part. I’ve seen attorneys do is they’ll ask Copilot, “Hey, suggest ways to make this section more persuasive, suggest ways to make it clearer, suggest ways to make it more concise.” Whatever it is that their adjective is. And then Copilot will go through and it’ll give them some suggestions of, here’s how you could rewrite this to be more whatever you asked for, persuasive, concise, professional, whatever.

And so that’s another way that Copilot can help you in the editing process on an existing document. This could be a document you’ve used many times and you’re just looking to improve it. I had an attorney one time who said, “I don’t want AI to do the writing for me because I love the writing. That’s part of the job I like. “

And I said, “Okay, well, here’s a way that Copilot could still have value for you. ” You write the document, then you ask Copilot to write its version of the document and then compare the two. Are you going to like Copilot’s version better than yours? Probably not. But you might be looking at Copilot’s version and go, “Oh, I like what it did here in Section two. That’s kind of nice. I like that. Let me pull that over.” And, “Oh, I didn’t even think about that thing on page five. Let me grab that too. That’s good. Okay.” So you might find little things, little bits of the Copilot version that you could steal to make your version better. That’s another way that Copilot can be useful in a writing scenario. Even if you want to do your own writing, that’s fine. Copilot can be a helpful co-author.

I call that parallel authoring where you and Copilot both write versions and then you steal the best bits of Copilot’s version to improve your own.

Zack Glaser:

I like that. I like that. Okay. So we got create with caveats and then we’ve got edit as well. What else can we do?

Ben Schorr:

So another one that’s really big for attorneys these days is ask and summarize. So somebody has just sent you a document, you can ask it to summarize the document for you. And that doesn’t necessarily absolve you from reading the document. You still might have to read it, but at least by getting a summary upfront, you kind of get primed as to, “Okay, what is it that I’m going to see here?”

But the other part of that that you can do is you can ask questions about the documents. So where I’ve seen attorneys use this pretty often is they’ve got some sort of long agreement document. And so what they can do is they can say to Copilot, “Create a table of all the dates and deadlines mentioned in this document.” And then it’ll very quickly create a very quick, easy template table you can read that shows, “Okay, this thing that’s going to happen, here’s the date. This thing’s going to happen. He’s the date.” It gives you that nice handy table you can reference. Here’s the dates and deadlines that are mentioned. It can be amounts, it can be jurisdiction. Any fact that’s mentioned in the document, you can ask it, create a table or tell me about this. What jurisdiction is specified? Is there an arbitration clause?

I mean, you can ask any kind of questions like that and Copilo will go through the document and very quickly pull that out and importantly give you a link to where in the document that is so you can verify it for yourself because we trust, but verify, don’t assume the AI is the absolute unwavering truth.

So that’s another great use of AI in the ask and summarize thing. And not only with documents, but I see that done with Teams chats and Teams meetings

Frequently. If the meeting’s been recorded or transcribed, you can use Copilot to ask questions about the meeting, same kind of questions, like what dates and deadlines were there. Another one that I love at the end of a meeting or as a meeting is wrapping up is to ask Copilot what questions were left unanswered because then Copilot will say, “Oh, okay. Well, at minute 30, Alice said something about this and it was never really resolved.” So that can flag for me things that we should close the loop on before we end this meeting because they’re still hanging out there or after the meeting, oh, we still need to finish figuring that out. So whether it’s an email or another meeting, I don’t know, but it can basically help you get a little bit more out of the meeting because it helps flag those questions that didn’t quite get resolved.

Zack Glaser:

Well, and I imagine I can ground that kind of flagging in any document. So let’s say I create an intake playbook for my intake and I record the meeting that my associate has or that I have or whatever with the new client and I can say, “Did I miss anything?”

Ben Schorr:

Yeah, 100%.

Zack Glaser:

Hell, I could probably do that in real time. Have I missed anything?

Ben Schorr:

Yeah.

Zack Glaser:

Yeah,

Ben Schorr:

You

Zack Glaser:

Probably could. Against that playbook because again, even when we’re in Word, we’re not limited to just that document, right? We’re not in the four corners of that specific document. We could go query into anything that we have in SharePoint. So we could say, “I’m making a lease and I want to add this type of provision. Could you go find multiple different types of provisions that I’ve already written and show me the options that I have? “

Ben Schorr:

100% you could do that. You could also say, “Look at this lease I just created, compare it to these leases in this folder. What in this new lease have I overlooked or what doesn’t seem consistent to the way I usually do it? ” Or I mean, you could basically ask Copilot to compare that lease to other leases you have as reference material or if you have a playbook, you could say, compare this, looking at our playbook, look at this document, what have we overlooked?

Zack Glaser:

So now data, the data that I have in my SharePoint area not only becomes valuable, but the curation of that data becomes valuable, knowing where I want to point Copilot at in order to have it do something. I like that.

Ben Schorr:

And by the way, you can also, this kind of circles back to the create and edit mode. You can tell it when you’re creating a new document, whatever, and it’ll try to match the structure and layout and content of that previous document because it already knows you’ve got a contract or whatever it is that you really like the way it’s structured. You can ask Copilot, use that as a model when you create your new one so that it’ll try to adopt the same structure. Now, is it a perfect replica? No, but it’ll often do a pretty good job of trying to match the way you had that previous document structured in the new document. So yeah, you can absolutely have Copilot look at playbooks, at previous examples of documents and try to adopt the same voice and tone or the same structure, same kind of language and verbiage.

Yeah, absolutely.

Zack Glaser:

Okay. Well, are there other things that you generally tell people that Copilot can be used for today?

Ben Schorr:

Yeah. So another one is brainstorming. Brainstorming is another really handy thing you can do. You can ask it for suggestions of things. So for example, I’ve seen it used, you’re writing a blog post for your blog and you’ve written the post, but you’re maybe stumped on a good title. You could tell Copilot, “Look at this draft of a blog post and suggest 10 possible titles for this. ” And then you can give it a little bit more guidance. Yeah. Yeah. So you could say, “Make them catchy and fun, but not silly.” Things like that, or make them very professional or very concise or only three words long or whatever it is you want to give it, but you can give it some guidance that sort of points it in a certain direction. And then it’ll come back with 10 suggestions and you’re not going to love all 10 suggestions, but hopefully there’s a couple in there that you think, “Oh, I kind of like that one.” And then you can also riff on that.

So I’ve occasionally done something like that where it came back with 10 suggestions and seven of them were trash. I didn’t want those, but the other three, okay, that’s kind of nice. And then I would say, “Hey, looking at numbers four, six, and eight that you suggested there, let’s try a few more variations on that. And lean into the seafaring theme or whatever.” Right? And then Copilot will give you another set of suggestions based on that. You can go back and forth and have a conversation with Copilot. And then it’ll get you hopefully to a title you like or at least to an idea that’s useful for you. So that kind of brainstorming’s helpful. The other way we see lawyers using brainstorming quite a bit these days increasingly is with personas. And so let’s say you’re creating a lease agreement, you’re representing the owner. You could say, act as an experienced commercial lessee.

Looking at this lease agreement, what possible concerns or objections might you have? And then it’ll come back and it’ll give you a list of what concerns it thinks an experienced commercial lessee might have with the document you’re preparing to send over to them.

Now, in most cases, what that does is it basically prepares you for what might come back.

Zack Glaser:

Yeah.

Ben Schorr:

It lets you pre-think, okay, they might not like the deposit or they may not like the whatever part in here. We just need to be prepared to address that. Or you could be proactive about it because you might look at it and go, “Oh, actually that is a reasonable objection. Let’s address that before we send it over.” But it gives you that perspective. Now, it’s important to remember the map is not the territory. The AI is not really an experienced lessee. It’s Just an AI simulation of an experienced lessee. And so when you ask it questions, it’s helpful, but you also shouldn’t think it’s gospel.

Zack Glaser:

Well, I like that kind of flip on it on where it’s acting like an attorney. I don’t want it to act like me when I’m being the attorney. I don’t want it to actively say what something should be ultimately. But being the other side, I like the idea of it acting as an attorney on the other side, because it doesn’t have to be right, because I’m going to nitpick it. Me as the attorney is going to nitpick its argument. And so great. I’m comfortable with it with having it saying be a quote unquote, I’m doing air quotes here, be a quote unquote experienced lessee attorney. Yeah, I like that.

Ben Schorr:

I was working with an attorney who used it for almost exactly that. He was preparing some litigation and he basically act as the attorney for the other side and he gave it some details about who the other side was.

“What would be your counter to this theory? This is my theory of the case. What would be your counter arguments?” And it came back and it gave him eight or nine fairly detailed counter arguments to what his position was. And he looked at that and he went, “Yeah, no, no, that’s not right. Oh yeah, that one maybe this one, no.” He went down the list and he eliminated about half of them. And there were a couple that he’d already thought of, but there was one or two that he was like, “Okay, that’s interesting. It’s good to know.

And it took seconds, right? In the matter of seconds, he got that perspective of what the other side might throw back at him. And he found that to be pretty helpful just as a priming exercise.

Zack Glaser:

I think that’s a really good point, what you brought up there is the ROI on some of this. The investment is so low on a lot of this stuff. Now, you don’t want to get into the weeds. You don’t want to just create AI slop that’s going to make you go through and read a ton of stuff. But if we’re talking about just like, give me the high level, what are the arguments here? Check my work. Let’s get a sniff test. Let’s have something else do a sniff test on this thing. Yeah, I like that.

Ben Schorr:

And then I guess the fourth area, other than brainstorming that I see attorneys using it for, there’s a lot of sort of business of law things that they’re doing, things like triage my inbox. I see attorneys a lot of times, they come in on Monday and they’ll say, “Look at my inbox. Which five emails seem to need my attention most urgently?”

Because they’ve got 200 messages that came in since Friday. And so they want to get that, just pull out the highlights. Which ones seem to need my attention right now? And so they can start there. They know where to start. So I see that kind of thing getting used. Looking at my calendar, which meetings stand out as needing the most prep time, things like that. And even with a specific meeting, I often use it for this myself is I’ll say, “Help me prepare for my Wednesday meeting with Alice.” And it’ll look at the meeting invite. It’ll look at any materials that it can find that seem related and it can help you prepare basically for that meeting. So there’s a lot of that sort of day-to-day productivity stuff that Copilot can help with also.

Zack Glaser:

A lot of that stuff, and I love all those things because I really like the structure that you put to that, the create, the edit, the brainstorm, the business of Law stuff. But a lot of that is kind of like that LLM sort of like I’m asking a question, it’s giving me an answer, I’m getting it to brainstorm. Copilot also has the ability to do agentic stuff. Can you talk to me about that a little bit?

Ben Schorr:

Use the A word.

Zack Glaser:

Yes.

Ben Schorr:

Yeah. Agentic is a word that gets tossed around a lot. A lot of the things that we see referred to as Agentic, really that Agentic. Just to sort of clarify, when I think of Agentic AI, I’m thinking of AI that does things either autonomously or semi-autonomously. It’s not just you type in a prompt and then it gives you a response. It’s the AI is actively … So maybe you’ve got an AI agent, for example, that’s looking for email that comes in from prospective clients

And then takes some action. Or it’s looking for a new document that gets created in a particular folder and then it takes some series of actions based on that intelligently. And so those would be examples of agentic AIs. You could also trigger an agentic AI. So for example, you need to book a trip to go to Chicago for a meeting. Theoretically, you could have an agentic AI that you could say, “Book my trip to Chicago based on my calendar, whatever.” And it’ll look at your calendar. You’re supposed to go April 9th, and so it’ll go ahead and book the travel, it’ll book the hotel, et cetera.That’s the promise of Agentic AI. I don’t see a lot of Agentic agents that are actually doing that yet. And I think there’s a couple of things that we need to consider with Agentic AI. One is how autonomous do you want them to be,

Especially in law. We just talked about never show the output of an AI to a client or the court without reviewing it first. You probably wouldn’t want an AI to draft a document and send it to your client without you looking at that document first because who knows what crazy stuff could have ended up in that document. And so you’d want to catch that. So there needs to be some human checkpoints along the way in the Agentic process. A big objection I hear or concern I hear with Agentic AI, especially anything that has to do with money. The book My Trip to Chicago sounds like a good idea, but if I’m giving it my credit card and now it’s on American Airlines website and next thing I know, I’ve got 43 trips to Chicago booked because it didn’t recognize that I didn’t want to fly there and back over and over again the same day.

Zack Glaser:

I got all the good ones. Or I’m going to Manhattan, Kansas.

Ben Schorr:

Yeah, that’s right.

Zack Glaser:

I wind up in Manhattan, Kansas. And yeah, that’s not where I wanted to go. Well then what? I know the Agentic stuff is a little bit newer to people. And I kind of think of the Agentic stuff as like Power Automate on steroids. Power Automate with a little bit of thought. But I also, I don’t think people even use Power Automate very well or can envision what they want Power Automate to do. What would you use Agentic for in a law office, do you think?

Ben Schorr:

Yeah, I could see, there’s a few things I could see it being used for. If you can identify any workflow really where the set of steps are fairly repetitive, they might need a little bit of intelligence to discern A from B, but for the most part, they’re pretty repetitive. Now in the past, you’re right, we would’ve used something about Power Automate or Zapier or something like that and create a script, but then you have to delineate every step, right? If this word is in there, then it’s A. If this word’s in there, then it’s B. If this word’s in there, then it’s C. And you had to create these sort of elaborate, carefully scripted workflows. And even then, some outliers showed up where they didn’t use Word A, but they had an A scenario

Zack Glaser:

And it

Ben Schorr:

Didn’t work.

Zack Glaser:

There’s always a rip cord that has to be pulled. There’s

Ben Schorr:

Always that. Yeah. And so with the Agentic AI, in theory, the AI is a little bit more flexible, a little bit more intelligent to be able to look at the email or the whatever it is, whatever the trigger is and correctly identify this is an A, this is a B, this is a C, take the right actions in theory. And so I think anything you can identify a workflow, like for example, a prospective client has emailed your firm about a new wills and trust case, for example, right? You could have an agentic AI that takes that email that then launches off the start of an intake process maybe. Maybe it does a cursory conflict check if it needs to. That shouldn’t be the final. I mean, you still want to double check that because that’s an important step, but at least could identify their name does not appear in our practice management system currently, so that’s a good starting point.

And so it then could kick off, well, here’s an initial questionnaire that goes back to this prospective client, for example. Gotcha. So some of those early steps are something that an Agentic AI might be able to do a good job with. And especially because intaking a wills and trust case could be very different from intaking an immigration case, from intaking business litigation, from intaking different practice areas. And so if your firm handles multiple practice areas, an Agentic AI might be pretty good at identifying a prospective client, what area of law they’re probably talking about and then initiating at least the first steps of an intake process or an evaluation process.

Zack Glaser:

Okay. Yeah. And as you’re talking, I think there’s a place that you could put a human in the loop there where it’s like, okay, I see that this person came in, I’m going to use a certain file structure to potentially create this client file folder area, and then I will draft an email response and put it in your inbox, but you got to send it, but I will have drafted it for your review. And now me as the attorney or as the person that does intake, I don’t have to go through some of those drudgery steps. Yeah,

Ben Schorr:

100%. At Microsoft, they had created a thing, their Clo, which is their internal attorneys, corporate counsel. They would get these requests via email all the time, which were something to do with intellectual property. I don’t know the details, but they were very pretty routine. And they actually created the workflow that would, when one of those emails would come in, Copilot would take that, would draft the Word document with the requested release, I guess it was, all filled out with everything in it. That would then go to one of the Microsoft attorneys who would review it very quickly and either approve it or decline it. And if they approved it, then it went back to the requester job done. In the past, when it was being done manually, when that was all having to get routed to humans and humans were having to draft the documents and et cetera, that process could take three or four days from start to finish.

Once they got Copilot involved, that process took two or three hours. And the biggest bottleneck was the last step where it went to the human attorney who then had to just look it over, make sure everything was okay and send it out. Sometimes that attorney was at lunch, sometimes that attorney was in a meeting, sometimes it was … So really in a perfect situation where that attorney just happens to be sitting at their desk just checking their email and not doing anything else important, that request, that fully finished document or fully finished, I’ll put in air quotes, that finished document comes in for review, theoretically it could cut that time down to 10 minutes because the document got created automatically in seconds or minutes and then the attorney could review it. It looks good to me, approve, send. But that was a good example of a somewhat agentic workflow that was able to identify the request that came in, create the document automatically.

But then you had the human in the loop because it got routed to a human attorney who could then look at it and go, “Yeah, that looks good.” Or, “No, no, no, no, no, we’re not doing that.

Zack Glaser:

” I like that one. I like that one. And the mind reels with ideas, I think, but I also think that a lot of people don’t have the ideas of what I could do. And I think some of this goes back to processes. Some of this goes back to I don’t know what I don’t know sort of thing. So as we wrap up here, I kind of want to do a call to action to the people listening here. I would love for people to, in the comment sections on our LinkedIn pages, wherever you run across us, if you’ve got a cool way that you use Copilot in your office, we’d love to hear about it. We’d love to hear about it. Love to hear people share about it. And Ben’s got a bunch of them, a bunch of ideas and a bunch of people that he’s run across, but we’re just scratching the surface on a lot of this stuff too.

And it’s really about structure. It’s about thinking differently. But before we go, Ben, I want to not necessarily put you on the spot, but we’ve been talking about how people can use Copilot. I want to quickly ask, how are you seeing people … What is the way to not use Copilot, I guess? What is something that people think that we can do with this stuff? And it’s either not quite there or it’s just Moo, you’re asking the wrong question, you’re thinking about the wrong thing.

Ben Schorr:

I guess the obvious one, I guess, is don’t tell it to draft a brief and then just file that with the court. Yes.

Zack Glaser:

Yeah. That would be it.

Ben Schorr:

Yeah. So a couple things. So first of all, one of the great things about AI is that AI is, you don’t have to be monogamous with your AI. I think I’ve used that line before with you, But AI doesn’t care if you use other AIs and it’s good to use the right tool for the job. Copilot is not tuned for legal specific work. And so for example, could you use Copilot for legal research? Yes. Should you? Probably not. Because Copilot doesn’t have access to Lexis. It doesn’t have access to Westlaw. It doesn’t have access to VLex. It doesn’t have access to all those proprietary databases with the really high value legal research and legal content in them. Copilot doesn’t have access to that stuff. It’s got access to your content in your Microsoft 365, and it’s got access to things that are in the public web and things that are in its training set, which are not tuned for legal tasks. So can you do legal stuff with it? Yeah, but you could probably, if you’re doing legal research, that’s a good example of where you probably should use a tool other than Copilot.

Something like VLex or LexisPlus or Harvey Paxton, whatever. A tool that’s tuned for that that has access to that kind of research. So that’s one example of a bad place to use Copilot.

In terms of tasks that Copilot still doesn’t do very well, one thing that’s a big glaring miss for me right now is scheduling meetings. So I can ask Copilot, “Hey, schedule a meeting with Zack for next Thursday.” And Copilot will get me part of the way there, but it doesn’t get me very far. It basically will end up creating a blank meeting invite for the right day and time, but it doesn’t put Zack in there. It doesn’t put the topic in there. I’ve gone to all the trouble to say, “Schedule a meeting with Zack so we can talk about A, B and C next week.” It’ll find a time next week when you and I could meet, but then it still makes me do a whole bunch of other stuff. So that’s an example of where Copilot still isn’t quite what it should be, is on meeting scheduling.

It can do other things with your calendar that are useful because that’s not one of Them. I’m trying to think of a good other examples there that … Oh, so Copilot, this is when it trips people up sometimes. We’re working with email, for example.

If you ask Copilot a question about something that’s in your email, it’s pretty good about being able to find it. I used it just earlier today, I had a specific question about an upcoming conference I’m speaking at. And I said, “Looking at my email, what’s this answer?” And it went through all the email I’d received about that conference. In fact, it went through all my email and found the emails related to that conference and then it was able to answer my question about the conference based on the email. If I had asked it, give me a list of all of the emails I have received about that conference, that probably wouldn’t work very well. Copilot’s not great at indexing and giving … Oh, here’s a full list of all the emails about that conference. If I’m asking a specific question about the conference, that it can probably do, and it does pretty well, but it’s not really intended as a search engine per se, and that it’s not going to give you a comprehensive list of things generally.

Zack Glaser:

I like that one. And honestly, I like that one as one to end on because what I think happens a lot of times when people go and use Copilot, and I want this to be a encouraging, go use Copilot, provided you’re using it in the right place and for the right things, people will use it for something that they usually would use Google for or a search engine or a search bar or something like that. And then when it doesn’t do it well, they go, “Well, this is a stupid thing and it’s never going to work for anything.” So I like that as an example, as a really specific example of it doesn’t think like that. That’s just not the way that you use it, but all the other things that we’ve outlined here today, those are great uses of Copilot.

Ben Schorr:

I’ve seen that one trip people up too, because I had somebody who said, “I wanted a list of all the emails that I’d received from Bob in 2015 or 2025. And So I asked it to give me that list and it gave me a list, but I could see there were a bunch that weren’t there.” It’s like, yeah, it’s not great at that comprehensive, here’s your search results. There are other tools that can do search results, CoFilet’s not really a search results tool.

Zack Glaser:

Yeah. And I’ve had that frustration myself and I’ve cursed its name many times. And so yeah, I think that’s a really good one to say. It’s not its jam. It’s good at a lot of things and that’s not its thing. Well, Ben, I really appreciate you taking the time to talk with me about this. And we have talked before about some of the fundamentals related to security of Copilot and what Copilot can see and what it’s grounded in and things like that. On a previous episode, I don’t have the number in my brain right now, but we will put it into the show notes. So if you want to hear more from Ben, you can catch him on that. And you can also catch him on our sister channel for Affinity Thought Hub on YouTube. We’re doing a AI calibration weekly video that Ben is a very, very, as you can imagine, regular guest on.

So Ben, thank you again for talking to me about AI and specifically Copilot.

Ben Schorr:

Yeah, my pleasure. Always enjoy talking with you, Zack.

Continue Reading