Learning AI by Doing, Not Watching, with Ben Schorr

Episode Notes

Transcript

Bernadette Harris [00:00:00]:
Hi, I’m Bernadette.

Zack Glaser [00:00:02]:
And I’m Zack, and welcome to The Lawyerist Podcast, one of the many ways that we help lawyers build healthier firms, better businesses, and, you know, more sustainable lives. Today, I talk with Ben Schorr about building your team’s AI confidence through promptathons. And honestly, like, I love this one. I had a really great time talking to Ben about this.

Bernadette Harris [00:00:25]:
That sounds fun. I’m, I’m really, really loving prompting and AI and all of this stuff. So, yeah, I can’t wait to listen to it.

Zack Glaser [00:00:32]:
So, one of the things that, that is with this is the, the point of the Promptathon is to build confidence in your, in your team. How did you build confidence in using ChatGPT and Claude and things like that?

Bernadette Harris [00:00:48]:
I think I really started to build confidence when I started building dashboards. I remember the very first dashboard that I built for a labster. And I didn’t know, like, I had no idea what the, uh, what the output was going to be, but I thought, yeah, I could probably do this in, um, ChatGPT.

Zack Glaser [00:01:10]:
Uh-huh.

Bernadette Harris [00:01:11]:
And, and I just told it like, hey, this is what I wanna track. This is what I want the outputs to look like. And it built this amazing dashboard that would’ve taken me 2 and a half days to build on my own. And I think that is when I really— I kind of popped my collar on that one.

Ben Schorr [00:01:29]:
Like, yeah, yeah, I was really feeling it. Yeah.

Bernadette Harris [00:01:33]:
And I’ve since built several dashboards though.

Zack Glaser [00:01:37]:
I, I have the same kind of, of story on that, on like how I built my confidence with it. Because I, you know, you think of confidence and it’s like, oh, is it the confidence to do? And sometimes it’s the confidence of like, this means something, you know, like I can do it Well, it’s not just, I can do it. And so for me, it’s the same type of thing. It’s, it’s being able to take data and, and put it into, into certain, um, ways of viewing it. But I started building like little websites that were dashboards essentially, you know, but it was like, how can I take this data and make it to where we can view it? easier or consume it more easily, I built a dashboard for our da-da-da-da podcast.

Bernadette Harris [00:02:27]:
Nice.

Zack Glaser [00:02:28]:
And took all of our— I was able to take our RSS feed. I was able to take our YouTube metrics. I was able to load all of this information into this dashboard and then use the Anthropic API to then go out and like, consume this stuff and, and like categorize these things. It’s difficult to categorize these, these podcasts sometimes, but to, to have it categorize the podcast. And so, yeah, it was like creating a thing where I’m like, that would have taken me— well, I wouldn’t have done it.

Bernadette Harris [00:03:00]:
Yeah, that’s it. You know, like, I honestly, I think that even the dashboard that I built for this Labster, I think that if I had to build that I, I probably wouldn’t have done— I probably would have looked for some software or some add-on to some existing software, um, before I actually would have even thought to build a dashboard to do these things. Because I know Excel, but I don’t know Excel, you know what I mean?

Zack Glaser [00:03:29]:
Yeah, yeah, yeah, yeah, yeah. I’m a power user, but I’m not that kind of power. Yeah, yeah, yeah, yeah. I love that. Okay, well, if you want to get more confidence with, with, uh, your prompting and your AI, follow my next guest here, Ben Shore, on LinkedIn and really anywhere that you can find him. He has a lot of great things to say. So now here is my conversation with Ben Shore.

Ben Schorr [00:04:11]:
Hi, I’m Ben Shore. I’m an innovation strategist at Affinity Consulting Group, and I help people be more successful with Microsoft 365 and AI.

Zack Glaser [00:04:19]:
Man, I like how tight that is, Ben. It’s always tough to do intros, especially for yourself. I like that. I help people be successful with Microsoft and especially AI.

Ben Schorr [00:04:31]:
Well, I have put more than a few minutes of thought into that.

Zack Glaser [00:04:36]:
I, you know, it, I hesitate to say that it shows, but it shows. Like, again, I, you know, I found it necessary to comment on it even.

Ben Schorr [00:04:44]:
For years, I was somebody who poo-pooed the idea of mission statements. I thought they were corny and just corporate, you know, gamesmanship or whatever.

Zack Glaser [00:04:52]:
Yeah.

Ben Schorr [00:04:52]:
And I have to say, in the last few years, I’ve sort of gained an appreciation of it as a North Star for, for what you do every day. Because sometimes you wake up in the morning and the caffeine hasn’t kicked in yet and you have to remind yourself what you do. And having that mission statement Sometimes that really helps to remind yourself, this is what I’m about. This is what we’re doing.

Zack Glaser [00:05:11]:
I like that. And there’s an element of affirmation in there too. I do daily affirmation stuff in the morning of like, this is where I provide value and things like that, just to kind of pump myself up a little bit. But the proper mission statement like that, and that kind of has an element of affirmation too. I love that. Well, let’s talk about one of the things that fits into your your mission statement. This last weekend, this last week, you did a prompt-a-thon with some people, and I thought that would be an interesting thing to talk about on the podcast, and that a lot of people would get a lot of value out of that. But let me just start with, what is a prompt-a-thon?

Ben Schorr [00:05:56]:
So a prompt-a-thon is an event where we get people together to work in small groups. Really, they’re working individually, but they’re collaborating, if that makes sense.

Zack Glaser [00:06:06]:
Okay. Yeah.

Ben Schorr [00:06:08]:
And the idea is that we want them to experiment with using AI tools, and it can be whatever AI tool they have or multiple tools if they want to do a scenario. And we want them to do it in a low-risk environment, you know, hopefully, you know, around the table with their friends so that it’s enjoyable. And it’s— the goal is for them to gain confidence and enthusiasm, we hope, for using AI to solve real-world problems.

Zack Glaser [00:06:37]:
So, so in this scenario, people are, you know, I’m envisioning like a conference room and 8 tables of like 5 people, and they’ve all got their computers out, and you and maybe another facilitator are kind of moving forward and saying, hey, let’s, let’s, I mean, learn about and work on some real-world prompts. And I hope that nobody in, in lawyers podcast land is thinking, what is a prompt right now? But if they are, tell the, tell the people what they want. What’s a, what’s a prompt?

Ben Schorr [00:07:10]:
Yes. A prompt is how you tell the AI what you want it to do. Okay. All right. And it can just be in your own language. Often accomplishing a task takes a series of prompts. You’re rarely going to get from 0 to 60 in one prompt.

Zack Glaser [00:07:22]:
Yeah.

Ben Schorr [00:07:23]:
And so usually there’s a prompt and then there’s follow-up prompts and then there’s additional things to do to accomplish whatever you need to do.

Zack Glaser [00:07:29]:
Okay.

Ben Schorr [00:07:30]:
And so yeah, your description of the promptathon’s pretty accurate. Yeah, we get a group of people hopefully in the same room and around tables, and then we give those tables suggested scenarios to work on. So for example, one of them was your inbox is overflowing and you want to get control of it. How can AI, how do you use AI to get control of that? Another scenario we gave was your managing partner has just asked you for a status update on a client. He has a meeting with that client in 45 minutes. How are you using AI to try to generate, to help generate a status report for the managing partner on what’s going on with that client? That would be another one. We had half a dozen different suggested scenarios that we gave the participants. And part of the goal of these suggested scenarios is We don’t want to waste time with them having to try to figure out a scenario.

Ben Schorr [00:08:19]:
So we gave them a head start there. And also the suggested scenarios are generally pretty low-risk things that they can do. You know, the goal of the promptathon is not to have them doing high-value, high-risk things in this environment. It’s for them to play in our safe scenarios, right?

Zack Glaser [00:08:38]:
Delete all my emails that you don’t think are necessary, in air quotes.

Ben Schorr [00:08:43]:
Completely redesign my billing system here while I’m sitting in the account.

Bernadette Harris [00:08:46]:
That’s not, we don’t want to go Not yet.

Zack Glaser [00:08:49]:
Not yet. Not yet.

Ben Schorr [00:08:51]:
No.

Zack Glaser [00:08:51]:
So yeah, I kind of envision this. I’ve been to more than a few hackathons, as I know you’ve been to more than a few hackathons. It’s kind of— you’re jumping over one of the initial problems that we have at those, which is like, what are we going to do? You know, because we don’t have an infinite amount of time. And these, these problems, I mean, those sound like issues that either specifically somebody has had, or, I mean, they’re a pretty good facsimile of other issues that they’ve had. So you talked about a primary goal. I think that’s the question I have is like, what’s the point of a prompt-a-thon? Like, what are we solving here by having everybody in the same room and a Ben Shore, you know, walking around?

Ben Schorr [00:09:36]:
Yeah, so training is usually the first step people are doing with AI, right? And training is really important. Don’t get me wrong, training’s important. But what we find is that people leave the training and they’re like, okay, that was cool. It can do some interesting things. And then they go back to their desk and they don’t really know how to do them themselves because probably they were watching somebody else do it on a screen. Hopefully they were watching and not just checking their email, right? So they were watching somebody else do it on the screen, but it’s one thing to watch somebody do it on a screen and then the following day to sit there at your own screen and do a thing, right? And Miro can do that.

Zack Glaser [00:10:08]:
Yeah.

Ben Schorr [00:10:08]:
And so, and we’ve heard from more than one client that their users, liked the training, but they lacked confidence when they got back to their own devices. They lacked the confidence to actually put it into action. And so, we wanted to develop a workshop kind of experience where the users, the people are doing it themselves, they’re not watching somebody else do it, and they have the room to play and to experiment and to try some things with a coach present in the room, basically. And so that they can ask questions again and block in. Also, again, sitting across from their colleagues and friends who they can share ideas with in real time. Like they tried something, it didn’t work or it did work and they can talk about it right across the table.

Bernadette Harris [00:10:53]:
Yeah.

Ben Schorr [00:10:54]:
Which hopefully builds that confidence and that energy to do the thing. Right. And then hopefully that translates. The goal is that the following day when they get back to their own desk or that afternoon or whenever it is, when they get back to their own desk, they’re feeling like, okay, yes, you know, I just spent an hour doing stuff, prompting, working on this scenario. Let me apply that to some of my own real scenarios, my own real cases.

Zack Glaser [00:11:16]:
At the end of the day, you know, I run into a lot of, you don’t know what you don’t know when people enter into this AI space, especially when we enter into like, how do I use Copilot inside of Microsoft 365? And so not only does this get into like, Well, so let me back up. You have to kick it around in order to know what you don’t know. You got to try some things. You’ve got to, you’ve got to do it wrong, you know? And in a law office, it’s not really helpful most of the time to do something wrong. Um, there are big consequences. And so having, having the ability, I, I find having the ability to kind of like go break something. without breaking it is really, really helpful in gaining confidence. But the thing that I thought about while you were talking about this is the communal aspect of it, is that people now, now I go back to my desk and I and Jamie have worked on this together.

Zack Glaser [00:12:20]:
And now I can send a Teams message that says, hey, here’s this skill I just did. Here’s this prompt I just came up with. help me like you did the other day with brainstorming this prompt. So it’s more than just individual confidence, even it’s collective confidence.

Ben Schorr [00:12:37]:
100%. In fact, one of the things I think we found that, that is a key factor in successful AI implementation in firms is having that community of usage, is, is having people, having a place where people, a safe place where people can talk about what they’re working on, who can They can banter around ideas. They can ask questions. They can answer questions. They can think about it together, you know, having that safe space where they can do that. And in most firms, that takes the form of a team, a Microsoft Teams team, right?

Bernadette Harris [00:13:05]:
Yeah.

Ben Schorr [00:13:06]:
That’s usually what that takes. And now in a physical firm, that could take the place that could, could even be a, you know, a regular meeting. I, well, I guess it would be a regular meeting on Teams too. Uh, I mean, it could be a meeting, but more often it’s asynchronously in some sort of team.

Bernadette Harris [00:13:19]:
Yeah.

Ben Schorr [00:13:19]:
And so having that, having that community that, that, that develops around AI usage, um, or any kind of tool usage, honestly, is a big, big factor in successful implementation. Um, because then they see their peers using it. They can talk to their peers about it. They can share ideas. Um, and it’s not just some random consultant who’s 1,000 miles away in Seattle. It’s, you know, it’s the guy who’s, who’s, or the, or the young lady who’s, who’s right down the hall. who does the same job, works at the same firm, who’s pulling on the same rope, right? And yeah, you know, we’re all, we’re all working together and, you know, now we can collaborate on ideas of how to use these tools more effectively.

Zack Glaser [00:13:57]:
I like that. I like that a lot. Well, okay. So, um, just so I know that people have the proper kind of shape of this, walk me through, I guess, one of the most, you know, you said you had a couple of different prompts. Walk me through one of the prompts or what happened in this actual promptathon.

Ben Schorr [00:14:13]:
So as I said, we’ve got the groups, you know, divided by tables. We’ve randomly assigned them each a couple of scenarios. Um, and so generally speaking, you know, they’re looking at what their scenario is. So let’s say their scenario is get my inbox under control. All right. So maybe that’s their, that’s the scenario they got. So usually at this point, they’re opening whatever their tool is for most of them in this group, the group we did last week, it was Copilot.

Zack Glaser [00:14:35]:
Okay.

Ben Schorr [00:14:36]:
Certainly you could do this with Claude. You could do this with ChatGPT. It just has to be connected to your inbox for this particular scenario.

Zack Glaser [00:14:42]:
Yeah.

Ben Schorr [00:14:42]:
You know, the tool is not as important as having that connection. So then what they would do is they were opening the tool and they were starting off with something really simple, like, you know, what are the most important messages in my inbox?

Zack Glaser [00:14:54]:
Right.

Ben Schorr [00:14:55]:
And then they’d look at the result they got from that. And we gave them some ideas around what good might look like, but we also encouraged them to think about what good might look like. Because good looks different for everybody in scenarios like that, right? Maybe what you want is a table of the 3 or 4 most important emails with suggested next steps. Maybe what you want is a longer report that categorizes your inbox and sort of says, you’ve got 20 messages that are important. You’ve got 37 that aren’t that important. You’ve got 28 of them that are newsletters. You know, maybe you want something like that. Um, what different people want is different.

Zack Glaser [00:15:24]:
Yeah.

Ben Schorr [00:15:25]:
But for each of them, we encourage them to think about what good looks like. Right. And so they’d start with something simple, uh, like what are the most important messages in my inbox? And then they refined it. They said, oh, okay, well, that’s okay if it’s a little broad. Well, what if I added unread?

Zack Glaser [00:15:37]:
What are the most unread messages in my inbox?

Ben Schorr [00:15:37]:
Because that’s kind of where I’m going. I want the triage, right? So, okay, good. Well, that was helpful, but, but now it brought back some stuff that’s kind of old. Like maybe I want to put in some time scope. Um, the other thing they found is, uh, it’s the message that brought back. They’re important, but they’re not that important to me. Like Copilot was, was, didn’t, I didn’t tell Copilot what important meant to me. So then they went, oh, okay, let me refine that.

Ben Schorr [00:16:00]:
Let me add, let me add in some instruction about what I think important is when it comes to email. And so they kind of iterated and iterated and iterated at the end of it, you know, end of it. After, after 15 or 20 minutes, they had a report that Copilot was generating for them on their inbox that looked genuinely useful to them, right? They, they had, yeah, it was identifying messages they thought were important in a, in a useful scope. It was giving them what they had asked it to give them. Uh, and so, uh, and, but just the exercise of going through that iteration of, okay, here’s the first try. Okay. That’s not bad. That’s not exactly what I wanted.

Ben Schorr [00:16:33]:
Let me, let me refine, refine that a little bit. And I did see a lot of conversation happening across the tables of people going, hey, you know, the same kind of thing. Like, it seems like it’s bringing back too many things. And somebody said, well, try asking it just for the unread ones.

Bernadette Harris [00:16:46]:
Oh yeah. Okay.

Ben Schorr [00:16:47]:
You know, so just having that, that conversation across the table as the facilitator, I walked around and was there to coach and answer questions, but I was pleasantly surprised to see that they were actually facilitating each other as much as anything. Right?

Zack Glaser [00:17:02]:
Yeah.

Ben Schorr [00:17:02]:
Kind of working through it together, which was great. I mean, I was happy to answer the questions and there were a few questions for me, but as often as not, they were figuring it out as a group, which is great.

Zack Glaser [00:17:12]:
As you were going through this description though, and what they’re doing, I often go back to blank elements of a good prompt. And I think various people have different things and it’s context and guardrails and things like that. And as you’re going through that, I’m like, oh, You’re just kind of slyly hitting all those things. You’re working people through the elements of a good prompt as they learn them. Because it’s one thing to have a list there that says, when I do a prompt, I’m going to do, you know, 1, 2, 3, 4, 5 things, which is good. It really is beneficial to have that in your head, but to have that in like in your guts, you know, to have that like in your feeling of like what a prompt is, you know, you need to describe what it wants to do. I love what does good look like? The, you know, what is the output that we’re actually looking for, you know, and when it brings back too many things, okay, well, now I put some guardrails on it. Now I get a little bit more specific.

Zack Glaser [00:18:09]:
And so you were actually working people through the, you know, elements of what a good prompt is in here in real time. I love that.

Ben Schorr [00:18:18]:
Yeah. And then the other thing that came out of that too was, was interesting was for the email scenario specifically, but also I think one of the other scenarios, I can’t remember which one it was now. also came up there was they said, wow, this is really good. So I would just run this prompt every day if I wanted it. If I wanted this, if I wanted this triage every morning, I would just run this prompt every morning. I said, actually, you could schedule this prompt to automatically run every morning. And now they’re starting to think about more advanced features, like beyond just prompting right now. They’re starting to think I can schedule prompts.

Zack Glaser [00:18:43]:
Yeah.

Bernadette Harris [00:18:43]:
Oh my.

Ben Schorr [00:18:44]:
Oh wait.

Zack Glaser [00:18:45]:
Yeah.

Ben Schorr [00:18:45]:
Like this, one of the tasks was about policy review and were you like rewriting policies and internal policies at the firm? And, uh, so they started looking at ways to do that. And one of the things that came out of that was, could I, could I like pre-train it? Like you could do an agent. I could have an agent that would help me rewrite. Yes. So the, so it started with prompts, but then it kind of grew as they started to explore what the prompts could do. Then it sort of grew into thinking about what are some of the more advanced features I could leverage here? Could I do scheduled prompts or agents or skills? Should I be using Cowork? Maybe, right? Um, or Claude Cowork, either one.

Zack Glaser [00:19:20]:
Or GitHub Copilot.

Ben Schorr [00:19:23]:
Yeah.

Zack Glaser [00:19:23]:
Whatever you want to use.

Ben Schorr [00:19:24]:
Should I be using Markdown files? Should I be using PDFs? I mean, the questions, once they got past that initial, okay, I’ve typed in a prompt and it did a thing and that thing’s pretty good and I’ve done a few refinements, then they start to think bigger about like, what other tools could I leverage here to make this even better? And that was pretty cool to see and unexpected, by the way.

Zack Glaser [00:19:44]:
Yeah. Because that’s difficult to get to because like, that’s the thing is what we’re solving for is this confidence and this confidence is what builds and builds and builds. But But I want to go back to something you said right before this part of the conversation. And you said, the tool is not as important as the connection or the data or the information. And you almost, you like buzzed through that statement, but it’s a really important statement now that they see it.

Ben Schorr [00:20:10]:
Yeah.

Zack Glaser [00:20:10]:
On the other side of these prompts. And that’s why I don’t like to say, here’s the 5 elements. And by the way, the 5 elements that I actually use of a good prompt are role, task, source material, output format, and guardrails. Other people use different things, but like, that’s why I like to buzz through that because if you get that in your head, like the important thing is what data can I connect to? Can I get into my, can I connect my emails to my Teams messages to my project management system? Huh. And then what sort of prompt could I do with that? That’s mind-blowing.

Ben Schorr [00:20:44]:
It was cool to see the light bulbs go on and then to start thinking like, about things like that. Because first they ran it like one or two of them, not, not with Copilot necessarily, but, uh, one or two of them were using Claude, I think, and they had not connected their Outlook email to it yet.

Zack Glaser [00:20:58]:
Okay.

Ben Schorr [00:20:58]:
And so they went to try to do the prompt. They went, oh, it can’t see my inbox. Oh, but could it see my inbox? Like, you know, one of them was about to give up and the other one was like, well, maybe it could see our inbox. Like, can we, inboxes, can we turn that on? They went to investigate and they discovered, oh yeah, actually I can connect Claude to my Outlook. Now, whether you should or not is a There’s a firm AI policy.

Zack Glaser [00:21:17]:
Hey, that’s your, that’s your IT team.

Ben Schorr [00:21:19]:
Yeah, your IT team. Right. Right. And what subscription of Claude you have, right? If you have a good enterprise or team level subscription to Claude, yeah, you probably should. You could. And if that’s okay with your firm’s AI policy, then good. Good.

Bernadette Harris [00:21:30]:
Good.

Ben Schorr [00:21:31]:
Um, but, uh, but yeah, it started to prompt those thoughts and some of the other ones, the AI policy one, you know, I want to start rewriting. And at first it was just a hypothetical AI policy. Then they started to think, well, how could I apply? Well, wait, we actually are. We were working on our on our dress code right now or something? Or, you know, could I— could we actually use this? Like, where are our policies? Like, are they stored in SharePoint? Are they in NetDocuments? Are they in iManage? Like, where are they? Is our AI connected to that? Can I use it?

Bernadette Harris [00:21:55]:
How?

Ben Schorr [00:21:56]:
Um, it was great to see that excitement building and the thoughts going.

Zack Glaser [00:22:00]:
So that’s interesting because what, you know, we get to what data you connect to and what information you connect to and all that. And then the question is, Well, what data do I have? And now we start getting into my favorite documentation of how you do stuff. Because if you’ve got the doc— if you’ve got documentation, well, now your prompts are ridiculously powerful because you can prompt like, use the way that I do this in order to do this.

Ben Schorr [00:22:31]:
What is your procedure for this? Yes.

Bernadette Harris [00:22:34]:
Yeah.

Ben Schorr [00:22:35]:
Our colleague Adam likes to call that making the implicit explicit. Right. I love that. You sort of understand the policy right now. The procedure just lives in somebody’s head, right?

Zack Glaser [00:22:45]:
Yeah.

Ben Schorr [00:22:46]:
Some paralegal who’s been with the firm for 32 years and the procedure is just whatever Alice says the procedure is, because she’s the one who’s been doing it since, you know, 1994. Right.

Zack Glaser [00:22:57]:
Right.

Ben Schorr [00:22:57]:
And, you know, that’s just what it is. Um, and so somehow getting that procedure out of Alice’s head and into some coherent form that other people and tools could use becomes important, really important, especially for some of our folks who’ve been terrific, valuable key pieces of our firm, but they’ve been key pieces of our firm for a pretty long time and they would like to move on and do something else.

Zack Glaser [00:23:20]:
Right.

Ben Schorr [00:23:22]:
At some point, Alice is going to leave our firm and then all that knowledge goes with her. So how do we capture that? How do we make that explicit so that not just people, but tools could potentially leverage it to keep the firm working well.

Zack Glaser [00:23:34]:
Yeah. Well, let, let me, let me back up to the confidence thing again, because again, that’s the— I, I got a little bit off topic, but that’s the point of this promptathon, is people walking out of here with the confidence. And we’re kind of showing like what that confidence could do. But do you have examples inside of, of last week specifically of like being able to see people walk out of there with confidence or, or have that confidence while, while they’re at this promptathon?

Ben Schorr [00:23:59]:
You know, I haven’t heard a lot of direct feedback afterwards other than just positive, you know, positive messages. Like I haven’t heard specific examples from folks. I did get a couple of emails afterwards from folks saying that was great and learned a lot and excited to go ahead and use it. And, you know, I’ve already started playing with things, but they didn’t give me any specific examples there, but, but definitely you could tell, um, you know, I did hear a couple of people say that this was the first time that they had really sat down and attacked a problem with a tool like that, you know, and that just having their own hands on their own keyboard.

Bernadette Harris [00:24:29]:
Yeah.

Ben Schorr [00:24:29]:
typing in stuff in their own words and seeing that it did things, right? It’s doing stuff. Oh, wow. Okay. So I typed that in and it did a thing. Okay. Well, how do I fix that thing? How do I make that thing better? Right. And just, just to have the confidence that their own hands on their own keyboard, instead of just watching somebody on a screen do it, they, they actually were able to type in stuff and do it. But you could definitely see, I, you know, my sense was that, that people came out of it with some idea.

Ben Schorr [00:24:54]:
And actually, I shouldn’t say sense. I mean, there were definitely people who not only came out of it with ideas of how they might use some of these techniques the next day in their own, at their own desk, but there were a couple of examples of people who they finished a little early. We gave them 30 to 40 minutes to work through the scenarios. They finished the scenario in 10 minutes, 15 minutes. And then I thought, well, maybe they’ll go on and try and do a second scenario. What they actually did for a couple of them was, wait a minute, I have this thing I’ve been working on at work. Let me try that right now. Because they had their own devices in front of them, right? They were signed in as themselves on their own.

Zack Glaser [00:25:28]:
Yeah.

Ben Schorr [00:25:28]:
Yeah.

Zack Glaser [00:25:28]:
And so, well, that, I mean, yeah, that’s examples of that confidence in real time.

Ben Schorr [00:25:32]:
100%. I wasn’t expecting that it would manifest that quickly, right? Where they, I thought, you know, tomorrow they’re going to be back at their desk and they’re going to sit down and go, okay, let’s do some things. I’m going to buy some things. But instead they thought, I’ve got another 20 minutes before 15 minutes before this period, this ends. And the facilitator’s walking around helping us. Let me try something right now.

Zack Glaser [00:25:52]:
If I, if I have a question, I got, I got Ben right here.

Ben Schorr [00:25:55]:
You know, one lady, she has to generate a monthly report for her firm stakeholders based on new hires and people who’ve departed the firm. And it’s a monthly report that she emails out. It’s pretty standard stuff. It’s the source data is in Excel and the output was an email for a formatted email. And I, and we, I said, she said, could I, wow, maybe I could do that in Copilot. And she started trying it and I said, great. I gave her the tip of show it a couple of your previous ones that you’ve sent out so that it understands the format. She said, oh yeah, I could do that.

Ben Schorr [00:26:27]:
And she, she showed it that and where’s the source Excel? Oh yeah, there’s the source Excel. Do it. And so right there in real time in front of me, she did one and it came back and she was like, that’s 90% of what I, what, what it should do. I mean, right there. I mean, I’ve been in just sitting here in 2 minutes.

Zack Glaser [00:26:43]:
Right. The human needs to stay in the loop. She’s probably got to, got to tweak some stuff, but like, yeah. There’s a getting all that stuff together and mashed into a, into a mound of clay that has the same, the right basic shape.

Ben Schorr [00:26:54]:
Yeah. And she was super excited about it because she said it was, you know, an hour or more of pretty tedious, you know, copy paste work going on to make that report every month. And she said the fact that I was able to basically do it in seconds with this prompt. And once she said, and she, she caught onto the idea that once she kind of had figured out the prompt, she’d just be able to reuse that. Um, in fact, she even tried, she even said, I bet I could schedule this, huh? I’m like, yes, you could.

Zack Glaser [00:27:20]:
Uh-huh.

Ben Schorr [00:27:20]:
So she left there like pretty jazzed with a real workflow in hand that was mostly already done. That was going to save her, you know, an hour or so a month. I mean, that’s an hour or so a month’s not huge, but she’s now, she’s got an hour or so a month, right? That’s going to add up.

Zack Glaser [00:27:32]:
I also think about saving an hour or so a month. You don’t just save that hour. You, you now put it to something else productive.

Ben Schorr [00:27:38]:
And I think she also came out of it just as importantly with a, okay, that saved me an hour or so a month. What else could I— that was not as hard as I thought it was going to be. So that confidence and that energy of, okay, I did that thing and that thing’s useful. And what else could be good?

Zack Glaser [00:27:54]:
I’ve got an hour this month to go figure out what else I can automate.

Ben Schorr [00:27:59]:
So that was one good example. Another fellow had a similar thing where he finished early and then he had a task where he had to mine email. He was in the marketing side and He would get a lot of email from prospective clients. And so he had to be able to, for each of those clients who emailed in, they needed to mine that client’s contact info to put into their sales software, right? So they could track the deal as it went through. And so he was manually copying and pasting from Outlook to Salesforce, right? And so he was like, wow, I wonder if— so he had the same experience, right? He did the scenario and then he started to think, Oh, I wonder if I could. And he opened Copilot in Outlook and he had Copilot like, could you, can you extract all these persons, the person, the contact info from this email? And it did it. And then he went, could you extract the contact info from each of the emails in this folder? And it did it. And he was like, you could almost see him getting very joyful.

Ben Schorr [00:28:57]:
This was many hours of very tedious work. Uh, and then he said, could you put that extracted info in an Excel workbook? And Copilot said, yes, of course. And it did it. And he was like, I could just import that into Salesforce. Oh my God. Um, he was pretty excited.

Zack Glaser [00:29:12]:
I wait till he learns about the Salesforce MCP or connector. Yeah. You know, but that, that’s, that’s for a different day. But the thing is, is like what I, what I really want to highlight before we get into kind of like the shape of how these actually like, um, show up the different ways you can do them. What I want to highlight is none of this stuff is legal thinking, legal advice. You know, it is stuff we do in a law office, you know, but none of this requires Copilot to intuit or think or to be a lawyer or any of that stuff that we kind of really get scared about over here, which would be a totally different podcast. But like, all of this is just mundane stuff that people didn’t, you know, I, It’s amazing the stuff you can chip away at so quickly and so early.

Ben Schorr [00:30:02]:
Yeah. And I was just so excited to see, because again, our goal is they come out of it confident and excited, right? We want them confident that they could try some things, excited to try some things, and to see it in real time that, yeah, they gained the confidence to try a real workflow right there on the spot and that they were excited to do it, you know? And then I, yeah, that was tremendous. That felt very good. And so, um, so it was good to feel that the goal was, was achieved.

Zack Glaser [00:30:31]:
Yeah, I, it, I, I’m excited. Ben, I talked to you about, you know, you doing this, and we both knew that this would be successful in some way, shape, or form, right? Um, before you, before you did it, it was gonna have— because we were gonna get information.

Bernadette Harris [00:30:44]:
Yeah.

Zack Glaser [00:30:45]:
Oh, you know, so it was gonna, it was gonna be successful in that way, but to, to see it be like, hit the things that we really, that you really anticipated and wanted out of this has been, it’s really cool from my perspective as well. But when we were talking about this though, you went in person to facilitate this and we’ve got Ben floating around in a conference room with tables full of people. There are a couple different ways that you can do this as well though, that one can facilitate this because I want to encourage people Whether you grab Ben or not, like, do this. You can do this in your own office, you know. But I mean, I— if I could get Ben to do it, I’d prefer that. But talk to me about the different ways that, that, but, you know, offices could facilitate these things.

Ben Schorr [00:31:35]:
So I’ve kind of been thinking about this a bit. Um, so I think there’s kind of 3 different broad venues, uh, that we can talk about. The first one, of course, is the one we just did, which is the in-person, right? Everybody’s in the room. Yeah, the participants are in the room. facilitators are in the room and everybody’s in the room. And to me, that’s the gold standard. I mean, that’s, you’re not going to beat that. You get great energy.

Ben Schorr [00:31:57]:
People can collaborate. You get them away from their desks and away from their phone and away from their email, right? They’re there. They’re in the room. And the facilitator is there in person for face-to-face. You can look over their shoulder and help them with whatever they’re working on. Right.

Zack Glaser [00:32:11]:
That’s the first way.

Ben Schorr [00:32:12]:
That’s the gold standard. The second way you could do it is what I’m sort of referring to as a hybrid, which is all the participants are in the same room. So you get all your participants into your conference room at the office. So they’re around tables, but the facilitator is remote. That is not as good. I mean, I think it’s good from the participant standpoint that you’ve got everybody together face-to-face around the tables, away from their desks, away from their daily grind, I hope, at least for a little while. And then, but it’s not quite as good for the facilitator because now the facilitator is a talking head on the screen. It’s a little harder for the individual tables to flag them down to ask questions.

Ben Schorr [00:32:52]:
It’s a little harder also because if table 3 has a question and the facilitator answers it on the screen, all the other tables are getting the same answer, which may not be relevant to them. So it’s not quite as good. Plus you can’t really see what they’re working on. You can’t look over their shoulder and say, oh, I see what this is doing here. Try it this way.

Zack Glaser [00:33:10]:
Yeah.

Ben Schorr [00:33:11]:
You know, it just gets a little more complicated that way.

Zack Glaser [00:33:14]:
But they’re together. And so there is that, you know, we were talking about the collective, you know, talking to each other, but you don’t, you just don’t have that like backstop of the facilitator walking around the room that can, you know, there’s not as much comfort. And if the goal is confidence, you know, I think we’re losing some potential for that confidence in that.

Ben Schorr [00:33:36]:
Yeah, it’s a little bit worse that way, I think. Um, but it’s also cheaper. I mean, you know, you’re not flying a facilitator to your facility for a day. Okay. I get that. Uh, and then the third way is completely remote. Everybody’s at their own desks or their own, you know, home offices or wherever they are. The facilitators are remote.

Ben Schorr [00:33:50]:
Everybody’s remote.

Zack Glaser [00:33:51]:
Yeah.

Ben Schorr [00:33:52]:
And you’re just doing this over Teams. Um, so that way you lose the in-room energy, which I think is big. When I was at, you know, before I worked at Affinity, I worked at Microsoft. Microsoft does a big annual, they do multiple actually, uh, hackathon events. Uh, we do one big one every year. Um, and when I first started at Microsoft, the big one every year, they put giant tents up on the sports fields. They had, I mean, everybody went and you were in this purpose-built, you know, temporary facility kind of for a whole week. Like, you know, and they brought in—

Bernadette Harris [00:34:23]:
Oh, wow.

Zack Glaser [00:34:24]:
Yeah.

Ben Schorr [00:34:24]:
Around tables and you had the big easels with the, with the, with the pens and everything. Right. I mean, you were immersed. And the energy was tremendous, right? Creativity and the collaboration and it was big. With COVID naturally it had to move remote, right? That was just, that was just the outcome of it, right? Hackathon had to go remote for COVID. Uh, and then it stayed, you know, hybrid at best after COVID. And honestly, I could say as somebody participating in hackathon every year, the, the change was palpable and it wasn’t positive. It was, it was a loss.

Ben Schorr [00:34:57]:
So much of that energy and enthusiasm. And so much of the participation just went down because now you got people working from their own desks. And what ends up happening is instead of before where you were at hackathon the whole day, you showed up in the morning, you grabbed a donut from the table, you sat down with your team around your table and you, you hacked, you thought, you brainstormed, you got creative. Now you were sitting in your own office. Oh, look, an email came in. Let me answer that email. Oh, there’s a hackathon thing happening over here. I’ll get to that later.

Ben Schorr [00:35:26]:
And, you know, now you’re spending maybe a distracted hour or 2 hacking and the rest of the time you’re doing your day job. The palpable reduction in energy was there. And so that’s another reason why I think doing the full remote thing where everybody is in their own environment at their own desk, surrounded by their, their, that brief that needs to go out next week, surrounded by that client email that needs to get answered, right? It just, it just, I just feel like that is going to be a much less effective experience. You really need to immerse. It’s only a 60 to 90 event usually. If we can’t get people together for 60 to 90 minutes to shut off all that other distraction, then maybe it’s not the right thing to do.

Zack Glaser [00:36:07]:
I don’t know.

Ben Schorr [00:36:08]:
But on the plus side, doing it that way, you can do breakout rooms more easily in Teams or Zoom, which might make the facilitator engagement a little bit easier.

Zack Glaser [00:36:18]:
A little bit.

Ben Schorr [00:36:19]:
A little bit.

Zack Glaser [00:36:20]:
But I think definitely balance the benefits because the goal isn’t to do something.

Bernadette Harris [00:36:25]:
Right.

Zack Glaser [00:36:25]:
It’s not just to, to, to do a promptathon in name only, right? The goal is to have people leave with confidence. Like, if you’re, if you’re gonna do it remotely, at that point, in a way, just let’s just have somebody on screen talking at you, you know, let’s have a video. But it, I, I still think having like giving people the, the space to do the prompts and stuff like that. But, um, I was just at HubSpot, uh, their, their annual conference. It’s Unbound. And, and one of the things that I found the single most helpful was this place called Builder Bar. I kept calling it Build-A-Bear, um, just as a joke the entire time. So now I can’t say it right.

Zack Glaser [00:37:10]:
Um, that’s the problem with being an idiot. Um, but at the Builder Bar, They just had computers set up that let you kick around their agent CLI, command line interface, let you kick around their HubSpot MCP, let you kick around their HubSpot CLI. All of these things that I just went in and did it, but there was a human behind the bar that was able to go, hey, what do you need? What questions do you have? How is this? And so like there were some fundamental questions that I needed answered before I started on things and That was the single most helpful thing because I left there with, you guessed it, confidence about how to use this.

Ben Schorr [00:37:56]:
So yeah, it’s big. And, you know, and the other thing we tried to remind them of at the Promptathon is that the destination doesn’t matter in this event. It’s the journey, right?

Bernadette Harris [00:38:05]:
Yeah.

Ben Schorr [00:38:06]:
It’s not what you get at the end of it. At the end of the promptathon, after we let them work on it for, you know, 30 to 40 minutes, They worked on the scenarios. Then we had them report out. We came back together as a room and we went around the room.

Bernadette Harris [00:38:17]:
Yeah.

Ben Schorr [00:38:17]:
All right. What did you try? What delighted you? What disappointed you? Right? What, what, uh, what questions did you have? What tricks did you learn? Anything that stood out for you? And then the other thing is, you know, what are you excited to maybe try tomorrow when you, when you get back to your desk? What are you excited to maybe apply this to? Um, and so we had that report out, but not once did we ask, what did you make? Because that didn’t matter. It doesn’t matter what you made. The output of the Promptathon is not a finished polished workflow, right? That could happen. Don’t get me wrong. It’s possible.

Zack Glaser [00:38:48]:
Yeah, sure.

Ben Schorr [00:38:49]:
Would come up with a, and a couple of people kind of did actually.

Zack Glaser [00:38:51]:
Yeah.

Ben Schorr [00:38:52]:
But that’s not what we’re there for. Right. That’s, that’s not the goal. That’s just a side benefit if it happens. The goal is what did you try? What did you learn? What delighted you? What disappointed you? Right. What are you feeling about what you typed and what came back?

Zack Glaser [00:39:04]:
Yeah.

Ben Schorr [00:39:05]:
And so that it’s the— it’s not the destination, it’s the journey. I kind of, you know, it’s— it was exciting to see that, that, that get reinforced with folks too.

Zack Glaser [00:39:15]:
I love that. But Ben, this, this— I was so excited to talk to you about this today because it is such a cool idea. It is a thing that, you know, like obviously we facilitate these, you know, and if anybody wants to holler at us, you know, we facilitate these things. But it’s also something that people can facilitate on their own. Um, in my mind, they don’t get Ben Shore walking around to, to, to help with this, but like, you could, you could do this as a lunch and learn, internal lunch and learn for your office. Um, so with that in mind, do you have a piece of advice after having done this that people should think about, um, if they’re going to do this on their own?

Ben Schorr [00:39:56]:
You know, a couple things that, that, that came up. Uh, one, have a good intro to it because When the, when the, when the participants come in and sit down, they don’t know what we’re doing, right? What’s a promptathon? What are we doing here?

Zack Glaser [00:40:08]:
What’s our goal?

Ben Schorr [00:40:09]:
Yeah.

Zack Glaser [00:40:09]:
Right.

Ben Schorr [00:40:09]:
So having a good, crisp, and not too long intro that says, all right, here’s what we’re going to do. Here’s why we’re doing it. And here’s a couple tips and tricks you can use. All right. So give them guardrails, lay out the game, right? If you don’t tell them the rules of the game, they don’t know if they’re kicking field goals or hitting home runs. Right. So, yeah. So lay out the rules of the game.

Ben Schorr [00:40:26]:
What are we doing here? That’s the first thing. Secondly, suggest, have some suggested scenarios for every table. Don’t just make them figure out their own scenarios because what you don’t want to do, especially when you have limited time, you’ve got 30 minutes or whatever for them to do it, right? You don’t want them to spend the first 21 minutes arguing about what scenario they should do.

Bernadette Harris [00:40:43]:
Right.

Zack Glaser [00:40:43]:
Okay.

Ben Schorr [00:40:44]:
And that’s the other reason, by the way, why each participant is kind of doing it themselves. Like the table is working on, is collaborating hopefully on scenarios, but they’re not They’re not a team in that you’re going to do part of the scenario and you’re going to do part of the scenario. Like you’re not dependent on each other.

Zack Glaser [00:41:00]:
Yeah.

Ben Schorr [00:41:00]:
So it doesn’t really matter if Alice chooses scenario A and Bob chooses scenario B. It’s okay. But you don’t want to waste a lot of time with that. You want them to get right into it. So having some suggested scenarios that they can run with, you know, is great. That was helpful too. And just try to keep it fun, try to keep it fun and engaging. Don’t make it a chore, you know?

Zack Glaser [00:41:23]:
Yeah.

Ben Schorr [00:41:23]:
Don’t, oh, and don’t make it a competition either. We didn’t, we didn’t, we weren’t grading people. We weren’t, you know, you got first place, you got second place. When you make it a competition, then people start to get a little, you know, edgy about it.

Zack Glaser [00:41:37]:
We wanted to— And they don’t get confidence, right? They don’t all walk away with confidence. I love that.

Ben Schorr [00:41:42]:
Somebody walks away not having won and now they’ve lost confidence. We want everybody to be a winner. This is the place where we’re going to come out of this with, that was great.

Zack Glaser [00:41:52]:
Love it. Well, Ben, thank you for talking to me about this. I really appreciate your thoughts, your ideas, and I, this thing, this is so cool to me. I love it.

Bernadette Harris [00:42:05]:
Yeah.

Zack Glaser [00:42:05]:
Thank you.

Ben Schorr [00:42:05]:
I’m excited about it. We have clients already asking us, inquiring about maybe doing one. And so I’m excited for the next one.

Zack Glaser [00:42:13]:
Nice. Well, um, I always tell people that, uh, you know, we’ve got Ben Schorr here. Find him on LinkedIn. Follow Ben. Um, you can easily find him on LinkedIn. It’s just Ben Schorr, S-C-H-O-R-R. So, well, Ben, thanks again for being with me. I really appreciate it.

Ben Schorr [00:42:32]:
Always a pleasure, Zach.

Continue Reading