Four Law Firm Business Challenges and Analytics-Based Solutions

Episode Notes

Transcript

The Florida Bar Podcast

Four Law Firm Business Challenges and Analytics-Based Solutions

05/25/17

[Music]

Intro: Welcome to ‘The Florida Bar Podcast,’ where we highlight the latest trends in law office and law practice management to help you run your law firm, brought to you by The Florida Bar’s Practice Resource Institute. You are listening to Legal Talk Network.

[Music]

Christine Bilbrey: Hello and welcome to ‘The Florida Bar Podcast’ brought to you by The Practice Resource Institute on Legal Talk Network. We are so glad you are joining us. This is Christine Bilbrey. I am a Practice Management Advisor at PRI and one of the hosts for today’s show, which is being recorded from our offices in Tallahassee, Florida. My co-host is Jonathon Israel.

Jonathon Israel: Hello. I am Jonathon Israel and I am the Director of The Florida Bar’s Practice Resource Institute. Our goal at PRI is to assist Florida attorneys with running the business side of their law practices. We will be focusing on a different aspect of technology each month and will carry that theme through our newsletter and website, with related tech tips and articles.

Christine Bilbrey: So this month at PRI our topic is Law Firm Hiring. And joining us today is Dr. Evan Parker. Evan specializes in the design and implementation of statistical analysis for the legal market. At his company, Lawyer Metrics, Evan has developed several products and services designed to optimize law firm business development and hiring. He also conducts wide-ranging research on the legal industry. His work has been cited in ‘The American Lawyer’, ‘The ABA Journal’, and ‘The Canadian Lawyer.’

Welcome to the show, Evan.

Dr. Evan Parker: Thank you for having me.

Christine Bilbrey: So Evan, tell our listeners a little bit about yourself and what you do to assist law firms?

Dr. Evan Parker: I would be happy to. So Lawyer Metrics is a company that was began in 2010 and it was really a company that was motivated and came to creation because of what was being seen in the marketplace. As I’m sure your listeners are aware, the view of the legal market today is really one that you can capture with a single word, “increased competition,” I guess that’s two words. But this competitive pressure that’s being felt both at the level of large firms but also midsize and small firms is something that Lawyer Metrics recognize and set out to address. They have essentially seen areas related to legal business, legal talent, the way that firms manage their people including areas related to diversity.

What we’ve really tried to do is address these competitive pressures in a way that we can give firms confidence to move forward in this challenging environment.

Christine Bilbrey: So what are you seeing law firm struggling with the most today?

Dr. Evan Parker: That’s a great question. I think there are ways that you can break that out in sort of four main categories and I would say to preface this, law firms probably have lists with hundreds of problems, but if they were to prioritize them I think they would identify four main areas that they’re really working to address.

The first has to do with the business of law. The fact is the market for legal services has been relatively flat for the last five to seven years, and what this means is that firms that are looking to continue growing are needing to do that not by growing organically, but by capturing the market from their competitors. So that’s a business challenge and Lawyer Metrics has worked with firms to address that.

Another area that I think there’s intense pressure being felt has to do with talent, the sorts of attorneys that firms are hiring. For one thing the number of attorneys that are hiring has gone down. They are hiring fewer people. For another, there is a lot more pressure on hiring the right people the first time around, and those two factors go hand in hand.

I think there the talent issue is one that really does fit nicely with the theme of your program related to hiring. Two other areas that I will just mention briefly, I think firms continue to struggle with this idea of how to manage their people and how to get the best out of their people, and this is not a unique challenge for law firms, I think this applies to any organization.

But, really in an environment where there are competitive pressures and there are opportunities elsewhere for the best people, it’s very critical that organization leaders understand what’s working and what’s not and therefore can take corrective action as needed.

The last piece that I will mention quickly, and this is something that I think is a growing prominence in the legal industry and also the corporate world more broadly that the fact that if you look at large law firms, small law firms, there is really a lack of diversity, and this is true both as you move up the ranks for different diversity categories; gender, race ethnicity, LGBT. And so I see a lot of firms working to improve their diversity profile and frankly struggling to do so; and so, some of the challenges there are things that we pay attention to at Lawyer Metrics.

(00:05:09)

Jonathon Israel: And how does your research helped the law firms address some of these challenges, especially like the diversity and inclusion one?

Dr. Evan Parker: Yeah, that’s a great question. I do definitely want to get into that. Let me start though at a higher level, because I think what you want to understand about Lawyer Metrics in our core focus — I’ve just outlined some competitive pressures and some areas that law firms are really struggling, and I want to make it absolutely clear how we’re tackling those problems is consistent across those areas.

And what we are doing at our core is we help firms address their challenges using data. And I know in 2017 this idea about using data, you hear common language like Big Data, Artificial Intelligence, these are kind of hot topics and there are catchphrases and buzzwords that people are paying attention to, but actually this focus on data has really been with Lawyer Metrics since its founding, and in that way I think it was a very innovative and novel approach to thinking about these problems. It’s not new for us.

And so, what that really has done is given us an entry into a market that on the one hand really doesn’t see what Lawyer Metrics does as unique but at the same time we have that long experience and we’re different because we bring a much more rigorous approach to the data. That’s my background. I don’t want to bore your listeners by getting into the weeds with respect to the sorts of analytics we do, but I do want to say the question related to research it is rooted in that data. And the reason that that data can be helpful is because what you’re essentially doing is providing firm leaders or other members of the firm with information that can support how they think about their job and support what they ought to be focusing on.

So we work hard to deliver results that improve understanding, helps these leaders set priorities. When you’re working in data it gives you that opportunity to focus, but also to set a benchmark and say, okay, for example, you asked about diversity. If we see in 2017 that about 18% of our work is being build by diverse attorneys, we now have that number as a benchmark. We can look at initiatives that are designed to improve that and the nice thing about anchoring the information on data is that the next year we can look at that number and ask whether that 18% figure moves up and the hope is that you see it does.

So that’s really I think the core of our business is thinking about the problems and the pressures law firms are facing and how we can solve those problems with data.

One of the other benefits that we have — having been doing this a long time is that we’ve learned how to communicate what are often fairly complicated results and analyses in a way that a lawyer audience can appreciate and understand and use. And we do a lot of that through data visualization and — I have to be perfectly honest, it’s not that we knew to do that at the outset. The fact that we’ve been doing this for seven years has allowed us to make a lot of mistakes, and those mistakes that we’ve made have really I think improved the way that we communicate and educate. So I think that’s really what I would say about that sort of big picture.

You asked about diversity, specifically we’ve been engaged in several different initiatives, whether it’s working with particular firms or with organizations that are set out to improve diversity in the legal profession. At the end of the day it begins with establishing the facts and using data to set a factual foundation so that everyone is working from the same set of facts, and I think one point that is of note is that what’s beneficial in that approach it helps to change the conversation.

When everybody is starting from the same set of facts you can do away with lots of the gut-level intuition and the statements that people just in their heads know must be true but in fact are not true if you look at the numbers.

Christine Bilbrey: And I love anything evidence-based because if I’m going to recommend something to one of our members that calls in I want to be able to prove it to them that it’s worked. So that’s why we reached out to you. So when you go in to help a firm what specifically do you analyze at that firm to customize your advice if they have come to you about hiring and retention?

Dr. Evan Parker: Great question. When we think about what we do on the hiring and the talent side I think we have to divide it into the entry-level attorneys and then the more senior level partner attorneys, and a lot of the work in fact the way that Lawyer Metrics began was by focusing on that, that associate level hiring, that the people fresh out of law school or those who have really been in the practice for just a few number of years.

(00:10:07)

What we want to do with law firms first when we come to them and say, we think you can do a better job at hiring is to make sure that they understand the challenge, and I always find it effective to tell them about a Google study that was done. This was published in ‘The New York Times’ I think in 2014. Google had someone named Laszlo Bock who was really an expert at thinking it through how to hire in a way that you’re getting selection that’s both valid in terms of you are measuring what you want and then also reliable so that you’re getting that again and again.

Well, what Bock did was he took that over tens of thousands of interview ratings that the Google interviewers had collected and he correlated those with eventual measures of performance for the people they hired and what he found was actually no relationship between the ratings that people were giving candidates in their interviews and the eventual performance of those candidates once they were on the job and this is a lesson that firms I think understand but maybe not they don’t understand the basis or the reasons behind this and to really get into it would require a much longer conversation but the essential part is that the standard criteria and the standard processes that firms use to go about their processes of selecting at the entry level just produce invalid information, firms like to look at things like a GPA or they’re often kind of persuaded by a pedigree where the candidate went to law school, and in our research and in other research it’s very clear that those items are not predictive of eventual success.

So what Lawyer Metrics has is a couple different ways of thinking about how to improve the hiring process. One of those is directly related to the sorts of ideas I mentioned with Laszlo Bock at Google which Google actually implemented, and that is to use a more structured approach to the hiring process. So you’re using behavioral-based interviews which means you’re using interviews that are designed to elicit information about the candidate’s past experiences both job-related and otherwise, and the idea there is, when you get a candidate talking about themselves you’re getting information that is predictive. If they’ve done something in the past, they’re likely to do that in the future, and so one of the things that we help firms do is to understand what sorts of behaviorally based questions they should be asking, and to do that we actually go inside the firm and we have people complete surveys, we have people who actually participate in an assessment process so we collect data on the sorts of behavioral factors, traits, behaviors, tendencies on the job that are found among the better performers at these firms and then we map those back to questions that the firm can use.

And the other part of that is — and this is sometimes more feasible than others depending on how many people you have at your firm and the resources available, but to really get these questions to be as valid and reliable as possible, it’s best to have the interview being conducted by more than one person, and so, that’s where this panel interview comes into play.

So in addition to developing behaviorally based questions that get at a valid measure of the person’s characteristics, personality traits and so on you’re zeroing out or balancing out the fact that different people hear things different ways and so you’re really designing an interview system that’s going to produce the best possible information about the candidate’s likelihood of success at a particular firm.

Christine Bilbrey: And I like that, so you’re taking something that has always been very subjective and trying to pull it into a more objective analysis, but what else should someone do like, I know there’s some best practices and are you using the exact same questions for every candidate that comes in, are the same people interviewing each candidate, what are some of the other tips that you would give someone?

Dr. Evan Parker: So the first thing you need to do you certainly have to understand what it is that you want to be measuring and that sounds straightforward in the sense that you in some ways know you want people who are excellent and you want people who are good at doing legal work, but it turns out that different firms have different cultures and so what we like to do is what we call an Organizational Success Study, and I mentioned a few things earlier, but the basic idea is you want to really get at a profile of what it takes to be excellent at a particular firm and that really boils down to the development of a competency model and that’s where we come in and we can help firms build that they don’t have it.

(00:15:03)

A lot of firms actually already do have something like a competency model where they have between say 7, 10, 12 factors that really define the profile of an excellent performer at the firm, and once you have that it’s really just a matter of devising the questions that are going to allow people to talk about themselves in a way that if they can give you an answer that reflects that factor, for example initiative. Initiative is often something that firms believe is really important in their attorneys. If you can get to questions that let the candidates speak about times that they took initiative then you’re starting to get at whether they have that characteristic and whether it’s persuasive, and so, first you would ask that question. In a structured panel interview you would actually have four people involved in the interview along with the candidate and so one person would ask about those questions. Tell me about a time that that you really took charge of a situation on a job, how did it go, why did you do it and so on.

The other three people are sitting there listening and taking notes, and at the end of the interview what will happen is, each of those individuals is going to score the candidate on that particular factor and then they’re going to have a conversation because it’s often the case that those scores at first blush don’t align. You may have someone giving a 6, another person giving an 8, in part of the structured panel processes they have to reconcile that difference and explain why it exists, and this is so important because we know that when people enter into an interview situation even if people hear the same thing, they may actually be having different interpretations.

We don’t really like to always think about this but all our minds are subject to implicit biases and certain things are familiar to us, if it’s familiar we tend to be more favorably inclined toward it and this is just a sort of feature of the mind and Daniel Kahneman, he’s written the book ‘Thinking, Fast and Slow’ that really I think lays this out in very detailed ways but the core problem is that when you’re being affected by that unconsciously it’s shaping your views, and so, even if you were doing these behaviorally based interviews in a way that was designed to lift that past experience. If you don’t have that panel component you’re running into a challenge of — you’re still getting that differential interpretation. So the panel is designed to balance out those inconsistencies.

Christine Bilbrey: And that’s very important going forward. A lot of attorneys think that they are great interviewers and we’ve all heard the questions that I hope most people now know are not helpful, the ones, where do you see yourself in five years or what’s your greatest weakness, those kind of things, and I was showing Jonathan your website. We looked at one of the infographics where it showed that you could just kind of randomly pick anybody and it would be like tossing a coin on the basis of would this person be a successful new hire.

Do you have some figures, or just generally like, what did the odds improve if you are going from a traditional interviewing style? How much can you bump your odds up to getting a successful hire if you do a structured behavioral interview?

Dr. Evan Parker: Another great question. So we’ve done some work. One of the things that is challenging is when firms are hiring they are actually hiring for someone who’s really not going to be at that firm and have accumulated evidence to indicate performance for several years. So we’re actually just getting involved in some back test analyses where we’ve got a firm that’s been working with us for upwards of I think seven years and that’s going to let us really I think provide the exact numbers you are referring to. In a more cursory way I can tell you, what we have seen is there is a relationship between the scores that come out of the interview process and what comes out of eventual performance evaluations and other metrics like hours, work, utilization, and so on, bonuses. So there’s definitely a positive relationship there and by and large you’re improving your predictive ability.

There are some studies that show a structured behavioral interview improves over — this idea of a random coin flip by upwards of 20% and I think one of the things that we do that is worth mentioning is in addition to the structure behavioral interview process we also have a what we call Moneyball model and the details of that are somewhat complicated, but if your listeners have seen the movie ‘Moneyball’ with Brad Pitt the idea behind what we do is exactly the same.

We take information about your current employee roster and how those employees have performed at the firm and we marry that up with information about their demographic and biographical profiles and we begin to build out a biographical profile that’s found among the people who are doing better at the firm.

(00:20:14)

And that turns into a firm-specific algorithm that we can then just apply to candidates who are applying going forward, and I think at first, firms feel like what, is this some kind of a magic or something, but it’s really just a straightforward aggregation of information that’s on a resume. It’s just doing it in a way that human beings can’t do.

So firms I mentioned, GPA and sort of the Law School Pedigree, there are some firms that we see that think there’s a silver bullet. So if I see a candidate and they participated in varsity sports in undergraduate education, then they’re going to do well here.

And that may be true but really that needs to be characterized and taken with all the other information, and so, in this Moneyball model what we’re doing is getting at a way of summarizing all that information with a single number, and it’s not meant to say definitely only hire people with high scores and don’t hire people with low scores, but just like the structured behavioral interview, it cleans up some of the thinking and helps people, nudges them in one direction or the other.

And so when you combine a structured panel interview with information like that you’re really upping your chances of getting good people. When we do the modeling for the Moneyball analysis what we tend to see is you can shift your percentages of As to Cs in terms of grade performance where the As are good and the Cs are the ones you’d like not to hire going forward, and we tend to see a distribution that would shift from about 40% As to 75% As.

And so, there’s a big gain in using these kinds of inputs, and part of the reason that occurs is, because right now, the selection methods if anything are like you say a coin flip or worse because people may be relying on cues and heuristics that really actually predict worse performance.

Christine Bilbrey: Now this is so valuable because — and I’ve seen different studies but the cost of hiring someone new, training them, getting them into your salary and benefit system and setting them up is enormous. So a bad hire can cost a firm a lot of money. So I think that this is so valuable.

What other things, I’m just curious because you said, number three, one of the things you are doing is managing people. How are you using your research to help firms manage the people once they make these A hires?

Dr. Evan Parker: Great question. So a lot of the organizations we work with our larger firms but we’ve worked with several midsize firms and this is the sort of thing even a smaller organization could do. The primary way that we’ve approached this is in a people sense to think about what is going well, what’s working, and what isn’t.

And to do that, we engage firms in an engagement and satisfaction survey where we conduct surveys of various subpopulations within the firm and ask them a broad range of questions that essentially get them to respond to their sense of how things are going at the firm. So we might ask about the nature of the work they’re doing and the communication process, we might ask about coaching and mentoring, we might ask about a whole range of issues, and fundamentally, what we’re trying to do is understand which of these issues are really the drivers of people being happy and engaged at the firm versus not.

And so there what we would do is include some more general summary takes on how people feel about their job. We’ll ask them, are they proud to work at this firm? Do they generally speaking feel happy about working at this firm? And then, what we’ll do is, this is where we get back into the data, we’ll use some statistical techniques to identify the factors that most strongly differentiate the strong and happy and very proud and satisfied employees from those who are not. Because what that tells us is there’s a dividing line and the firm can focus on that particular driver in order to improve satisfaction more broadly.

So to give you one example; if you think about kind of someone saying that I feel that I have a mentor at the firm, if that is a strong differentiator than what we would see is that people who disagree with that statement are typically unhappy and disengaged and people who agree with that statement are happy and engaged.

What that tells the firm is that mentoring is really important and that’s something that they need to focus more on because those people who don’t feel they have a mentor also don’t feel very happy and as you mentioned in your last comment, attrition and retention, these are big issues partly because of the investment of resource that go into hiring somebody.

(00:25:05)

So in addition to finding someone who is the right fit, you want them to stay and in order to know how they’re doing and whether they’re happy and likely to stay, you need to understand I think what’s going on, and so what we advocate is a more systematic way of getting that information, the watercooler is only so effective and it’s a bias sampling and what we do is come in and say, look, we’re a third party, we are not going to share any of these responses, we’re going to report them back to the leadership in a very anonymized and aggregated way.

But at the same time, we’re going to give the leadership some action items and takeaways that say these are the really important factors here, these are separating people who are happy and not, you got to get more people agreeing with statements like, for example, I believe I have a mentor at this firm because people who say that are happy to be here, and when you’re happy, there’s a body of research that shows a whole host of downstream consequences to an engaged workforce.

The one that I think gets the attention of leaders most often is that organizations with higher engagement are also more profitable. So that really — it does get back to the dollars and cents.

Christine Bilbrey: Yeah, now you’re talking, that’s what the lawyers want to hear. I’m curious at Lawyer Metrics, is it a package deal or can someone say I want you to come coach us on that, the actual interviews or do you — like do they come to you with a specific problem or are you a turnkey that comes in and kind of assesses the whole situation at the firm?

Dr. Evan Parker: We tend to be more of a specific targeted approach. We would love for some firm to say come in and just develop a data-driven organization top to bottom, and we have some firms that have brought us in and we do variety of things. I think that one of the issues or challenges that we face is although these ideas about data and big data and AI are starting to enter the conversation, it’s still unclear for people what that means and so part of our challenge is just getting people to understand what it is we’re talking about, what it is we’re going to do.

So in that regard I think it is helpful for us to approach clients and potential clients with an understanding of the particular issues and challenges they face. The nice thing about all of this is — at the kind of core, what we’re really trying to do is help people improve, help firms improve and really improve what it means to be a lawyer.

Benefit is giving back to the legal industry and making this a profession that’s more I think effective, more enjoyable and really as a kind of offshoot of what we do. We’re really trying to give people information that can improve their organization and efficiency and quality so that at the end of the day they can get back to what they really enjoy doing which frankly I don’t think it’s usually management and that sort of thing. It’s being a lawyer and doing the type of work that really drove them to go to law school in the first place.

Christine Bilbrey: I could talk about this forever and it’s funny because our new president-elect of The Florida Bar, her feature issue in her year is going to be quality of life for lawyers. So I think that you’re speaking to that so maybe we could have you back and delve even deeper into this.

But it looks like we’ve reached the end of our program. I really want to thank Dr. Evan Parker for joining us today.

Dr. Evan Parker: Thank you very much.

Christine Bilbrey: And if our listeners have questions or they want to follow up with you how can they reach you.

Dr. Evan Parker: Well, the best way to reach me would be to go to our website which is  HYPERLINK “http://www.lawyermetrics.org” lawyermetrics.org and they can find my email there and contact me directly that’s [email protected].

Christine Bilbrey: And we’ll put that up when we post the recording as well. So this has been another edition of ‘The Florida Bar Podcast’ brought to you by The Practice Resource Institute on Legal Talk Network. I’m Christine Bilbrey.

Jonathon Israel: And I am Jonathon Israel. Until next time, thank you for listening.

[Music]

Outro: Thanks for listening to ‘The Florida Bar Podcast,’ brought to you by The Florida Bar’s Practice Resource Institute and produced by the broadcast professionals at Legal Talk Network.

If you would like more information about today’s show, please visit  HYPERLINK “http://www.legaltalknetwork.com/”legaltalknetwork.com. Subscribe via iTunes and RSS. Find The Florida Bar, The Florida Bar Practice Research Institute and Legal Talk Network on Twitter, Facebook and LinkedIn, or download the free app from Legal Talk Network in Google Play and iTunes.

The views expressed by the participants of this program are their own and do not represent the views of, nor are they endorsed by Legal Talk Network, its officers, directors, employees, agents, representatives, shareholders, and subsidiaries. None of the content should be considered legal advice. As always, consult a lawyer.