How the NCBE will move the NextGen bar exam to personal computers

Episode Notes

Transcript

Announcer:

Welcome to the ABA Journal Legal Rebels Podcast, where we talk to men and women who are remaking the legal profession, changing the way the law is practiced, and setting standards that will guide us into the future.

Julianne Hill:

For decades, the Bar Exam has been old school with bar candidates using pencil and paper exam books, but for the first administration of the next Gen uniform bar exam next year, the new test will be entirely conducted on the examinee’s personal computers. Now, history shown that tech used for bar exam can be filled with snafus pandemic error experiments, remote bar exams had hiccups, and of course, California’s attempted February to launch a new tech dependent exam was nothing short of a disaster. So the National Conference of Board Examiners has been very carefully treading through a maze of tech challenges, knowing that nothing can go wrong on the high stakes exam. I’m Julianne Hill and I’m a legal affairs writer at the A B ABA Journal. I’m filling in for Victor Lee. I cover the world of legal education, so that means lately I’ve been very focused on theBar exam, including the shift from the UBE to the next gen. My guest today on Legal Rebels is Kara Smith. She’s n CBEs Chief Product Officer. It’s her job to get the tech involved for the new exam, exactly right to keep test days free of tech induced drama, but no pressure. Right. Welcome to the show, Kara. First, let’s talk about you and your career. You’ve worked in various positions in the assessment industry. Can you tell me a little bit how that’s prepared you for this big job?

Kara Smith:

Sure. Jillian, first, thank you so much for having me here today. I’m really excited to be here. I love nothing more than to talk about NextGen and the incredible work the team is doing. So looking forward to the conversation. As you’ve mentioned, I’ve been in assessment for many, many years now over 15 years. I am a trained psychometrician. My education was in teaching and then in psychometrics. And then as I got into the assessment industry, I became really excited about thinking how psychometrics and technology can marry together to really deliver an incredibly valid experience for test takers. And so that’s sort of been the through line in my career. Previous to national conference of bar examiners, I was with educational testing services where I ran the product innovation and development division, which gave me a great opportunity to really use advanced technologies in a way that support assessment both formative and summative.

When Judy Gunderson reached out to me about the national conference, I was excited because this is so much more than a new assessment. It really is an incredibly new product experience for the candidates and jurisdictions that we serve. And thinking about how I could really bring a examinee centric focus to the product development area has been exciting. One of the things that we’ve done at NCBE is we’ve reorganized so that we’ve brought product management, user experience, psychometrics, test development, all under the same umbrella. By having all of those groups under me we’re able to really look across the assessment experience and deliver something great for candidates.

Julianne Hill:

So let’s talk about the next gen UBE. As of today, 44 jurisdictions have signed up and 10 will launch the new test in 2026. Now there’s a whole other team working on the questions themselves and that’s a whole other logistical issue, but I’m wondering about the delivery system, which is where you come in. When did work start on that?

Kara Smith:

Well, work began on the delivery system. Well before I even joined NCBE when I joined about 11 months ago now, I was able to take a holistic look at the delivery system. We had run a large scale prototype exam and found opportunities for optimization. And so actually in January of this year, we pivoted to a different delivery system that we’ll be using. We’re partnering with internet testing systems, ITS, pat Ward’s organization to use their delivery platform. So we began implementation of the operational delivery platform in January. We tested it for the first time in April at a pilot test where the technology was really stable, but it gave us a great opportunity to conduct user experience research with the examinees and get great feedback. So since April we have been optimizing the user interface and the feature set and the capabilities of that exam delivery platform. We’ll test it again in August with another pilot test. And then finally we’ll have a large scale beta test across four jurisdictions with 1500 candidates in January, which we’re calling sort of the dress rehearsal for the operational launch in July.

Julianne Hill:

Now, legal ed sources that I talked to on the regular say that NCBE has been very prudent and very steady in its move to the new exam. It’s been about five years since it’s been announced. Why so much time?

Kara Smith:

Well, Julianne, an assessment like theBar exam, which is of such high stakes and impacts individuals’ lives in such a meaningful way, simply can’t be rushed. We want to make sure that of course, as you mentioned, the content is as high quality as can be expected from theBar exam and from NCBE historically. So most of the content that will have been tested on the exam has been in development for three years now. We really wanted to make sure we got it right, but then similarly, as you said, introduce the technology and there’s an even greater layer on top of it. My core first principle with this assessment is that we will not let the technology get in the way of examinees demonstrating their proficiency. In fact, we want to be very careful to make sure that the technology enhances an examinee’s ability to demonstrate what they know.

And so the team has spent months and months and will spend more thinking really critically about every feature that we implement in this delivery platform. Something interesting about NextGen is that the innovative item types that we’re using, especially the integrated question sets are complex. And so we needed to make sure we understood how those questions were going to render themselves on the user interface. And that takes a lot of time to get right, but also we are very, very keen to make sure this is a completely accessible experience for examinees. And so we have partnered with a third party organization that is doing a long-term audit of the accessibility of the delivery platform. By the time we launch jurisdictions will have in hand what we call a VA or it’s a certificate that indicates that we have not only met but exceeded accessibility standards. And so thinking about the content, making sure the delivery platform is right and making sure it’s completely accessible for all examinees just takes time and we want to have it tested enough that we have no concerns going into exam day.

Julianne Hill:

Whenever I talk to people about the delivery of theBar exam, the first thing they mention is the importance of security. And that goes two directions. It has to be that the examinees can’t cheat and the questions can’t be released because they’re reused. And that sounds like huge, huge hurdles for delivering a bar exam via computers. Is this something that you worry about at night?

Kara Smith:

Well, there are many things I worry about at night, Julianne, but this is certainly one of them. I mean, security is critically important and again, our partners at ITS are doing such an incredible job of working with us both on the technology, the security component in the technology and the process for the security. And so I don’t want to go into too much specific detail because we don’t want too many people understanding the specifics of how we’ve built the security in, but I will say that we’re taking great insurances to make sure that the examinee that is sitting for the exam on test A is who they say they are. We ensure that there are session IDs that can only be used in the location. We can do things like check IP addresses, make sure that people are where they say they are. And then we also do process components like staggered start times across time zones to ensure that content exposure is greatly reduced. And of course we have wonderful proctors in the room who are monitoring things. What I find really interesting about examinees who are taking theBar exam, everyone I’ve talked to during user experience research, they really seem like they want to maintain and protect the security of this exam and they have everyone’s best interest and that’s wonderful, but again, we just have to make sure all of the insurances are in place. So we have taken great steps here.

Julianne Hill:

So it sounds like the tech involved is the entire experience from registration to bar prep to taking the test to getting the scores that security and all of the tech involved. It’s not just that day, it’s soup to nuts.

Kara Smith:

Yes, and that’s as I mentioned before, what is sort of exciting, we’re really building an entire product ecosystem. So in March of 2026, we’ll be launching the new candidate experience inside of the NCBE account. The new candidate experience is the first time that next gen examinees will get some idea of what the next gen experience is like. So they’ll access this platform, it gives them all the information they need. We understand the importance of reducing test a anxiety theBar exam is such a high stakes assessment that we know there’s enough anxiety there. We don’t need examinees to be anxious about do I have to bring a power cord? What device am I going to be using? So we’re taking great steps through the registration and what we’re calling the ready to sit process to ensure that examinees feel confident, comfortable, and ready to demonstrate their proficiency on exam day.

And then following the delivery platform, there’s the new scoring platform that we’re putting in place. The grading platform and scoring platform also needs to be highly, highly secure given that there is secure content housed in there. So we have a whole through line security integration from the candidate experience through the delivery platform, through the scoring grading platform. And then when the results are shared with the jurisdictions, that needs to be secure as well. So in addition to a new candidate experience, we’re launching a new jurisdiction experience inside of the NCBE secure site. That new jurisdiction experience is highly, highly secure and ensures we have sort data security throughout,

Julianne Hill:

So the students are going to use their own computers. That kind of surprised me. I mean honestly, it doesn’t feel like it could be as secure as another lockdown. What are the risks and how does that work?

Kara Smith:

It’s a really great question. The NextGen UBE is a bring your own device experience and that can be risky. However, as I said, we’re putting steps in place to ensure that candidates are prepared. So for example, prior to exam day, examinees are required to complete the exam tutorial. The exam tutorial includes a whole training on all of the features of the delivery platform as well as sample questions. This allows for examinees to get comfortable with the experience, but in the background it’s also running a systems check on the device to ensure that it has the right capability it needs to help the examinee be successful, but also maintain security on exam day. The browser that examinees is used is a locked down browser to ensure that they can’t get to other components of their devices, for example. So we have that security measure as well. And so we recognize that bring your own device can be risky, but again, we’re really putting measures in place to ensure that we reduce those risks. We also ensure we have redundancies. So if a device fails because of the software, for example on exam day, we don’t expect that to happen, but we’ll have backup devices available for examinees at the location so that we can quickly swap out the devices.

Julianne Hill:

So I’m wondering ahead of this, if law school faculty and support staff will have a chance to run through it themselves to help students and potential candidates because candidates aren’t always students to get comfortable with it, to support teaching this software or if there’s connections through the law schools as well.

Kara Smith:

It’s an excellent question and I think that it’s so important because as we’ve been discussing with digital assessments, the delivery platform and understanding and getting comfortable and helping educate on the delivery platform becomes part of the validity narrative. So on August 5th, we will be releasing the updated NextGen UBE website. On the website there will be a link to the exam tutorial and sample questions. Anyone can have access to the experience there. So you can play around with the question types. You can look at the features, not just examinees, but law school faculty or other support systems that are supporting.

Julianne Hill:

Let’s take a break here and we’ll hear from our sponsor when we get back. We’ll talk about what happens on test days with Kara Smith and we’re back with Kara Smith of NCBE talking about tech and the new bar exam and what examinees can expect on test day during the 2026 administrations. So test day is the big day and despite the exam being conducted on computers, it’s going to be conducted live in person with proctors. Talk about that decision and how you walk in and fire up your computer.

Kara Smith:

So a lot has been discussed about remote testing and as we mentioned, I have been in assessment and digital assessment for quite some time and my view is just that the technology is not there yet to ensure a really secure, really stable remote proctored experience. We believe strongly in the validity and fairness and reliability of this assessment. And so given where the technology is, it just makes more sense to deliver this assessment at assessment venues with proctors. So here’s what examinees can expect. First, I recommend examinees arrive at least an hour before the exam. That will give them a chance to sit down, get themselves set up, feel confident, feel comfortable, and have plenty of time. It’s very important that examinees bring the same device that they completed the exam tutorial and sample questions on because as I mentioned that has run the device check in the background, we’ll be able to see if the device is a different device and they ran the exam tutorial on when they launched the secure browser. The examinee will not be able to launch the assessment if they have not completed the exam tutorial on the device. We’re not trying to block anyone from taking the assessment, we just want to make sure they’re set up for success. So if an examinee gets the test site early enough and they haven’t taken the exam tutorial, they can do it right there on site to make sure that they’re stable.

Julianne Hill:

So then they fire up their computer and the software will be loaded on that.

Kara Smith:

Yes, so as part of the ready to sit steps that we ask examinees to take, the first thing is downloading the secure browser onto their desktop. When they get to exam day, they’ll just click on the icon on their desktop and it’ll launch the secure browser. The secure browser will lock down their device. As I mentioned, examinees will be asked to put their NCBE ID and last name into the system. They’ll then be prompted for a session id. The proctors in the room will read out the session ID. Once everybody has logged in, once the session ID has been put into the device, the proctors will be able to see on their administrator screen. If everybody is ready to go, the exam will not start until everybody in the room is ready to go. Once they are, the proctor will hit start all and everyone will begin the exam at the same time.

Julianne Hill:

And the passwords are specific to that room. It’s not like somebody in the hallway can jump in on that

Kara Smith:

Exactly. The session IDs are specific to the room and a secondary check we have is that a proctor on the screen can see once all the session IDs are in the IP addresses, we expect that all the IP addresses will be the same. If there is an outlier or asking the proctor to identify who that person is, ensure they’re in the room again as part of those security measures that we’re taking,

Julianne Hill:

Then when do they get the questions and how?

Kara Smith:

So the next gen UBE is a connected exam. What that means is that examinees have to be connected to the wifi while they’re taking the exam. When the proctors launch the exam, the first three hours of content will be cached, what we call cached onto their device. What this means is that those, that first section will come down and they can proceed through the first section even if the wifi goes out, if the wifi is stable, as they’re proceeding through the section, their data are immediately being uploaded into the cloud and off their device. If the wifi happens to go down, we’re going to have redundancies, so hotspots or backup wifis in all of the testing rooms so that the technical proctors will be able to launch the backup wifi at the end of the section. And the data from that first section will be uploaded to the cloud.

Julianne Hill:

Will they be able to skip around and answer questions and go back within that three hour timeframe?

Kara Smith:

Yes, so we consider NextGen an integrated assessment. So each of the sections is structured the same way and examinees can jump between which questions they want to do first, for example, and if an examinee wants to complete the performance tasks first, they can go there and come back to the multiple choice questions. And then onto the integrated question sets, it’s really up to them how they want to manage their time and how they want to pace through the section once the first three hour section is completed. However, they cannot go back to that section once they’ve started the next three hour section, but they could always move freely in the three hour section.

Julianne Hill:

So at the end of the three hour section, what happens to the answers? I know in California people were concerned that maybe their answers weren’t uploaded or got goofy. Let’s just say

Kara Smith:

We really, as I said, want to instill confidence in the candidates. So we have a couple of measures here. One, as I said, if they’re connected to wifi, their answers are being uploaded in real time as they’re answering the questions. If they are not connected to wifi, we’ll get them connected to wifi so that their data get up, they’ll see on their screen a confirmation message that their data from that section have been uploaded and are no longer on their device. They’ll also get an email that the data have been uploaded. And thirdly, there’s an indication in the next gen component of their NCBE account that will tell them after each section, has the data been uploaded? Do we have their answers? Can they feel confident? So let’s say for example, they wanted to log into their account on the first break just to be triple sure their data are there. We have that capability for them in place. And similarly on the proctor screen in the room, the proctors will see as data are uploaded for each of the examinee. If an examinee wants to leave the testing room, the proctor will ensure if their data have been uploaded before confirming they can leave the room.

Julianne Hill:

This sounds like a big ask for the proctors, not your run of the mill proctors who are just walking around the room. Are they trained?

Kara Smith:

We have set up training structure and sort of a staffing plan here for in each of the rooms there will be an ITS certified technical proctor. That person’s responsibility is really troubleshooting any technology issues that occur in the room. That’ll be a certification that requires an assessment at the end of the certification to ensure those technical proctors can be in the room. Those are supplied by NCBE. Then there are administrators in the room. They will be highly trained on the delivery platform and on the administrator screens. What we’ve been doing is conducting a lot of user experience research with administrators because we want to reduce the burden on them and increase their capability to quickly search the portal, for example, to quickly be able to understand what’s going on in the room. So we’ve gotten a lot of feedback from them on what this is really going to look like and how we create efficiencies, and I think we’ve done a great job there.

Julianne Hill:

So for each section that process is repeated, the new questions are provided and there’s the three hours. And I’m wondering what happens at the very, very end of the very, very last day of all this, if there’s a different process, how do you know you’re done? Done?

Kara Smith:

The third, third and final section functions the same way as the first two sections have. When an examinee is done with a third section, they’ll get the same message that their data have been uploaded. Again, they’ll get an email saying, okay, we have your third section now. And if they log into their NCBE account, they’ll see on their portal that all three sections have been uploaded and there’s a little tracker in their NCBE account that shows them where their data are in the process. If the multiple choice questions are done being scored, if the constructed response questions are being graded at that time, if grades have been completed, we really want to help examinees feel confident that they can follow along in the process and know what to expect for where their data are and when they’re going to understand what their results are.

Julianne Hill:

So of course, at the end of the exam, the next big step is grading. Have there been any changes to the process of grading? And of course the question is, will AI be used?

Kara Smith:

Yes, thank you for asking. And really great improvements have been made in my opinion, to the grading process. We have implemented a global grading platform, again partnered through ITS. It’s a wonderful grading platform. We are implementing double grading, so what that means is that two graders from the jurisdictions will read the constructed response questions and score them. If they’re not in close enough agreement, what we would say outside of tolerance, then that score goes to a third grader who does a reconciliation of that process. We really think this implements a great deal of validity, but it’s almost like on the fly regrading because you’re having two graders graded at the same time and then adjudication or conciliation. We’ve also been able to implement a really robust greater training program. The constructed response questions on NextGen are really fantastic and complex, and so we want to make sure the graders feel confident to understand how they should approach the grading process. So we have a whole greater training workshop, we have videos, we’re there to support them along the way. I really feel like we’re supporting the jurisdictions with the grading in NextGen.

Julianne Hill:

So the graders have to learn this new technology too.

Kara Smith:

Exactly. So as part of the greater training workshop, the first module we call them is new to NextGen. We explain what is new from NextGen from the legacy UBE, and then we talk about the platform, understanding the platform and it’s a hands-on training taking ’em through that, and then they move into the question specific greater training workshops, and it feels like a big ask for these graders, but what we’re finding is they’re really loving learning about the innovations that we’re implementing in NextGen and they’re really excited to understand the efficiencies that are being delivered through this digital platform.

Julianne Hill:

This seems like a good time for a break. When we return, I’ll ask Kara Smith from NCBE about how the user experience has been tested and tweaked. Welcome back to Legal Rebels. We’re here with Kara Smith of NCBE talking all things tech and the new next gen bar exam. This new exam is different from the UBE in that each set of questions, each three hour grouping has the same mix of multiple choice questions, performance tests, et cetera. Was tech a consideration in that structure or the other way around?

Kara Smith:

Great question, Julianne, and it’s a complex answer because it really wasn’t one directional in either way. We really thought how can we marry the technology with the content? So we had our test development team, our user experience team, and our psychometrics team work hand in hand with the technologists that were implementing the delivery platform to understand how each of the components can enhance each other. If you take the integrated question sets, for example, the integrated question sets provide a library of resources and a case file and then a number of components that make up the integrated question sets. It took a lot of time to understand how those items could be rendered in the delivery platform. Things that you might think are not really consequential really are, so for example, understanding how an examinee can use the real estate on their laptop, we implemented a scroll bar in the middle of the screen so they can pull it all the way over to only see the resources or pull up the other way to answer the questions. We had to think about how we were going to anchor the content to the components on the right hand side of the screen. So it really was about test development psychometrics and technology working together to really collaborate on how we were going to build these questions and what the structure was going to look like in the delivery platform. And I was just so proud of how the team worked collaboratively to achieve that goal.

Julianne Hill:

It seems like since this is an event exam where everyone gets the questions and the answers are going to be uploaded at pretty much the same time that this is a big technical ask, it’s a big load. I’m wondering how you know the system isn’t going to get overloaded.

Kara Smith:

Yes, thank you. And honestly, the way that I know is that I have complete trust in our partner. I-T-S-I-T-S has been delivering digital assessments for decades, and one of the first conversations I had with them, pat Ward, shared with me that they have some days where 90,000 examinees are accessing the same delivery platform and they’ve never had a widespread technical failure ever. That really instills confidence When we’re thinking about the numbers we’re going to have.

Julianne Hill:

How many people do take theBar exam at once if they’re talking 90,000? What’s the range of, I mean, it is tens of thousands.

Kara Smith:

Tens of thousands. Yes. So in July of 2026, for example, we expect we’ll have somewhere between 30 505,000 examinees. That load is really easily handled by the ITS system once we’re fully implemented and we’re thinking closer to 60,000 examinees of accessing at the same time still that load is lighter. And I’ll tell you why we have these staggered start times because of the time zones we have though NextGen is an event-based assessment, folks in New York will be taking at a very different time than folks in Guam, so that also reduces the load. But even at full capacity, I would not expect we would have a technical failure based on the history of ITS and the assessments that they have delivered.

Julianne Hill:

You talked a little bit earlier about the pilot tests. Are there any specific things that you’ve changed as a result of these pilots that you went, oops, didn’t think of that or people were complaining about X and things that are different going forward as a result of doing all of this testing in pilots?

Kara Smith:

Yes, absolutely. We’ve learned so much from the pilots and I’m just so incredibly appreciative of the constructive feedback that all of the examinees and all of the administrators from the jurisdictions have provided along the way on the delivery platform. And so there have been some big optimizations. For example, we had previously had resources where PDFs would pop up and sometimes that would free screens or sometimes examinees wouldn’t be able to see the questions. We’ve completely overhauled that and now all of the resources are built directly into the delivery platform, so nothing needs to open. They can scroll freely. The feedback we got when we retested that was just unbelievable. People were just so thrilled about that large enhancement. And then Julianne, there have been other pieces of feedback that have been seemingly minor but aren’t. So for example, there’s a timer that counts down from your three hours to help you understand your time management.

Some examinees have said that timer creates anxiety for me. I do not want to see that timer counting down. So we’ve implemented a feature where there’s a toggle button. You can either have the timer on if it’s helpful to you or you could toggle it off if it creates anxiety for you. We’re really trying to give examinees agency with this platform. So we also have features that we heard from them. They were interested in like copy and paste, highlighting features that they’re typically used to using in their everyday life, and that feedback has helped us understand which of those features we should implement in the delivery platform. So the optimizations that we’ve made have been what seems small of different color blue on the screen because people said the blue is too dark. All the way to how we structure our content has really evolved. As we’ve gone through this testing,

Julianne Hill:

I see that the number of requests for accommodations has gone up dramatically in the past few years. How is the next gen UBE going to accommodate the accommodations?

Kara Smith:

Yes, there are many more requests for accommodations and we want to make sure that we’re meeting the needs of all examinees. We’re addressing this in a couple of ways. One is we have many capabilities built into the delivery platform that achieve the accessibility that’s required of examinees. So we have things like color contrast, we have things like Zoom where it used to be the jurisdictions needed to order different font sizes for the paper-based books. Now we have solve that with a UI of the delivery platform with zooming the text. So there are many accessibility features built right into the platform that will support examinees. We also have many accommodation capabilities that are available in the platform. We have things like speech to text so that the screen can be read to the examinee, the examinee can speak their responses if they need to, and they could be brought to text into the delivery platform.

We have the ability to deliver the assessment digitally for extended time candidates. We have stop the clock, pause breaks built right into the platform. So what we’ve done is thought through all of the accommodations that have been supplied through the legacy UBE and we have optimized how those accommodations are supported through a digital technology, and we’re really excited in January in the beta to not only be testing with standard candidates, but we’re going to be testing out all of those accommodations in the January beta as well so that we have data-driven evidence to give to the jurisdictions to help them feel confident that their accommodated candidates will be supported as well.

Julianne Hill:

Really interesting. What do you still have left to do?

Kara Smith:

Well, we are in the final stages of the user interface build, as I mentioned, we’ll pilot test it again in August. Interestingly, I don’t think we have too much left to do as it relates to the platform in terms of the delivery platform. We are hard at work at those other technology components around the assessment itself. So the new NCBE account, the new jurisdiction portal, the scoring platform is still being implemented. But my priority when I joined NCBE was to get the assessment and the delivery platform stable, highly functional and highly usable. That was the priority because that’s what really matters on exam day is an examinee being able to trust the technology is not going to get in the way of them demonstrating their proficiency. I think we’ve achieved that. Well really, really see in January when we test with 1500 examinees, we’ll be in Texas, Florida, New York, Massachusetts, and we’re going to have them all launch at the exact same time. We’re going to have them implement. We have some sort of disaster scenarios I’m calling them that we’re implementing intentionally to see what if the wifi goes out, what are we going to do and ensure that those redundancies and those risk mitigations in place have really been developed in a way that we can trust on exam day. So as it relates to the exam, I think we’re solid. I think now it’s really about optimizing all of the other experiences to ensure that examinees and jurisdictions have a really great testing experience.

Julianne Hill:

Have you learned things from other tests, other high stake exams, MCATs or SATs? Is there an exchange of information on how to handle these types of issues?

Kara Smith:

Yes, I think we’ve learned a lot from a lot of testing organizations. Personally, I’ve had the great pleasure of working at the college board and Pearson and McMillan Learning, so I’ve ETS. So I’ve experienced many of these types of digital assessments in the past, and so I’ve been able to learn so much. Of course recently, technologies advancing so quickly that you need to think about all these new components of it, and so we are learning a lot about what are the redundancies and risk mitigations we can put in place just to name a few. As I mentioned, we have the backup devices that will be in the room. We have a redundant wifi, so if the wifi in the venue goes out, we’ll be able to fire up a redundant wifi. We have tested the load of the delivery platform. We have no concerns about the load. We have highly trained technical proctors that will be there to support. So I think we have all of the right mitigations in place that we need to ensure a smooth exam day, and Julia and I say this with no expectation, anything’s going to go wrong at all, but with an event-based assessment like theBar exam that is this high stakes, we are not leaving anything to chance at all. We want to make sure everyone has a successful exam day.

Julianne Hill:

Another very basic question, but for examinees, I’m wondering Macs or PCs, any difference? Do we have to have a laptop? What gear is expected and what’s at work best on

Kara Smith:

It is not a basic question at all. In fact, it’s a really important question. My recommendation is a Mac or a pc, we will not be delivering the exam on tablets, for example, and so a Mac or a PC is what I would advise. One of the things that we have done with our user experience is taken great care to ensure that the delivery platform and the technology is exactly the same on a PC or a Mac. We don’t want the experience to vary based on device because that’s a validity implication, and so the user interface is exactly the same on each. So I would recommend that examinees come with a device they’re most comfortable using in their everyday life. I’ll also mention in the exam room, those backup devices that we’ll have will have an equal number of Macs and PCs. So if you’ve come working on a Mac and something happens, you don’t have to shift to a PC because I’ll tell you, if I had to make that shift in the middle of an exam, it would certainly throw me off. So yeah, so bring the device that you’re comfortable

Julianne Hill:

With. Right. When I’ve talked to people about the next gen and the fact that it’s on the computers, more than one person has said, is it a matter of time until the test is fully remote? Is that on the table?

Kara Smith:

A lot of examinees will say that they appreciate a remote proctored exam For the convenience of it, I think it’s critically important that we ensure the security and the validity of the exam. I want to meet all of the needs of examinees who would like to test remotely, but the technology is just not there yet with the rate at which technology is advancing. I think it’ll get there, and it is there for many assessments right now for mid stakes or lower stakes assessments. I fully support remote proctoring in those cases, but for a high stakes assessment like theBar or not there yet, I look forward to when the technology does get there and we can meet all the needs of examinees though

Julianne Hill:

Once the test is launched. What’s next? Is it continually updating the technology to, as you said, things are changing so quickly? Is that going to be a job for you to just keep refining?

Kara Smith:

Absolutely. Everyone is talking about July 26 as the launch, and it is the launch, but those 10 jurisdictions who are first adopting in July 26, that’s their launch in July 27, we’re going to have another launch for jurisdictions that are transitioning to next gen, and then in July 28 when we’re fully implemented, it’s sort of another launch for the July 28 jurisdictions. What we’re going to do is we’re going to learn a lot from the July 26th administration and what could help examinees more. We want to get more study aids out, we want to get more prep information out around the assessment. We want to use technology in a way that helps them prepare. You mentioned ai and while we are not using AI to generate any operational items for the next gen exam, nor are we using AI to score the next gen exam in 2026.

We are deep in research and development on how we can use AI to better support examinees and jurisdictions. So for example, we’re using AI to generate content for prep material for study aids. We’re not using it operationally yet, but we’re conducting research and development on it. We want to make sure that we are responsible with how we’re using AI or transparent with how we’re using ai, and we’re doing it in partnership with jurisdictions and examinees. And so a lot of the work that we’ll be doing over the next couple of years is understanding what our AI strategy is and how we’re implementing AI into our product strategy. That’s a really exciting horizon on the road. The last thing I think is really important for the future strategy of NCBE is it would be fantastic to move away from event-based testing. It would be great to get in test centers, have window based testing or on demand testing that requires an incredible amount of content, and we are thinking about how we deepen our item pool, how we broaden our item pool, and how we can get to a place where we can move away from event-based testing and really allow for more flexibility there

Julianne Hill:

To wrap up. Thanks for joining us, Kara. It’s always a pleasure. I’m Julianne Hill for the A b ABA Journal filling in for Victor Lee, and thank you for listening to today’s show. If you enjoyed it, please rate us on your favorite podcasting app and check out some other titles from Legal Talk Network. While you’re at it,

Anouncer:

If you’d like more information about today’s show, please visit legal rebels.com. Legal talk network.com, subscribe via iTunes and RSS, find both the a b ABA Journal and Legal Talk Network on Twitter, Facebook, and LinkedIn, or download the free apps from a b ABA Journal and Legal Talk Network in Google Play and iTunes. The views expressed by the participants of this program are their own and do not represent the views of nor are they endorsed by Legal Talk Network is officers, directors, employees, agents, representatives, shareholders, and subsidiaries. None of the content should be considered legal advice. As always, consult a.