Real Talk About Artificial Intelligence in Your Office, with Ben Schorr

Announcer: 

Welcome to The Lawyerist Podcast, a series of discussions with entrepreneurs and innovators about building a successful law practice in today’s challenging and constantly changing legal market. Lawyerist supports attorneys, building client-centered, and future-oriented small law firms through community, content, and coaching both online and through the Lawyerist Lab. And now from the team that brought you The Small Firm Roadmap and your podcast hosts 

Zack Glaser (00:35): 

Hi, I’m Zack. 

Sara Muender (00:36): 

And I’m Sarah Muender. And this is episode 497 of the Lawyerist Podcast, part of the Legal Talk Network. Today, Zack talks with Ben Schorr about preparing to use artificial intelligence in your law firm. 

Zack Glaser (00:49): 

Today’s podcast is brought to you by Rocket Matter, and you’ll hear more from them in my conversation shortly. 

Sara Muender (00:55): 

So Zack, we just got off of an internal team call where you presented for an hour on the topic of ai. And so here we are. We’re continuing this conversation. And I have to say it’s one of those things where it feels like that’s all that can be discussed all the time anymore, and it’s one of those topics that’s like everywhere, whether you’re going to a conference or listening to a podcast or our team meetings, it’s all about ai. And it sort of feels like it did when the internet first came out where it was just like this thing that was coming like a huge tidal wave, and it’s here and it’s sort of like you don’t know what you don’t know, but you know that there’s going to be huge worldwide implications. And so for the average listener of the Lawyerist podcast who is a solo or a small firm owner, why is this such an important topic for them to really pay attention to now, even if it’s something that they don’t think affects them in the immediate, but what are the implications and why do we all need to tune in? 

Zack Glaser (01:59): 

That’s a really good question. It’s because it does affect them in the immediate artificial intelligence. Whether you know it like it or whatever or not is a part of your life. And if it’s a part of your life, it’s a part of your practice. I send my wife text messages and my phone, I start to type in headed and it goes headed home and we’ve got predictive text. My Netflix account, whether it’s doing a good job or not, I don’t know, but my Netflix account is trying to tell me what movies or TV shows I’m going to like based on the things that I do. And so we are affected by artificial intelligence. This isn’t something where you can stick your head in the sand. This isn’t something that is going to be over soon. This is a fundamental technological shift in how we think of things. So I mean, I guess that’s just, I don’t want to get too far into it, but it’s here. It is affecting your office. And if we as attorneys are going to act reasonably, which a lot of our professional obligations require us to do, then we’re going to need to understand this. We’re going to need to at least have a basic idea of how we’re going to use artificial intelligence. 

Sara Muender (03:17): 

I mean, when you think about it, of all industries that should really pay attention to this, it’s the legal industry. 

Zack Glaser (03:25): 

From a lot of different standpoints. We are going to have to advise our clients on regulations. We’re going to have to advise our clients on how to use these things. We’re going to have to do these things internally. We’re going to have to know the regulations for ourselves. We’re going to have to know what tools are available for ourselves. And it seems like too much. There’s a water hose of information coming at us. There’s so many AI products that are out there or products that aren’t even AI that are just saying they’re ai. So I would encourage people think of it. I think of it like the gold rush. Yes, there were some people in the Gold Rush that made money mining gold, but the people that made money were the ones that sold shovels. 

Sara Muender (04:16): 

Love that. 

Zack Glaser (04:17): 

Or maps or where is the gold. So we as attorneys don’t necessarily need to be using every single product that comes across. We don’t have to make sure that we’re way, way, way ahead of the game or striking it rich with one particular product that we’re using. We want to kind of stick to the basics. We want to set ourselves up to use artificial intelligence intelligently. Is that in a responsibly? 

Sara Muender (04:46): 

Absolutely. Well, just like you talked about in our internal team discussion today on the topic, there’s ways to use it appropriately and there’s ways to misuse it and it can really be a benefit if you learn to leverage it, and that’s the future of law for sure. So let’s get on with your conversation with Ben right after this message from our sponsored guest. 

Zack Glaser (05:15): 

Hey y’all, it’s Zack, the legal tech advisor here at Lawyerist, and today I’d like to talk to you about timekeeping, passive timekeeping, timekeeping on your computer. I’ve got Joyce Bradford with me from Rocket Matter and Profits Solv, and she’s going to tell us about a new product, I guess called Rocket Matter Track. Joyce, thanks for being with me. 

Joyce Bradford (05:37): 

Hey, as always, Zack, huge pleasure. It’s kind of highlight of my day to be able to talk about the fun stuff with you, so thanks for having me. 

Zack Glaser (05:45): 

Absolutely, absolutely. And this is fun stuff. You actually came to the studio today and introduced me to a totally new thing that I hadn’t heard about called Rocket Matter Track. Let’s dive into that. What is this? 

Joyce Bradford (05:58): 

Okay, high level, it’s passive timekeeping, and this is not timekeeping just for people who need to bill on an hourly basis. This is for absolutely everyone. So why timekeeping, why passive and why capture everything, right? Yeah. Number one, you need data points to make good decisions about how to actually bill, about where to invest your time, about where to invest in your business, right? That’s number one. You have to have the data to do that. Why passive timekeeping? Because clicking a timer and moving from timer to timer to timer is the most tedious, annoying thing in the world. And everyone says you have to track all your time in order to be a good business person, but man, it’s so easy to miss it, and it’s really easy to not record the administrative things, right? If it’s not billable. So passive lets you capture absolutely everything. So Rocket Matter track is comprehensive passive timekeeping. And I’ll pause, I’ll pause there, Zack, but I could talk for another 17 minutes. Jump on 

Zack Glaser (06:56): 

This. Well, we’ve only got about four minutes left, but okay, so passive timekeeping, are we talking just inside of Rocket Matter? So I’ve got Rocket matter up and I’m just rolling. Is it just inside of Rocket Matter? What if I use a Mac my phone? If it’s not kind of comprehensive passive timekeeping, I don’t know that it helps me that much. 

Joyce Bradford (07:15): 

Oh, I love that question. So this is platform agnostic. It actually lives on the machine that you’re working on. So it doesn’t matter. It does not matter what type of machine you’re working on. It does not have to be inside rocket matter. So you’re working on emails, you’re working on documents, you’re working on legal research, you’re working on websites that are not your practice management system. It doesn’t matter. Rocket Matter track is going to work for you. And if you have an open timecard for a current client, it will pre track matters with certain keywords to that client. It will keep track of everything that you’re doing across your machine, across every application, across every program that you have open and say, Hey, this is what you’ve done and this is where you spent your time. But here’s the really cool part of this, right? 

(08:00): 

It’s not automatically uploaded to Rocket matter because how many times do you check the weather? How many times do you go to your preferred news source? How many times do you go to your social media that doesn’t have a thing to do with a client and you don’t want to bill a client for that time, right? So this audit trail, it’s fully editable with in Rocket Matter track. It’s completely reviewable and it’s comprehensive. So hold onto it for as long as you want. And if you are the type of biller who only bills once a month and only goes back to your calendar and your emails and your invitations and your documents and everything that you’ve done for a client at the end of the month and billing day is a disaster for you, and you don’t have a billing day, you have a billing week because it takes that long to put your invoice together properly. Rocket Matter Track is going to save you a lot of time. You’re just going to be able to drag and drop things to the correct client and you will see a true accurate record of what you have done, your client and for your law firm for the entirety of time that you’ve been using the tool, 

Zack Glaser (08:59): 

Right? So I think the big thing here, obviously it’s tracking stuff that I do for a client. I mean in and of itself is very big to be able to just kind of drag and drop that, make it really easy. But the thing that I’m kind of latching onto here is the admin aspect of this. So I’m passively tracking everything that I do. I’m tracking Rocket Matter, it integrates with QuickBooks in order to do your accounting. So I can then track what I’m doing in QuickBooks. It’s not billed to a specific client, but now I know how long it takes me to do this stuff and I can start, well, I mean Rocket Matter has all those, the business analytics, the business reports, and I can start to say, well, how long am I spending on this? How much does my client intake cost me essentially even when I don’t have anything? Yeah, that’s fantastic. Is Rocket Matter track a thing that’s built directly into, is this an add-on how would somebody get Rocket Matter track? 

Joyce Bradford (09:54): 

So you have to be a Rocket Matter client, but then you can download it and there’s no additional cost for the product or for the connection back to Rocket Matter. It’s just included in the subscription. 

Zack Glaser (10:03): 

Great. So we’ve got a lot of passive tracking here. I’m on Mac, do I download it, put it on my Mac and then on my, I mean honestly, I’ve got a Mac and I’ve got a PC sitting over here as well. Do I put it on my Mac? Do I put it on my pc? Where do I put this thing? 

Joyce Bradford (10:18): 

You’re going to put this on your primary machine. You want this to be able to track what you’re doing most efficiently. Where are you building your documents? Where are you responding to your emails? Where are you spending your time? Not your social time where you’re corresponding with your friends on your Reddit account, whatever you have. And that’s not a dig on you, Zack. I have Reddit too. I’m not saying you’re a standard redditer here. 

Zack Glaser (10:40): 

I’m floating around there. Absolutely, yes. 

Joyce Bradford (10:43): 

But now put it where you’re going to do the majority of your client work, especially if you’ve got A-V-O-I-P. So download it on the machine where you have your VO OIP where you’re answering those phone calls. 

Zack Glaser (10:53): 

Fantastic. And frankly, if you don’t have VOIP, this may be a good reason to switch to that sort of integrated system, especially if you’re a Rocket Matter user, because Rocket Matter is a very, very comprehensive system that likes to track it, likes to get information, it likes to get stuff that you can run reports on. Well, Joyce, so where can people learn more about Rocket Matter itself and Rocket Matter track? 

Joyce Bradford (11:19): 

Yeah, go to Rocket matter.com. That is the best place. We’ve got information on Rocket Matter track on the very front page. You don’t have to go searching for it. And if you want to learn more about Rocket Matter and some of the tools that we integrate with like A CRM e-signature tools to give you more e-signature options, more third party storage solutions, go to our parent company website, profit solve.com, and there’s no E on the end of Profit Solve. So just note, 

Zack Glaser (11:45): 

Yes, kind of like time solve. There’s no E there as well. Well, Joyce, as always, I really appreciate you being with me. Thanks for taking the time to share your expertise. 

Joyce Bradford (11:53): 

My pleasure. Thanks Zack. 

Ben Schorr (11:58): 

Hi, I’m Ben, and I am a senior content project manager at Microsoft and a lot of people at Microsoft. I work on copilot pretty much all the time, and I’m super excited to get to talk with attorneys about how it can help ’em and how they can best take advantage. A little bit about me, I guess, been in legal tech since the late 1980s, including in-house at law firms for about 10 years firms in Los Angeles and Honolulu, which was a great experience. I spent most of the time in between as a consultant, mostly working with law firms, although not entirely written a bunch of books. If you’re on video, you can see behind me, you can 

Zack Glaser (12:31): 

See in the background. Yeah, 

Ben Schorr (12:32): 

If you’re just listening, then you’ll just have to take my word for it. There’s lots of amazing books behind me that the ABA has undoubtedly been spamming you to buy for years, and my kid’s college fund, thanks you. Yeah. So I’m just excited to be here with you. So nice to meet you, Zack. 

Zack Glaser (12:46): 

Ben, nice to meet you as well. I have been following you on Twitter and LinkedIn and various social media platforms for a while, and we will put a link to at least one of the books that you have behind there in the show notes. But for those of you listening on the podcast, there are many, Ben is a wealth of information related to legal technology, but today we’re digging into AI because again, like you said, you’re over there in Microsoft working on copilot, and so I can go ask people how they use copilot all the time. I can ask attorneys what they think of it, but let’s get to kind of the source here. And I figured you’d be a great place to start with how lawyers can be using copilot or things like copilot. We will stay somewhat agnostic, but how they can be using AI in their practices and be preparing to get started with that. 

Ben Schorr (13:40): 

Sure. So it’s interesting because a lot of firms, a lot of attorneys I’ve talked to who are using Copilot album just got excited and jumped right in. The point about preparing is an excellent one, and it’s one I don’t, surprisingly a few people ask me about. There are some things you should do to prepare before you use copilot or any generative AI in your firm. For one thing, you should make sure your information is in the right place. One of the great things about copilot and most generative ai, I think, but copilot specifically, is that it respects your Microsoft 365 permissions and it’s constrained in your Microsoft 365 tenant. By that I mean that if you’ve got a document on your C drive on your local hard drive, copilot is not going to find that right? If you’ve got documents in your SharePoint, in OneDrive, it will. 

(14:20): 

Now, the other thing about copilot, which is really important, we get this question. It’s one of the most common questions attorneys ask me is that copilot is constrained by the user’s permissions. And so that means that if Alice and Bob are in a law firm and Bob tries to ask copilot a question about one of Alice’s documents, but Alice hasn’t shared that document with Bob, copilots not going to tell Bob anything about the document because copilot recognizes that Bob doesn’t have permissions to that document. It’s not even going to tell Bob that document exists. So it’s very much the same as what you have now in terms of if Bob just goes in and starts searching for Alice’s, he’s only going to see the documents he has permissions to. And the same thing’s true with copilot. Copilot only sees the documents the user has permissions to. 

(14:58): 

So Alice can ask copilot questions about that document. It’s her document, but Bob can’t unless Alice shares it with him. But the problem here is that we get a lot of firms that have been pretty sloppy with their permissions, and so they’ve thrown a lot of sensitive documents. Yes, I know they’ve thrown a lot of sensitive documents into SharePoint sites or into OneDrive file libraries that have been shared overly broadly, shall we say. And so we are seeing instances, and this didn’t start with copilot. These users always had access to these files. They just didn’t realize it before. copilot just made it easier for them to find those files. So it’s a pretty good idea to do a little self-audit before you deploy something like copilot to just make sure that your permissions are what they’re supposed to be and that you’re not sharing things with everybody. 

(15:38): 

Many years ago, I went into a firm, they called me in to help them with some things. We did some security auditing for them. One of the things we discovered was that a few years earlier, they had given their receptionist permissions to the HR system to enter people’s birth dates so they could do birthday cards and things like that for birthdays, but they were sloppy about it. And what they’d done is they had given the receptionist admin permission to the HR system. So she had access to everything in the admin system, and they’d never taken it away either. She could look up compensation reports, she could look up disciplinary actions, she could look up anything. She could change things in their HR system if she wanted to. Now, to be fair, we had no evidence she’d ever done any of that. She absolutely could have. 

(16:17): 

And that was just one of those things where we looked at it and we’re like, I’m pretty sure you didn’t want her to be able to look up everybody’s salary or be able to change everybody’s salary even. So we’ve seen a lot of firms that they didn’t take the time to do it. They didn’t know that they shouldn’t do it. Maybe they had somebody in IT who wasn’t diligent, whatever reason. Firms haven’t always been as diligent about their internal security as maybe they should have been. So it’s a really good idea before you deploy any kind of system like copilot, that you just take a quick look around and make sure everything’s where it’s supposed to be and that everything is only visible to the people who should be able to see it. 

Zack Glaser (16:50): 

Right? And that’s something that your IT department or whoever is managing your Microsoft account should be able to do or should be able to at least audit and look into. Because one of the things about Microsoft specifically is that there’s pretty granular permissions, and I think it confuses people sometimes. So to be fair, sharing documents really confuses people at times because you can share a document in multiple different ways. You can share a link to a document, and that lets you create very, very granular access. But if I give my access link to this document to somebody else, that doesn’t necessarily mean that they’re going to be able to use it. But I think on a higher level, just understanding the ideas of these permissions and understanding how all of this access goes. One of the things that I have talked to people about a lot of times is that artificial, these tools, when we get in trouble with them, they’re just exacerbating the trouble we’re already in. 

Ben Schorr (17:56): 

Yeah, definitely. And that’s the thing with those permissions, like I said, those people always had access to those files. It’s just the copilots just made it more obvious. 

Zack Glaser (18:04): 

So underlying this kind of getting your information into the right place, kind of want to stay on that a little bit of if I have stuff, if I have data, if I have information I have, and let’s talk about people that are already digital. I’m not going to talk about, we’ve got paper files sitting in the back file room like we had when I first started practicing with my father, but we’ve got our documents, we’ve got our data in a practice management system, then we’ve got our data in a client relationship management system. Then we’ve got information in Google Contacts, and then we’ve got information in SharePoint and OneDrive and Dropbox and all of this. And that’s not accessible at least naturally by something like copilot just right out of the Microsoft environment, right? 

Ben Schorr (18:56): 

That’s correct. Now, some of those systems are creating copilot add-ins and plugins. For example, when we were a tech show, I talked to, I think the net documents rep said they’re working on a copilot plugin, a law toolbox has one, and other vendors are working on those too. So if they have a copilot plugin and you’ve got that installed, then the data in those systems might be accessible to copilot. But natively, no copilots only going to see what’s in your Microsoft 365 tenant. So that’s your exchange server, that’s your OneDrive, that’s your SharePoint systems like that in your Microsoft 365. But things that you have in a client management system or practice management system, document management system, any of those third party locations, copilot won’t be able to see those unless you’ve got some sort of plugin installed. 

Zack Glaser (19:43): 

So kind of prepping for being able to use something like copilot, and I’m using copilot as an example, obviously because I’m talking with you, but also because it is one of the, I mean, it kind of sucks the air out of the room. It is the foremost generative AI product out there as far as I can tell, but it’s also very representative of something that is integrated into our lives, into our work lives and can touch a lot of things. You go onto something like chat, GPT, and this is looking at the internet essentially in one point of time, depending on what you’re using and things like that. But copilot being built into the Microsoft environment is able to not only look at those things, but also look at your own internal data. 

Ben Schorr (20:30): 

In fact, I use it pretty often for, because as you might suspect, I get a lot of email, I get a lot of teams messages, and one of the things I did yesterday with it, and that I do a couple times a week usually is I ask co-pilot to summarize my last 10 emails. I have a more involved prompt than that that I give it until I get a more detailed response. But even just summarize, my last 10 emails would work. I just want a little bit more. And so that’s a great example of it can help me very quickly catch up. I’ve sometimes asked it with varying degrees of success to summarize my most important unread emails so that it gives me, it catches me up on stuff that I’ve made have missed that might be important. Now, unfortunately, co-pilot’s idea of what’s important isn’t always my idea of what’s important. 

(21:10): 

It does a pretty good job with that, but it doesn’t, it’s not perfect. I mean, it’s an ai. So I often tell people to think about copilot or any AI you’re using. Think of it like a first year legal intern. Give it very clear instructions and then review what it gives you. And Jared, one of our vice presidents famously said that copilot will often be right and will occasionally be usefully wrong. And I love that quote because that’s pretty much right. Correct. You can ask it to look through your data. You can ask it to look through your emails, your team’s chats. A lot of the time. It’s going to give you very useful information every now and then. It’s not going to give you exactly what you wanted, but it may point you in a direction that’s useful and gets you moving in the right way. So I really like that quote because I think it sets a good expectation. 

Zack Glaser (21:54): 

I do too. And what it also kind of says is that you really do want to know when it’s usefully wrong. You want to be looking at it, you want to be checking its 

Ben Schorr (22:03): 

Work. Hundred percent. 

Zack Glaser (22:05): 

Because it’s a tool. 

Ben Schorr (22:06): 

Yeah. Oh yeah. Never ever show the output of a generative AI to a client or the court without reviewing it first. Never. I mean, we’ve had so many public examples now of attorneys who’ve just done something stupid and let a generative AI create something that they then sent to the court unreviewed and with catastrophic results. So please don’t do that. It’s a copilot, not an autopilot. 

Zack Glaser (22:30): 

I like that. That’s a really good point. It is a copilot, not an autopilot. I kind of go back to the, it can exacerbate laziness. If you’re not going to review citations, if you’re not going to review the evidence that you’re having it look at, if you weren’t going to review it in the first place, if you weren’t going to review a first year associates work, then you’re going to get messed up pretty quickly. 

Ben Schorr (22:51): 

Definitely. Yeah. And we’ve seen it happen too many times already. But as Brad Smith, our company president says, we named it co-pilot for a reason. And it’s there to work alongside you, not instead of you. 

Zack Glaser (23:01): 

Right. Well, so one of the things I think that lawyers are thinking about right now is the idea of leaking our information. So if I’m putting all my eggs in one basket, if I’m putting all my data into something like SharePoint, which for the most part kind of high level you need to do in order to be able to have copilot act upon it, unless you have add-ons and things like that, how do I keep my information from then going out into the broader kind of chat GPT land that is this ethereal sort of wild west of information? 

Ben Schorr (23:38): 

Sure. So that’s another question I get a lot. It’s important to understand that copilot keeps the customer’s content that lives in their Microsoft 365 tenant. We refer to that content as the graph. So you may hear that term occasionally. The graph is sort of our sort of catchall term. It means means all that the SharePoint OneDrive exchange, all of your data that lives inside your Microsoft 365 tenant. We refer to that collectively as the graph. And so the customer’s data in their graph is kept separate from the LLM, from the large language model that copilot is trained on. And so copilot is querying both of those things. It’s querying the graph and it’s querying the LLM, but the graph data never goes to the LLM. We do not train the LLM on the customer’s data ever. All of the customer’s data, all of the customer’s queries, all of that information stays in the customer’s Microsoft 365 tenant. It’s still constrained by the same privacy policies we’ve always had for Microsoft 365. So we’re not training the LM on customer data. That doesn’t happen. So hopefully that reassures some folks. And of course that means each customer, because each customer’s graph is unique. Your set of data is different from their set of data, is different from their set of data. So that means that when you run a query and some other customer runs a query, the results will usually be different because your copilot is querying your graph. The other customer’s query is querying their graph. So they get very different results. 

Zack Glaser (24:58): 

And I think that’s important to me because that starts to make our data, our internal information for our firm feel kind of like property and crap in crap out, but also good information in good information out. So if you as a firm are starting to compile a lot of, let’s say, very good contracts or very good playbooks or things like that, well now your data, your property has value because it’s going to be better than the firm next to you. 

Ben Schorr (25:31): 

Yep, that’s right. Because copilot will use your data for you and their data for them. If your data’s better, then you get better results, hopefully. I mean, that’s the theory, 

Zack Glaser (25:39): 

Right? Yes, yes, yes. And I hadn’t really thought about that because attorneys have trouble selling their firms because we have trouble with what is the thing that we own. And frankly, it’s infrastructure. It’s always been the books in the library, the building, the things like that. And now I think we can throw in the data behind all this, and attorneys as they start to create groups of attorneys and create separate businesses, probably need to have data agreements related to that as well. Sorry, that’s just kind of a weird tangent. So one of the things that I noticed the other day though, I was playing around with copilot because we have an instance here, and I was looking at add-ins and looking at a chat GPT. And as I’m going through this thing, because Microsoft is straightforward about where information is going, it starts to ask me if this add-in can have access to my internal data. And if somebody’s using a party add-in, let’s say it’s chat GPT for dummies or something like that, that isn’t necessarily constrained by the graph in one place, LLM in another place model. 

Ben Schorr (26:58): 

So that would probably depend on the add-in not familiar with that particular one as far as what it’s doing. It’s always a good rule that anytime you let any third party software have access to your data, that you should very carefully review their privacy policies and see what they’re planning to do. That’s not an area I have a lot of direct experience with in terms of that specific ad in though. 

Zack Glaser (27:18): 

And I just want to clarify, I’m actually not talking about a specific add-in here, but I wanted to harp on that point of if you’re using a third party copilot is one thing. It exists inside the system and doesn’t need to be put in each thing as an add-in. So if you’re in Excel and you have copilot turned on, it’s there. You don’t have to add it. And so if you’re looking at an ad in and it says that it’s chat GPT or something like that, you need to vet that thing. 

Ben Schorr (27:51): 

Oh, a hundred percent, yes. Yeah, that’s true. And I used to work before I moved to copilot, I worked to work in security and we used to try to feud, I mean my dad try to encourage people to, same thing with anytime you install a new app on your network, on your device, on your phone, and I mean if it’s just play Tetris game, maybe it doesn’t matter as much, but anything that’s going to touch your company data, you should a hundred percent be familiar with their privacy policies. A hundred percent have read the reviews, hopefully talked to some other people who’ve used it, really do your due diligence to make sure that this thing is going to do what you expect it to do and not more with your data. And that goes for us too. I mean, please read our privacy public microsoft.com/privacy. Please read our privacy policies too. We absolutely encourage that. So yeah, a hundred percent on those. Add-ins, check their privacy policies, make sure you understand what they’re going to do, what they’re going to do with your data and that you’re comfortable with that. 

Zack Glaser (28:44): 

Some privacy attorney sitting somewhere has spent, well, many, many privacy attorneys sitting somewhere, have spent a lot of time creating that privacy policy. I get pushback on that sometimes related to privacy policies and all that. And the attorneys say, how am I supposed to know this? And frankly, if you don’t as an attorney, if you’re not going to read the privacy policy, who is 

(29:09): 

Just read it. I know it takes some time. A lot of times they’re big and long and you probably don’t want to just ask copilot to summarize it, but go read the thing. And if you don’t understand something, be a lawyer, go look things up, ask a privacy attorney. I don’t know. So on that front, if there are specific kind of things that we would want to ask as attorneys want to think about high level with security, what is it we want to be kind of asking of our AI products and thinking about in terms of security? 

Ben Schorr (29:47): 

I mean, I think an AI product is not that much different from other software in that regard. I think it’s important to understand what data it’s going to access, what it’s going to do with that data, where it’s going to store that data if it stores that data, which it might not. I think these are the fundamentals that we get into with any piece of software. When you’re talking about security, the company behind it is important. Obviously I trust Microsoft, but certainly to not be too provincial about it. And most of the big tech companies I’m a little bit more confident in than Bob’s software. Not to disparate Bob, I’m sure he is a great guy, might be fine. Does he have a level of protection that most of the big tech companies have? I mean, I can’t speak for our colleagues at Google and Alphabet and other places, but I’m sure they have giant teams of security professionals on the job all day just like we do. 

(30:36): 

And Bob can’t afford that. Does that mean he is not secure? No, it doesn’t mean he’s not secure. But just saying all us being equal, I tend to have a little bit more confidence in the people with the resources. So I mean, that’s my interpret. Other people can take different impressions. And of course, keep in mind I work at Microsoft, so I may be slightly biased. So yeah, I think from a security standpoint, it’s important. You’ve got to do that due diligence. I see so many companies get themselves in a mess. The other issues that I see with security, and this applies to ai, but it also would apply to other things. Keeping your software up to date is important. With ai, that’s not as big an issue right now because most of the AI is run from a central location on a cloud server. 

(31:16): 

So it’s generally kept up to date for you, but you still want to keep your updates installed because there’s oftentimes security updates in there. So that would be another one. And then also, when we touched on this earlier, so much of it comes down to how things are set up. You could go buy all the security things there are in the world, and if you don’t set ’em up properly, if you don’t manage ’em properly, if you don’t maintain ’em properly, they’re not going to do you any good. I’ve told this story before in security webinars about, I went to visit a firm once. They brought me in to do some stuff for ’em. They had the entire floor of a building and in the center of the floor, it was a very large, very fancy, very elaborate conference room, which they referred to as the war room. 

(31:55): 

And it was the conference room that they used for all the very sensitive things when they had big trials going on, if they had important confidential negotiations going on, things like that. They used this conference room for that and it had windows in it. It didn’t look to the outside world, but looked to the hallways that kind of surrounded the room. And they had very intelligently installed blinds on these windows because they didn’t want people to be able to just look in and see what was going on in their super sensitive war room. But I noticed as I was walking around the floor that on one of the windows, the blinds had been installed on the outside of the window, which meant that anybody in the hallway could just lift the blind and look inside. And I thought this is an example of a firm that has gone out and spent the money and had a device installed for privacy, but it wasn’t installed properly, 

(32:41): 

And so it didn’t do what it was supposed to do. So the same thing is true in software. If you’ve installed your security software review, if you’ve configured it incorrectly, it’s not going to help you. So the same thing would be true, any software including AI software, and this kind goes back to what we said at the beginning is if you’ve thrown a bunch of sensitive documents in a SharePoint folder and set the permissions to all your security is a problem, you’ve just exposed that to everybody in your firm, hopefully only people in your firm and not people in the outside world. 

Zack Glaser (33:10): 

What I think of, a lot of times I just try to take that step a little bit because people like to trust the people that are in their office. So they think, okay, well, I gave our receptionist permissions for everything. Well, they’re trustworthy, they’re fine. Well, what if somebody gets their user permissions? What if they get phished and somebody winds up getting their access? And that’s a little bit easier way, I think, for attorneys to think about, okay, well yes, I trust my receptionist, but they may not be extremely good with their passwords. 

Ben Schorr (33:44): 

And the other end is that this particular firm that I had referred to before, they had assigned the permission not to the person who was the receptionist, but to the role of receptionist, which meant that anybody who sat down and signed in as a receptionist had that permission when their receptionist went to lunch or was out sick or took a vacation. They had other people sitting at that desk signed in as a receptionist, most of whom were employees, but every now and then they’d have a temp. 

Zack Glaser (34:08): 

Yeah, and that’s scary, at least to me, and it should be to you people out there on the airwaves that should be scaring everybody. 

Ben Schorr (34:18): 

That’s my job, scare people. No, that’s not my job 

Zack Glaser (34:22): 

To keep people safe here. So, okay, so we’ve kind of gone into how we need to be thinking about preparing ourselves for doing this, and you’ve talked about some of the ways that you’re using AI in your daily life. Do you have any other examples of how you’re using it internally to help you just kind of move forward or ease your day? 

Ben Schorr (34:45): 

Yeah, definitely. So I think of copilot specifically, but generative AI generally, there’s sort of three big buckets, and there’s probably others, but there’s three that come to mind for me of areas where it gets used. Some of ’em are obvious, like create and edit. That’s a big one. You can ask copilot to create stuff for you to edit stuff for you. Again, please, please review it before you share it with anybody. I used it not long ago on the side, I maintain a hobby blog and I had copilot write a blog post for me, and it wrote about six paragraphs, which was about what I wanted, and I looked through it and three or four of the paragraphs were good, so I was okay with those. A couple of the paragraphs needed a little bit of work. So then I also ended up adding another paragraph of my own. I removed a little bit of what copilot put in and people would say, oh, you had to do all that work. Oh, I guess copilot wasn’t that good. No, actually it took a task that in the past would’ve taken me 45 minutes or an hour and cut it down to about 15 minutes. I mean, that’s pretty good. 

Zack Glaser (35:41): 

That’s significant time in the day. 

Ben Schorr (35:44): 

And at the end of the day, probably half the blog post was written by Coot. About half of it was probably my own, but that definitely saved me some time. I frequently, when I’m writing things for work will take a paragraph and I’ll ask copilot suggest ways I could make this clearer. And sometimes I like it, suggestions, sometimes I don’t. But if it gives me a suggestion, that’s useful. Great, that’s very helpful. I’ve talked to an attorney who said that he’s using it now in a similar way where instead of clearer, well, he sometimes uses clearer, but he often uses more persuasive, how could I rewrite this to be more persuasive? And copilot makes some suggestions and says, you could try this, try this, try this. And same thing, same results. He says. A lot of the times it makes good suggestions sometimes that the suggestion isn’t helpful, but usually it is, and that’s great. 

(36:28): 

That’s good use of it. Another really powerful use, and probably the one that I think attorneys are going to lean into even more than create edit is ask and understand. I frequently get these long documents, these long spec documents at work, very technical documents. Some of ’em can be 50, 60 pages long, and I sometimes have to figure those out pretty quickly. I have a meeting in an hour to talk about this topic, and so I can open that topic in Word and I can tell Word, summarize this document, and it will, it’ll give me a few paragraphs explaining what the document’s about in regular plain English, I can ask it questions, what are the key dates and deadlines in this document? What people are mentioned in, what are their roles? There’s a whole bunch of questions. You can have a whole conversation about the document with copilot that helps you understand it. 

(37:15): 

Sometimes I’ll be reading a document and there’ll be something in a paragraph that I’m not quite sure I understand. And so I can ask copilot to explain that concept to me in simple language, and that could be very, very helpful. Microsoft is famous for R TLAs, our three letter acronyms, and so sometimes an acronym comes up in a teams meeting or in a document or in somewhere else. I mean, this isn’t just word of course, and so I can ask co, what’s this mean? One time I had it come back to me and give me a response that I knew wasn’t right. It mean it was correct in that that was one definition of that TLA, but I knew that wasn’t the one that person was going for. So I rephrased my prompt and said, in the context of productivity software, what does this TLA mean? 

(37:56): 

And it came back with the correct answer, which was great. The other place, and this is what I use quite often myself, is I get invited to a lot of meetings. Some of those meetings I don’t actually need to be at. It’s an hour long. Seven minutes of it applies to me, and I’m pretty sure nobody’s going to ask me any questions directly in that meeting. It’s a large meeting. And so what I can do is what I’ll sometimes do is I won’t go to that meeting. I often have multiple meetings scheduled at the same time. I may go to a different meeting. What I can do is after the meeting, I can go into teams and open copilot and say, summarize that meeting. Give me a few paragraphs on what happened in the meeting. I can then ask it because the product I work on is called copilot lab. 

(38:34): 

I can say was copilot lab mentioned? And if so, what was said? And it’ll tell me Sometimes yes, sometimes no. Were there any action items for me? Sometimes yes, sometimes no. So again, I can have a conversation about that meeting, even if I attended the meeting. Sometimes it’s helpful to have that conversation with copilot after the meeting to see if I missed anything. So that ask and understand capability is really powerful. And I should mention also that Coot does have the ability to also summarize PDFs. So if you’ve received a PDF from somebody and you want to get a summary of it, you can do that. Okay. And then the last section that I think of, last big bucket I think of is brainstorm. So we had a project the other day, I had to think of a title for a project we’re working on the other day. 

(39:10): 

And I thought of a couple, and I went into copilot and I opened, actually, I opened copilot in OneNote and I said, suggest 10 possible titles for A, and then I described the project a little bit and it came back with 10 ideas. Eight of those ideas I didn’t really like, but two of them were pretty interesting. And so those are kind of on the possible list. And so being able to just brainstorm things, be to kick things around, what do you think about this? What are some possible counter arguments to that? What are the pros and cons of this idea? Just being able to feed those kinds of ideas into the AI and have it give you a usually pretty interesting response can be super helpful. And it’s available 24 hours a day. You don’t necessarily, you wake up at three in the morning with that crazy idea. You don’t necessarily want to wake up your buddy and ask him. You can ask copilot, co-pilot’s not judging you. It’s just a machine. 

Zack Glaser (39:58): 

There’s 

Ben Schorr (39:58): 

Nobody sitting on the other end going, that’s a stupid idea right 

Zack Glaser (40:02): 

Now I’ve got this idea of judgy, copilot. 

Ben Schorr (40:04): 

It’s just a machine and it’s happy to talk as long as you want. Doesn’t get tired. It doesn’t get bored. 

Zack Glaser (40:10): 

And provided you have copilot turned on and all that, it doesn’t cost anything extra. 

Ben Schorr (40:16): 

Well, there is a subscription for it. 

Zack Glaser (40:19): 

Yes, yes. But provided you have the subscription to copilot, asking it this added question doesn’t cost anything extra. 

Ben Schorr (40:25): 

Yes, that’s exactly right. You don’t get charged incrementally. Yeah. Although I should point out, we do have a free copilot available at Bing. Now, the Bing copilot doesn’t have access to any of your files or data. Of course, thankfully. So all of that sort of stuff, it’s not going to be able to do. I do use the free Bing co-pilot for a couple of things. One thing I do with it sometimes is, again, the summarize this document. If it’s a publicly available file, PDF, an article, a blog post, something like that, that I want to get a summary of, then the free Bing copilot can be very helpful for summarize this page, summarize this article, or asking questions of that article. Again, I’m not going to use that on confidential work product, and it doesn’t have access to my OneDrive or anything, but if it’s a publicly available file, then that’s okay. The other thing I do with it, which I think is kind of interesting, especially in our current world, is you can ask it to fact check things. So if you see a statement somewhere on social media or whatever, and you’re not sure if it’s true, and especially if you think it is true, it can be sometimes handy to just copy and paste that and say, fact check this statement, and Bing will look at it and tell you if it thinks it’s true or false, and it’ll give you citations of where it got its information. 

Zack Glaser (41:30): 

I think the citations is an important aspect of that because it is. Yeah, it’s fact. Check it, and then it’s how I got to this, because if you’re just using AI as just kind of a black box where you just put stuff in and you trust what’s coming out, you might get in trouble 

Ben Schorr (41:46): 

A hundred percent. Yeah, definitely helpful to have those citations. And I should point out, by the way, copilot in Microsoft 365 will also give you citations, but if you’ve asked it for information inside your firm and it’s given you a response, if it can, it’ll try to tell you what documents, what SharePoint sites, what emails or teams meetings it retrieved that information from, and so that can be very helpful also, especially when you’re not sure if it’s being usefully wrong, being able to go back and look at the source material is very helpful. 

Zack Glaser (42:12): 

Right, right. That is very helpful. Well, Ben, we’re kind of coming up to the end of time. This has been really helpful for me. I appreciate your knowledge and your help with kind of sussing out some of the thoughts about copilot and how attorneys can get started. 

Ben Schorr (42:28): 

Yeah, thank you. Thanks for having me. Hopefully your audience found that useful. 

Zack Glaser (42:32): 

Well, if they didn’t, I did, and a lot of times that’s all I’m looking for. I’m just kidding. But Ben, thank you again. I really appreciate your time. We’re going to drop some of your books into the show notes and any of the links that we have that we talked about here, so thank you. 

Ben Schorr (42:51): 

You’re welcome. Thank you. 

Announcer: 

The Lawyerist Podcast is edited by Britany Felix. Are you ready to implement the ideas we discuss here into your practice? Wondering what to do next? Here are your first two steps. First. If you haven’t read The Small Firm Roadmap yet, grab the first chapter for free at Lawyerist.com/book. Looking for help beyond the book? Let’s chat about whether our coaching communities, are right for you. Head to Lawyerist.com/community/lab to schedule a 10-minute call with our team to learn more. The views expressed by the participants are their own and are not endorsed by Legal Talk Network. Nothing said in this podcast is legal advice for you.