Managed Review vs. Unmanaged Review: Which One’s Right For You?

Complex litigation cannot happen without some form of document review, whether that’s managed review or not. Document review cannot happen without reviewers. This raises a lot of questions that most lawyers never learned about in law school: Which reviewers do you hire? What parts of review can be trusted to technology, and what parts absolutely have to be done by humans? What work requires actual attorneys, and what work is better left to other litigation support professionals? 

There’s a myriad of considerations that go into these decisions depending on the case at hand and the capabilities of that particular legal team. One of those decisions is “who should make all the other decisions?” When legal teams are either spread too thin or simply want someone with a different realm of expertise, managed review services can be great for attorneys and their clients.

What eDiscovery Review Teams Do vs. What Lead Attorneys Do

Lawyers are good at a lot of things. They tend to be good researchers, and good at drawing connections between seemingly unconnected pieces of evidence. Unfortunately, those relevant pieces of evidence don’t just show up at a lawyer’s doorstep all wrapped up with a bow. Usually, they’re hidden somewhere in a massive pile of data.

If your legal case is a jigsaw puzzle, most lawyers could probably put that puzzle together just fine if there are relatively few pieces and they all come in the same box. Now, imagine the pieces of the puzzle that you want to make are in a box with thousands of other pieces that make up less relevant pictures. What happens when you’re not even 100% sure you have all the right pieces to make the picture you want to make? Can you find other pieces along the way that might make other helpful pictures?

It would be impossible for any one person to make their chosen puzzle in such a scenario. Instead, you need a coordinated team of people going through documents and coding them as relevant. These reviewers aren’t usually the ones making calls about big picture legal strategy, but they know what kinds of puzzle pieces to look for so that those other attorneys can actually put a picture together.

Finding the right pieces in that unruly pile is a separate skillset from actually putting the pieces together once you have them. Plenty of lawyers have great experience in both these areas, but it’s not a given, and there’s no shame in asking for outside help when you need it.

How Remote Review Services Can Help

Remote review and managed review services allow for legal teams to scale as needed in order to take on larger, review-heavy matters. Smaller teams who don’t have enough man power internally can staff up for one matter, and then scale back down afterwards.

Document review services aren’t just for when you need more reviewers, but sometimes simply different reviewers. Maybe you have documents in a foreign language that no one on your staff speaks; maybe you have a matter outside your normal realm of expertise, and need reviewers with different legal specialties.

Either way, remote review lets you take better care of your clients without having internal hires for every possible scenario that your clients might throw at you (which isn’t realistic for most law firms.)

Remote review is a great option for most law firms, but it comes with challenges of its own. Someone has to decide which reviewers are best suited for your matter. Someone has to decide how many reviewers you need to meet a deadline; those reviewers have to report to someone on a day-to-day basis, and that person needs to have an intimate understanding of the whole case, your overarching legal strategy, and whatever technology you’re using to assist human reviewers. Even after you agree on answers to all those questions, someone should be reevaluating things on a daily basis as new information comes to light. Of course, all these challenges multiply if you’re leveraging remote review for multiple cases.

So that raises a follow-up question…. Who should be that someone?

How Managed Review Can Help

In an unmanaged remote review situation, a client is still on the hook to answer questions from reviewers as they come up; the client needs to clearly communicate to reviewers what they’re looking for, and make sure all these different reviewers are taking a consistent approach to coding documents.

Unmanaged review does cost less money, but there can also be dire consequences if this point person is already spread thin and not able to give review management enough attention. What happens if an attorney gets caught up in court and can’t answer reviewers’ questions? What if they don’t notice that more reviewers are needed until it’s too late? What if they aren’t assessing progress often enough, and don’t realize a pivot in legal strategy is necessary until review is 99% done and there’s still no “smoking gun”?

This is why the biggest law firms that are dealing with complex litigation on a regular basis typically hire dedicated discovery project managers apart from their regular attorneys. They know that discovery can require near constant attention, and oftentimes it’s just not possible to give it that attention while dealing with all the other demands of being a lawyer: writing briefs, going to client meetings, court dates, etc. Such firms also know mismanaging discovery can have dramatic impacts downstream. Therefore, investing in constant vigilance at every step of the process will pay dividends later.

Managed review means you are not only temporarily hiring reviewers, but someone to manage them. Clients get to meet with one point person, describe their legal strategy and what they’re hoping to find in review. That point person turns around and handles all the other emails and meetings with the review team. Project managers can catch issues early on and bring them to the client’s attention.

How Do I Decide Which One Is Right For Me?

There’s different reasons someone might go the managed review route over unmanaged route. One might be that they simply don’t have attorneys with that discovery project management skillset. With managed review, a firm that cannot justify hiring discovery PMs internally can still have the same level of vigilance as a firm that could.

Another reason is that even if attorneys are perfectly capable of managing review themselves, their current caseload just doesn’t allow them to give it the attention it needs. By letting someone else answer more of the emails and sit in on more of the meetings, that attorney can focus on other things that a discovery PM couldn’t do, such as deposing witnesses or writing briefs.

Of course it’s going to depend on many factors specific to your case which can’t be addressed here, but generally the key factors that should shape your decision are:  

  1. Do you have discovery project managers internally? These could be either dedicated personnel or attorneys who have experience managing review.

  2. Do those people actually have enough room on their plate now to take on the responsibilities of managing this review?

If an unmanaged review situation is likely to result in either a) an attorney not having enough time to give all their matters the attention they deserve or b) review being managed by someone who’s never done it before and may not understand the intricacies of it, the managed review route is often best.

6 Document Review Metrics Every Lawyer Should Know

Document review can oftentimes be one of the most cumbersome parts of discovery. Separating the relevant from the irrelevant in a timely manner not only requires humans with specialized legal expertise, but someone at the helm who can keep track of it all. Whether you’re managing review internally or paying for outside managed review services, there are certain numbers regarding a matter’s progress that the attorney in charge should know at any given time.

1. How many documents do I have?

Yes, this seems pretty obvious but it’s still worth mentioning. The total number of documents in discovery is the metric that all other metrics are measured by. Oftentimes, countless other decisions stem from this number. How much will discovery cost? Should I settle because discovery is too expensive? How many reviewers need to be on this project to meet our deadline? There’s plenty of other variables that come into such equations, but there’s virtually no decision where the total number of documents isn’t part of that equation.

2. How many dupes do I have?

As important as the total number of documents is, it can’t be the only number you look at. That’s largely because it can sometimes be misleading thanks to dupes and near dupes.

A near-dupe dashboard within Contact Discovery’s Vu™ solution

Imagine we collect the phones belonging to both Jack and Jill. If Jack and Jill ever had correspondence with each other, then it’s likely those same threads exist on both devices. It’s a waste of resources to make reviewers read that twice.

Luckily, most eDiscovery platforms have gotten pretty good at recognizing dupes. In some cases, de-duping can make your pile of documents significantly smaller than what you initially thought. Since so many other strategic decisions will hinge on how costly review could be, you need to know this dupe number to have an accurate read on the scope of review.

3. What kinds of documents do I have?

Long gone are the days where discovery strategies were limited to emails and their attachments. Well at least, long gone are the days when good discovery strategies were limited to emails and their attachments.

Nowadays, “documents” can take the form of emails, text messages, Slack threads, and more. Those different communication channels can each pose different review challenges that hinder review progress if you’re unprepared.

For example, reviewing text messages can involve spreadsheets where iChat, SMS, and MMS are broken into different pages, and any attached images are another page. Reviewing communication this way without mobile-specific solutions to help can be significantly more time-consuming than reviewing the exact same conversation in email form.

Now, imagine 60% of your documents that need reviewing are text messages, but you’ve budgeted your time and money as though they were emails. This will cause major problems downstream when review progress isn’t happening at the pace you expected, and perhaps you need to hire a lot of extra reviewers at the last minute to meet your deadline. Your client is mad because the case is costing more than they initially thought. This could’ve been prevented if you’d known what kinds of data you had at the onset and how long it generally takes to review different data types.

4. How many documents are in other languages?

Documents in foreign languages are another curveball that can trip up large review projects. It’s hard enough to find attorneys with the right legal experience to be helpful to your case, but finding that attorney who’s also fluent in another language often proves even more difficult. Any precious time you spend tracking down qualified reviewers is time you could’ve spent reviewing documents.

Maybe non-English documents only make up a tiny portion of your total data, in which case you only need 1-2 reviewers who speak that other language, and the rest of your review team can review the English documents as usual. In other cases, you might need a team composed almost entirely of bilingual attorneys.

It’s also highly dependent on what languages you need your team to know. Reviewers fluent in Spanish are probably going to be easier to find than reviewers fluent in Bulgarian. Either way, the sooner you figure out that you have documents in other languages, the better you can plan for that added challenge and manage your client’s expectations.

5. How many documents still need to be reviewed?

This is another one that might seem obvious, but it’s not necessarily important for the reasons you might think. “Can I get through X documents by my deadline?” is a pressing question on any lawyer’s mind, but an equally important question is “do I have enough information to make good decisions about what to do next?”

The Master Summary screen within Contact Discovery’s Vu™ solution

Ultimately, the reason we do discovery in the first place is so that lawyers and their clients can reach a positive outcome. What lawyers consider “positive” can vary widely depending on the truth that lies in those discovery documents. Sometimes there’s enough exonerating evidence to win a trial; other times, there’s enough incriminating data that a “positive outcome” is a favorable settlement.

If your team has only gotten through 10% of their documents, it’s probably not wise to make any major decision regarding the case. You just don’t have enough information yet to make a good decision, and you don’t want to close yourself off to other potential strategies that might become evident later.

If you’ve gotten through 80% of the documents, you still don’t have ALL the information, but you might have enough to be a little more strategic about how you approach that last 20%. What other information would be helpful to the case? Can you make educated guesses about which documents might hold that information?

Knowing how many documents still need to be reviewed is a lot bigger than just “Am I on track to meet a deadline?” It’s a number that tells you whether it makes more sense to start building a specific case strategy, or more sense to hang tight and wait for more information before you commit to a strategy.

6. How fast are reviewers getting through documents?

Most lawyers already want to know how many documents are still in review. What not as many lawyers worry about is the pace of specific reviewers. 

It’s important to know the current pace of progress, but it’s also important to know if you should just accept that pace or if other changes could accelerate things. Knowing how many documents one reviewer is getting through on a given day is a massive help for gauging whether or not progress could be happening faster.

There’s all sorts of reasons why Reviewer A might get through 20 documents a day, and Reviewer B might get through 10 documents a day. Maybe one reviewer is dealing with longer documents or that time-consuming mobile data we mentioned above. In that case, there might not be much to change that, your data is what it is.

Maybe Reviewer A is more familiar with the review platform you’re using, and an hour or two of extra training for Reviewer B would have them reviewing their docs just as quickly. Maybe you’ll see that reviewers are doing the best they can, but they’re still not working at a fast enough pace to meet your deadline and you’ll need to expand the review team to make it. It’s hard to glean these kinds of insights if all your vendor is giving you is one collective “documents left to review” number.

—-

Knowing the numbers behind document review affects your ability to make good decisions quickly. Contact Discovery’s solution, Vu™, is designed to put these metrics back in the hands of the attorneys who need them most. You can schedule a demo to learn more.

Can Legal Tech Prove Santa is Real?: A Miracle on 34th St. Case Study

It’s the most wonderful time of the year, in part because it’s the best time to watch one of the great court cases of cinematic history. I’m referring of course to Miracle on 34th Street, a film that hinges on one lawyer’s ability to prove that Santa Claus is in fact Santa Claus.  

Our story begins when Kris gets a job playing Santa Claus at the flagship Macy’s store in Times Square. When Kris insists that he is the real Santa Claus, he’s committed to a mental institution. Kris’s friend, Fred Gailey, just so happens to be a lawyer and rushes to his rescue. Fred Gailey shocks the court when he announces that his game plan is to prove Kris’s true identity as Santa Claus.  

There’s also a cute kid who wants a house in the suburbs, but she’s not as important from a litigation support perspective.  

The original film was made in 1947, with a remake made in 1994. Needless to say, our way of storing information has changed since then, and that’s reshaped the way lawyers build court cases. How would the case in Miracle on 34th Street be different if it happened today, when records are stored electronically? Would Santa use a GPS or tag his locations on Instagram? Does he have “find my iPhone” on in case he gets stuck in a chimney again? Maybe he uses Microsoft Teams and Zoom to make sure the elves are still holding down the fort at the North Pole while he’s in New York. Let’s look at some more specific examples from the movie and how this information would be managed today.  
 

Employee Records  

One of the first indications that Kris might in fact be the real Santa Claus is his employee record at Macy’s. This lists Santa’s reindeer as Kris’s next of kin and says he’s from the North Pole. 

Today, these records would live in some kind of electronic database. Legal teams not only look at these records, but also have to think critically about how they might’ve been tampered with. That’s just the nature of electronic records vs. paper records. Honestly, who wouldn’t be suspicious if they saw a record that looks like this?:  

Kris Kringle's job application in Miracle on 34th Street | Miracle on 34th  street, 34 street, Movie quotes

Who has access to those records? Who has edit permissions? Can they access those records from personal devices as well as corporate owned devices? Is there any possible way that passwords could have fallen into the wrong hands? Those are the kinds of questions that a good forensics analyst can answer.  

To verify document integrity, analysts oftentimes look at something called “metadata.” That refers to information such as “date created,” “date last modified,” and “author.” All data comes with metadata, and since it’s not as easily editable as the records themselves, it often proves crucial in digital forensics investigations.  

Analysts can look at a hard drive and figure out if data was moved to other devices, if edit histories were deleted, etc. Assuming Macy’s had good information governance practices and required everyone with edit access to use different passwords, forensics teams could also deduce which passwords were used to make any edits. In some cases, they may even testify in court to assure judges that records are what they appear to be (or maybe that they’re not what they appear).  

Letters to Santa  

The “smoking gun” of the original 1947 movie are the thousands of letters to Santa Claus delivered to the New York City courthouse. Fred Gailey argues that the post office’s decision to deliver these letters to Kris equates to government validation of his true identity: Santa Claus.   

According to the USPS, kids still send hundreds of thousands of letters to Santa every year. The USPS even has a special address they ask people to use for such letters. This system means it’s unlikely that thousands of letters would be delivered to the courthouse like in the movie, since people have been instructed to address such letters to 123 Elf Road. 

However, snail mail is just one of numerous ways to get in touch with St. Nick these days. EmailSanta.com has been on the net since 1997. However, the site’s About Us page admits that it was built not by Santa himself, but by a man named Alan Kerr. There are also numerous Santa Claus Twitter accounts, albeit none of them have a blue checkmark.  

This is perhaps where Fred Gailey would have the biggest uphill battle if he were to try this case today. The Internet has made it infinitely easier for other people to claim they are Santa. Gailey would have to prove that 1) one of the numerous online Santa platforms is the definitive way to get in touch with the real Santa and 2) That Kris was the man behind such an account.  

Let’s assume that by 2020, Santa has gotten with the times and has a proper email account for children to email him with a parent’s supervision. Gailey wants to present Kris’s emails in court. During the Meet and Confer stage of litigation, Gailey and opposing counsel would agree on a certain amount of emails to review. This way, if Kris and Fred were trying to fake letters to Santa, opposing counsel could do their own forensic analysis and figure that out.  

Of course, since Kris really is Santa, there would be a few facts on his side. The litigation support team would be able to verify that he does receive numerous letters to Santa every year. Odds are those letters would be in various languages from all over the world. Sure, that makes it harder to put together a team for review, but it also makes it virtually impossible for opposing counsel to prove that Kris is faking it.   

The best way for Gailey to build a case is similar to the way many complex litigations happen now: build a narrative from both electronically stored information and real-life events. Imagine if Gailey could prove that 1) children asked for certain gifts in emails to Kris and 2) they actually received those same gifts and 3) those gifts didn’t come from parents or other relatives “playing Santa.”  

With presumably thousands of emails to Santa to choose from, this shouldn’t be that difficult. Platforms such as Relativity help review teams search for keywords in different emails. They could search for particularly high demand toys that parents would have trouble finding on their own. After finding such emails in review, Gailey could cross-reference them with Kris’s records of which children got which gifts, because obviously Santa knows the importance of maintaining such records. Nothing gets you on the naughty list as quickly as bad information governance.  Gailey could reach out to families and get them to testify on the stand that they did not know where these gifts came from.   

Exploring these sorts of cases helps us understand how people in this space think on a day-to-day basis. Legal tech has to be about more than fast processing and aesthetically pleasing interfaces, though those certainly don’t hurt. It’s also about having the right people who understand how to build a case and know what to look for in discovery. If you were Fred Gailey, what would your discovery strategy be? How could you verify Kris’s identity and what kind of technology would make it easier? Let us know!