Category Archives: Legal Research Instruction

Natural language processing and machine learning tools in HeinOnline

Check out Shannon Sabo’s two part series on HeinOnline’s implementation of natural language processing and AI: Part One and Part Two. — Joe

Can legal research be taught?

Yes it can but … “legal research instructors should discuss the role that legal analysis and argument play in determining the relevance of legal documents. Likewise, to the extent that legal research is taught as part of an integrated legal research and writing class, the instructor must make the connections between legal bibliography, legal analysis, and legal research clear,” wrote Paul Gatz in his RIPS Law Librarian Blog post, Can Legal Research Be Taught? Part 2: Varieties of Relevance. See also his Can Legal Research Be Taught? Part 1: The Relevance Paradox. Recommended. — Joe

The Algorithm as a Human Artifact

In what will be considered a seminal article of the 2010s in law library literature, The Algorithm as a Human Artifact by Susan Nevelow Mart, Associate Professor and Director of the Law Library, University of Colorado Law School, has been published by LLJ. From the abstract:

The results of using the search algorithms in Westlaw, Lexis Advance, Fastcase, Google Scholar, Ravel, and Casetext are compared. Six groups of humans created six different algorithms, and the results are a testament to the variability of human problem solving. That variability has implications both for researching and teaching research.

Highly recommended. — Joe

Bloomberg Law’s New Feature, Points Of Law

Bloomberg Law announced a new research feature, Points of Law, a little over a week ago.  I’ve been playing around with it using the ATV injury problem I created for teaching online legal research concepts.  In summary, An ATV rider was injured while riding on someone else’s private property without permission.  The problem called for the researcher to identify relevant cases where assumption of risk was a viable defense and collect them for later analysis.  The jurisdiction is New York.

Let me explain a little about Points of Law before I dive into my experience with it.  Bloomberg’s press release describes the feature:

Points of Law offers a more efficient way to conduct case law research.  Through the application of machine learning to Bloomberg Law’s database of 13 million court opinions, Points of Law highlights language critical to the court’s holding, links this language to governing statements of law and relevant on-point case law.

Bloomberg Law provides context – connecting keyword search results to governing statements of law – and unparalleled breadth of coverage, generating one million Points of Law from our state and federal court opinion database.

I found the press release accurate.  I used one of the sample searches I set up for the research problem, <all-terrain vehicle and assumption of risk>.  The case law I expected to see in the list of results was there.  Some of the cases, not all, had a Points of Law icon on the right side of the text.  Clicking that highlights text that the AI in the database considers to be significant.  My search highlighted what I would describe as a combination of black letter law on a keyword related topic or significant points on how the courts treat that topic.  The focus here was on assumption of risk, obviously,  as and all-terrain vehicle is not a legal concept.

Here are some example results extracted from Marcano v. City of New York, 296 A.D.2d 43, 743 N.Y.S.2d 456 (App Div, 1st Dept 2002):

Generally, the issue of assumption of risk is a question of fact for the jury.” (Lamey v Foley, 188 AD2d 157, 163-164 .)

“The policy underlying this tort rule is intended to facilitate free and vigorous participation in athletic activities.” (Benitez v New York City Bd. of Educ., 73 NY2d 650, 657 .)  [Discussing how assumption of the risk in sports is handled by the courts. – MG]

Because of the factual nature of the inquiry, whether a danger is open and obvious is most often a jury question * * *.”

What I found most interesting about using Points of Law is how viewing multiple extracts informed me about assumption of risk without requiring a lot of lengthy analysis.  Now, not all cases in the search results were useful in my context where an ATV rider was injured.  At the same time, a researcher will find what they need to know conceptually about assumption of the risk as treated by the New York Courts.  I assume that applies to other legal doctrines as well.

Another feature worth mentioning is that clicking on the highlighted phrase will open a side window that cites other cases expressing the same point of law (up to 10).  There is also a button that shows a citation map of the Point:

Bloomberg Cite Map.

Another button shows a list of opinions that expressed related concepts along with the Point text:

Bloomberg Related Points of Law

All in all, I think this is a nifty feature that researchers and litigators will actually use.  I wonder if it will integrate with any of the current general search products on the market, as in “Hey Google, find me cases in New York State that discuss assumption of risk in the context of recreational activities.”  If we now think that first year law students take the lazy route in legal research based on their Google use, just wait for the future to show up.

In the Not Everything is Perfect category, one case, Bierach v. Nichols, 248 A.D.2d 916, 669 N.Y.S.2d 988 (App Div, 3d Dept 1998), had one Point of Law listed but not highlighted in the text.  It was short enough that I was able to guess what was the likely text that would have been highlighted.  Oh well.  –Mark

Integrating PACER in the case method of instruction

Here’s the abstract for Teaching with PACER: Improving Understanding by Harnessing Transparency by George Kuney:

The article discusses teaching law school courses with the Public Access to Court Electronic Records (PACER) system and both student and instructor e-writing projects relating to that system.

The case method of instruction in its classic form does not capture the context and intricacies of bankruptcy practice, especially in United States Chapter 11 cases. What is often missing is the rich compilation of detail found in the mountain of filings that is a Chapter 11 case, whether large or small. These details often evidence the shifting loyalties, relationships, interests, and emotions of the participants. These dynamics are largely lost in even trial court decisions in the case, much less appellate opinions.

With the advent of national federal court adoption of the PACER system throughout the United States, however, a new resource for teaching has arrived. Over the last several years, I have been teaching Chapter 11 seminars that augment a text with a research project that uses PACER (or a similar service available in some large cases from the claims agents in the case) to provide access to the docket and substantially all the pleadings and other documents in the case. This research project allows students to research the background of the debtor company, examine what led it to seek relief under Chapter 11, and to follow the case to conclusion, through all its twists and turns, even if there are no appeals. Student reaction has been positive.

Interesting. — Joe

Approaches to incorporating research instruction into transactional skills courses

Professors Lori Johnson, Jeanne Price, and Eric Franklin discuss methods of teaching legal research skills in the context of transactional skills courses in Research Instruction and Resources in the Transactional Skills Classroom: Approaches to Incorporating Research Instruction into Transactional Skills Courses, 18 Tenn. J. Bus. L. 635 (2016).

H/T Legal Skills Prof Blog. — Joe

Using Congressional Research Service Reports in LR&W instruction

Most readers are probably aware of legislative research reports produced by the Congressional Research Service. Two come readily to mind:

But there are plenty more where those came from. Here’s just a sample of CRS reports that can provide useful background reading on the federal legislative process for the legal research component of “how a bill becomes a law.”

Bills and Resolutions: Examples of How Each Kind Is Used (Dec. 2, 2010 98-706)

The Senate’s Calendar of Business (Apr. 21, 2017 98-429)

Calendars of the House of Representatives (Mar. 2, 2017 98-437)

House Floor Activity: The Daily Flow of Business (April 16, 2008 RS20233)

Introducing a Senate Bill or Resolution (Jan. 17, 2017 R44195)

Introducing a House Bill or Resolution (Jan. 12, 2017)

Sponsorship and Cosponsorship of House Bills (Jan 12, 2017 RS22477)

Types of Committee Hearings (June 28, 2017 98-317)

Hearings in the U.S. Senate: A Guide for Preparation and Procedure (Mar. 18, 2010 RL30548)

Hearings in the House of Representatives: A Guide for Preparation and Procedure (June 13, 2006 RL30539)

“Holds” in the Senate (Jan. 17, 2017 R43563)

Enrollment of Legislation: Relevant Congressional Procedures (May 18, 2017 RL34480)

Veto Override Procedure in the House and Senate (Feb. 2015 RS22654)

How Bills Amend Statutes (June 24, 2008 RS20617)

— Joe

Kerr: How to Read a Legal Opinion: A Guide for New Law Students

Orin Kerr’s How to Read a Legal Opinion: A Guide for New Law Students, 11 The Green Bag 2d 51 (2007) celebrates the 10th anniversay of its publication in The Green Bag. It should be required reading during the first week of 1Ls’ law school careers. — Joe

A more rigorous approach to case briefing

In A More Rigorous Approach to Teaching the Reasoning Portion of Case Analysis: A Key to Developing More Competent Law Students, Edwin S. Fruehwald argues that LR&W programs superficially teach students how to analyze a holding for case briefing assignments. Fruehwald presents a more rigorous approach to case analysis, by including in the reasoning section the types of reasoning (rule-based reasoning, reasoning by analysis, distinguishing cases, policy-based reasoning, synthesis) a judge is using and how the judge employs these types of reasoning to obtain the holding. Recommended for 1Ls. — Joe

Should, and if so, how should legal research and writing instruction be integrated in law schools?

Check out Liz McCurry Johnson’s brief article, Teaching Legal Research and Writing in a Fully Integrated Way. Here’s the abstract:

This Article outlines the past model of legal research and writing at Wake Forest School of Law, the problems it presented and the redesign methods implemented over the course of two academic years. It details student evaluations and evidence of better student learning when legal writing and research is taught together, in an integrated fashion, rather than in two paralleling courses. Most importantly, this Article provides evidence to support an argument that legal research should never be taught outside the context of a legal writing, or a doctrinal, problem set. Collaboration among faculty is the key to successful student learning.

— Joe

Just in time for incoming 1Ls: Learning how to read and write legal citations

Alexa Chew’s Citation Literacy adds a new chapter to LR&W pedagogy. Here’s the abstract:

New lawyers and law students spend a lot of time worrying about legal citation. But most of that time is spent worrying about the wrong thing—formatting. The primary purpose of legal citation is to communicate information to the reader. Thus, legal citations are integral parts of the legal documents that lawyers read and write. But rather than viewing citation as communication, law students, and the new lawyers they become, tend to view it as a formatting sideshow dictated by the Bluebook or other citation style guides. This view is both inaccurate and counterproductive.

I argue that the reason for this detrimental and misplaced focus on citation form is because law schools do not teach what I call citation literacy: the ability to both read and write legal citations. Despite the pervasive presence of citations in judicial opinions, which first-year law students spend most of their time reading, pondering, and discussing, law schools teach students only how to write citations. This “write-first” approach to teaching citation deprives legal novices of opportunities to learn to make meaning from citations as readers. Were students able to understand as readers how legal citations operate as communication, the tedious task of writing legal citations would be grounded in an intrinsic purpose rather than a seemingly arbitrary style guide. As things stand now, students must be prodded to follow citation norms for extrinsic gains, which satisfies neither student nor professor.

The citation literacy pedagogy I propose would first teach law students to study the citations in the documents they read—the edited judicial opinions they find in casebooks, the unedited judicial opinions they read in legal writing courses, and sample memoranda or briefs. To teach students to make meaning from citations, citation should be introduced concurrently with foundational lessons in legal authority. Once students understand what citations mean, they can more easily write the citation forms set forth in citation style guides and adapt that knowledge to the heterogeneity of citation forms they will encounter in practice.

Here’s the conclusion to this highly recommended article:

Legal citations play an integral role in legal analysis and legal documents, communicating important information from writer to reader about the support for the writer’s claims. Skilled legal readers incorporate that information into their understanding of the legal texts they read, making meaning from them and using them to assess the quality of legal arguments. However, the prevailing write-first citation pedagogy subverts this communicative purpose, focusing almost solely on teaching students to write citations without first teaching them to read citations. Lawyers need to be able to both read and write legal citations—to be citation literate—and law schools can advance this goal by upending the write-first citation pedagogy, which cabins citation instruction into legal writing courses and deprives students of opportunities to practice making meaning from the citations in the legal documents they read. By promoting citation literacy over citation formatting, law schools might even reduce the misery associated with legal citation and produce graduates who can adapt to whatever the future of legal citation holds.

— Joe

Is artificial intelligence causing a premature disruption in legal research?

Here’s the abstract for Jamie Baker’s 2017 A Legal Research Odyssey: Artificial Intelligence as Disruptor:

Cognitive computing is revolutionizing finance through the ability to combine structured and unstructured data and provide precise market analysis. It is also revolutionizing medicine by providing well-informed options for diagnoses. Analogously, ROSS, a progeny of IBM’s Watson, is set to revolutionize the legal field by bringing cognitive computing to legal research. While ROSS is currently being touted as possessing the requisite sophistication to perform effortless legal research, there is a real danger in a technology like ROSS causing premature disruption. As in medicine and finance, cognitive computing has the power to make legal research more efficient. But the technology is not ready to replace the need for law students to learn sound legal research process and strategy. When done properly, legal research is a highly creative skill that requires a deep level of analysis. Law librarians must infuse law students with an understanding of legal research process, as well as instruct on the practical aspects of using artificial intelligence responsibly in the face of algorithmic transparency, the duty of technology competence, malpractice pitfalls, and the unauthorized practice of law.

— Joe

Artificial intelligence and legal research instruction

Jamie J. Baker’s 2017 A Legal Research Odyssey: Artificial Intelligence as Disruptor discusses the current state of artificial intelligence as it applies to law. The article provides a background in current technological capabilities, shows how these capabilities are being used in various professions, including finance and medicine, and provides an overview of current natural language processing capabilities to discuss how the latest technological advances will realistically be applied to legal research. The article ultimately argues that law librarians must still infuse law students with sound legal research process and understanding so that they have the ability to confidently rely on algorithms in the face of various ethical duties. Here’s the abstract:

Cognitive computing is revolutionizing finance through the ability to combine structured and unstructured data and provide precise market analysis. It is also revolutionizing medicine by providing well-informed options for diagnoses. Analogously, ROSS, a progeny of IBM’s Watson, is set to revolutionize the legal field by bringing cognitive computing to legal research. While ROSS is currently being touted as possessing the requisite sophistication to perform effortless legal research, there is a real danger in a technology like ROSS causing premature disruption. As in medicine and finance, cognitive computing has the power to make legal research more efficient. But the technology is not ready to replace the need for law students to learn sound legal research process and strategy. When done properly, legal research is a highly creative skill that requires a deep level of analysis. Law librarians must infuse law students with an understanding of legal research process, as well as instruct on the practical aspects of using artificial intelligence responsibly in the face of algorithmic transparency, the duty of technology competence, malpractice pitfalls, and the unauthorized practice of law.

— Joe

CRS guides for researching federal legislation

Two regularly updated research guides produced by the Congressional Research Service were updated in the last year.

Researching Current Federal Legislation and Regulations: A Guide to Resources for Congressional Staff (Feb. 6, 2017, RL33895) introduces congressional staff to selected governmental and nongovernmental sources that are useful in tracking and obtaining information on federal legislation and regulations. It includes governmental sources, such as Congress.gov, the Government Publishing Office’s Federal Digital System (FDsys), and U.S. Senate and House websites. Nongovernmental or commercial sources include resources such as HeinOnline and the Congressional Quarterly (CQ) websites.

Legislative History Research: A Guide to Resources for Congressional Staff (July 6, 2016, R41865) provides an overview of federal legislative history research, the legislative process, and where to find congressional documents. The report also summarizes some of the reasons researchers are interested in legislative history, briefly describes the actions a piece of legislation might undergo during the legislative process, and provides a list of easily accessible print and electronic resources.

See also, The Framing of the United States Constitution: A Beginner’s Guide on In Custodia Legis.

Useful. — Joe

‘Finding law:’ Out-of-date metaphors for 21st century legal research

About their very interesting essay, New Wine in Old Wineskins: Metaphor and Legal Research, 92 Notre Dame L. Rev Online 1 (2016), Amy Sloan and Colin Starger write

This essay examines a different set of metaphors currently doing damage in law. Though not as life-and-death dramatic as the War on Drugs or the struggle against patriarchy, these metaphors affect every law student and practicing lawyer. What’s more, our examination implicates broader philosophical issues that resonate well beyond specifically legal discourse. The metaphors we examine pertain to legal research—how we conceptualize the task of ‘finding law’ to make arguments and solve legal problems. The broader philosophical issues concern changes wrought by technology. When technology radically alters our material world, sometimes our conceptual world fails to adjust. To successfully evolve, we must interrogate and change our deepest metaphors. This Essay undertakes this foundational task in the brave new world of legal research.

This Essay argues that conceptualizing emerging legal technologies using inherited research metaphors is like pouring new wine in old wineskins—it simply doesn’t work. When a primary challenge of research was physically gathering hidden and expensive information, metaphors based on journey, acquisition, and excavation helped make sense of the research process. But new, technologically-driven search methods have burst those conceptual wineskins. The Internet and Big Data make information cheap and easily accessible. The old metaphors fail.

Recommended. — Joe

Beyond the traditional view of legal reasoning

Here’s the blurb for Beyond Legal Reasoning: A Critique of Pure Lawyering (Routledge 2017) by Jeffrey Lipshaw (Suffolk University Law School):

The concept of learning to ‘think like a lawyer’ is one of the cornerstones of legal education in the United States and beyond. In this book, Jeffrey Lipshaw provides a critique of the traditional views of ‘thinking like a lawyer’ or ‘pure lawyering’ aimed at lawyers, law professors, and students who want to understand lawyering beyond the traditional warrior metaphor. Drawing on his extensive experience at the intersection of real world law and business issues, Professor Lipshaw presents a sophisticated philosophical argument that the “pure lawyering” of traditional legal education is agnostic to either truth or moral value of outcomes. He demonstrates pure lawyering’s potential both for illusions of certainty and cynical instrumentalism, and the consequences of both when lawyers are called on as dealmakers, policymakers, and counsellors.

This book offers an avenue for getting beyond (or unlearning) merely how to think like a lawyer. It combines legal theory, philosophy of knowledge, and doctrine with an appreciation of real-life judgment calls that multi-disciplinary lawyers are called upon to make. The book will be of great interest to scholars of legal education, legal language and reasoning as well as professors who teach both doctrine and thinking and writing skills in the first year law school curriculum; and for anyone who is interested in seeking a perspective on ‘thinking like a lawyer’ beyond the litigation arena.

Recommended. — Joe

Using linguistics to determine the ordinary meaning of the language of the law empirically

In the abstract for Judging Ordinary Meaning, Thomas R. Lee and Stephan C. Mouritsen write:

We identify theoretical and operational deficiencies in our law’s attempts to credit the ordinary meaning of the law and present linguistic theories and tools to assess it more reliably. Our framework examines iconic problems of ordinary meaning — from the famous “no vehicles in the park” hypothetical to two Supreme Court cases (United States v. Muscarello and Taniguchi v. Kan Pacific Saipan) and a Seventh Circuit opinion of Judge Richard Posner (in United States v. Costello). We show that the law’s conception of ordinary meaning implicates empirical questions about language usage. And we present linguistic tools from a field known as corpus linguistics that can help to answer these empirical questions.

When we speak of ordinary meaning we are asking an empirical question — about the sense of a word or phrase that is most likely implicated in a given linguistic context. Linguists have developed computer-aided means of answering such questions. We propose to import those methods into the law of interpretation. And we consider and respond to criticisms of their use by lawyers and judges.

Interesting. — Joe

A trip down memory lane: a visual history of the development of Google Search Engine Results Pages

Google Search Engine Results Pages (SERPs) have changed dramatically over the past 20 years. In A visual history of Google SERPs: 1996 to 2017 (Search Engine Watch), Clark Boyd writes

The original lists of static results, comprised of what we nostalgically term ‘10 blue links’, have evolved into multi-media, cross-device, highly-personalized interfaces that can even adapt as we speak to them. There are now images, GIFs, news articles, videos, and podcasts in SERPs, all powered by algorithms that grow evermore sophisticated through machine learning.

Search Engine Watch’s infographic identifies the evolution of Google Search Engine’s results pages here. Recommended. It could be used in a teachable moment about the consequences of algorithmic change generally before moving to the great unknowing of algorithmic changes engineered by WEXIS and displayed in WEXIS search output. — Joe

 

Has West Search sacrificed comprehensiveness?

As a quick follow-up to my earlier post titled 10,000 documents: Is there a flaw in West Search? (March 20, 2017), it appears that a West reference attorney has confirmed my conclusion that Westlaw does not offer as comprehensive a searching capability as Lexis.

In reading Mary Whisner’s (Reference librarian, Gallagher Law Library, University of Washington School of Law) research paper Comparing Case Law Retrievals, Mid-1990s and Today, Appendix B records an exchange between Whisner and a West reference attorney. Here’s the pertinent parts:

11/18/2016 01:14:35PM Agent (Anna Wiles): “Those searches seem to be maxing out.”

11/18/2016 01:14:51PM Agent (Anna Wiles): “Sometimes, the algorithm will cut off before 10,000.”

11/18/2016 01:23:26PM Agent (Anna Wiles): “If you run the search in all states and all federal, it will max out because it is a broad search.”

11/18/2016 01:23:53PM Agent (Anna Wiles): “If you narrow by a jurisdiction, the results will not max out.”

But Whisner was attempting to perform a fairly comprehensive search. Note that West Search sometimes will max out at under 10,000 documents too according to the West staffer.

More evidence that in an attempt to find the Holy Grail of legal research —  the uber precise search result — West Search may have sacrificed comprehensiveness. — Joe

10,000 documents: Is there a flaw in West Search?

10,000 documents is an awful lot. Truly a low precision, high recall search. But sometimes, one starts off searching very broadly because Westlaw and Lexis Advance provide a “search within results” option to narrow down initial search output. While I do not perform many broad searches in Westlaw, I have never once seen a figure higher than 10,000 documents in my search results. I have, however, seen “10,000+” documents in equally broad Lexis Advance searches on the same topic.  Unfortunately 10,000 documents appears to be a search results limit in Westlaw.

If an initial search pulls up 10,000 documents in Westlaw, there is no reason to believe all Westlaw documents identified by one’s search are really all the potentially relevant documents in the Westlaw database. Searching within the initial 10,000 documents search results would be, therefore, based on a seriously flawed subset of the Westlaw database, one defined by West Search, not one’s search logic. This is not the case in Lexis Advance where a broad search may yield 10,000+ documents for searching within initial results. If this is indeed a flaw in West Search’s output, one must conclude that Lexis Advance offers more comprehensive searching of its database than Westlaw. — Joe