Large language models (LLMs) are both incredibly powerful and incredibly limited. On the one hand, Generative AI is demonstrably improving contracting efficiency at scale today. On the other hand, Generative AI is a limited, nascent technology–the good news is that it is limited in certain predictable ways, and developing a familiarity with these limitations can help legal professionals navigate potential pitfalls when using the technology in their work. 

Why develop Generative AI skills?

If you are a legal professional, it is important that you develop your Generative AI skills for a few reasons: 

1. LMs are already improving contracting efficiency—in some cases 10x.
2. Current LLM capabilities are the worst they will ever be.
3. Generative AI expertise is a learnable skill that you can develop, and a skill that will only become more important as capabilities improve.

Many people see LLMs as opaque and intimidating, but it is more helpful to think of LLMs as just another technology. And like any other technology, LLMs provide leverage; the better you are at using Generative AI, the more efficient and effective you will be relative to your peers who are not using Generative AI.

How can you get good at using Generative AI in day to day work? The best teacher is immersion. If you find opportunities to use LLMs day to day, you'll go through an iterative and experimental process where you're trying different things.

Mastering Effective Prompting Techniques

‍1. Be Clear and Specific: Detailed prompts yield better results. This is almost so obvious as to go without saying, but Generative AI is not completely magic—you do need to give clear instructions.

2. Use Examples: Including examples within prompts can significantly improve AI outputs. This technique is known as “one-shot learning” (if providing a single example) or “few-shot learning” (if providing multiple examples).

3. Chain of Thought: Encourage the AI to reason through its process step by step. Consider breaking down complex problems to their components parts and adding "let's think step by step" to prompts can enhance accuracy.

4. Role Play: Framing the AI as an expert and assigning a specific role can improve the quality of outputs. For example, starting with "You are an excellent contract lawyer" can yield better results.

Navigating Common Pitfalls

LLMs are prone to two categories of errors:

• Legal Reasoning Errors: These commonly occur when the AI struggles with complex nested logic, such as exclusions to limitations of liability. Breaking down prompts into simpler, single-statement queries can help mitigate these issues.

• Issue Spotting Errors: These happen when the relevant part of the document isn't processed by the AI. To avoid this, users should narrow down the scope of their queries by only passing relevant passages of the agreement into the AI at a time.

We also suggest avoiding the use of LLMs for research tasks due to the risk of hallucinations and outdated information, particularly in cases where you cannot easily verify the AI’s output.

Practical Use Cases for Generative AI

Min-Kyu highlighted three practical use cases for generative AI in contract review:

• Tidy Up Review: AI can check for correct clause references and consistent use of defined terms, significantly reducing tedious manual work.
  
  Prompt: Identify and list any instances where a clause reference points to a non-existent or incorrect clause. If a reference is found to be erroneous, specify both what the incorrect reference is and, based on the context, suggest which specific clause it should actually refer to.

• Red Flag Reviews: AI can flag terms that are generally unfavorable to the user’s position, helping to streamline the review process.
  
  Prompt: You are acting for Party X. Review the agreement and provide a list of the top 5 provisions that are unfavorable to Party X and a brief explanation for why the provision is unfavorable to Party X. An unfavorable provision could be provisions that are particularly onerous to Party X, not market standard, and/or exposes Party X to significant legal or commercial risk.

• Consistency Checks: AI can verify that agreements are consistent with predefined legal positions, providing a brief explanation for any discrepancies.
  
  Prompt: You are acting for Party X. Review the agreement. For each of the requirements below, assess whether the agreement meets each of the requirements. Provide a brief explanation for why the agreement meets or does not meet the requirement.
  • Requirement 1
  • Requirement 2
  • Requirement 3

For more information on the above, check out our webinar replay here where we walk through these areas step by step.