ChatGPT and legal

With the rage that it has quickly become, what can we learn about ChatGPT and the legal profession? CoinDCX in-house counsel Shalini Saxena takes a look down the rabbithole

Artificial intelligence tools cannot yet replace a human’s ability to create connections with clients. Law businesses still need people to empathise, engage, personalise and individualise service for each client.

But ChatGPT systems and its successors may be capable of handling some of the behind-scenes work for law firms, like drafting individual clauses for contracts.

At its heart, ChatGPT is a chatbot; a conversational tool that performs routine tasks efficiently. For earlier examples, think of Apple’s Siri or Amazon’s Alexa virtual assistants. ChatGPT is a step further, notable for its artificial learning capabilities and ability to have human-like conversations.

ChatGPT launched in November 2022, and by February 2023, India had “Lexi”, the country’s first AI powered by ChatGPT, which has been integrated with analytical tools offering business solutions. India’s Ministry of Electronics and Information Technology is, apparently, planning to integrate ChatGPT with WhatsApp to help Indian farmers understand and learn about several government schemes.

ChatGPT recently passed law exams in four courses at universities in the US, according to professors at these schools. After completing 95 multiple choice questions and 12 essay questions, the chatbot performed on average at the level of a C+ student, achieving a low but passing grade in all four courses.

At the Wharton School of the University of Pennsylvania, ChatGPT fared even better, earning a B to B- grade on a five-question experiment. In a paper detailing the performance, a Wharton business professor said ChatGPT “did an amazing job” at answering basic operations management and process-analysis questions, but struggled with more advanced prompts and made “surprising mistakes” with basic math. “These mistakes can be massive in magnitude,” he wrote.

On the flip side to these results, India has joined other countries in banning ChatGPT in universities.

So, there are concerns alongside the excitement, and we all need to start thinking how to leverage this new technology while discouraging misuse.

Legal profession and ChatGPT

To quote Professor Kristian Hammond of the Northwestern University (US) and director of the Center for Advancing Safety of Machine Intelligence who spoke from a panel hosted by the university on 10 January 2023: “You have to understand the length and breadth of the technology and where it collapses, and make sure the task is not one that demands something beyond its limits. ChatGPT might be good at taking a test. But, because of the nature of the underlying mechanism, it may never be capable of genuine reasoning, being imaginative, or thinking beyond the moment.”

Hammond predicts a new “prompt engineer” role will emerge in the short term to improve the system and refine for specificity. “Users who understand enough about a domain will engineer the appropriate prompts to guide the system in the right direction, and the prompts will become the learning driver for the next generation,” he says.

And it all makes sense. In what we can predict of the near future, AI tools cannot make lawyers obsolete, but they will be helpmates that we will all have to learn to build into our working style. This extends beyond the legal fraternity; any function will need to question how to add value in a world where the simpler tasks will be taken away by machines.

ChatGPT-level AI tools are a much-needed solution in multiple areas. For instance, they can:

• Instantly search vast amounts of data and provide relevant results, saving time on tedious tasks like researching case law, finding legal documents, or verifying facts;
• Prepare early-stage documents;
• Automate tasks like document retrieval, summarisation and analysis; and
• Generate blog posts and other content quickly creation.

Let’s look at some useful opportunities in the legal industry, where some drawbacks are also obvious.

Clause creation. The ChatGPT tool is limited in the amount of text it can generate. The longer the prompt, the less reliable the results. It would seem that ChatGPT is not well suited to drafting an entire contract. However, it is perfect for drafting individual clauses in accordance with specific instructions.

This capability will allow legal teams to create error-free clauses in minutes, provided a human lawyer properly indicates the requirements beforehand and checks the end result.

Document search. Data currency is a basic necessity. However, ChatGPT is updated only up to 2021. This means that, as far as ChatGPT is concerned, everything that happened in 2022 never happened. And we are now in 2023.

Let’s talk about how much data is created in the world every day. The current estimate stands at 1.145 trillion MB per day.

If we talk data available in courts in India alone, a good place to start is the National Judicial Data Grid (NJDG). This is an online database of orders, judgments and case details of 18,735 district and subordinate courts and high courts. Litigants can access case status information on more than 22 billion cases and 20 billion orders and judgments. To track cases related to land disputes, land records data of 26 states have also been linked with NJDG.

This is just the tip of the iceberg. Every day, data is being generated across the globe. How will ChatGPT stay relevant and perform the way the creators intend?