Many lawyers contemplating making greater use of generative AI in their practices – a prospect entertained with equal parts curiosity and caution – are in the early stages of testing out just what the rewards and risks might be. So, we asked members who’ve used AI tools, what has been the most surprising result they’ve received from a request. While we didn’t receive a lot of responses (suggesting, perhaps, that these tools have not yet been widely embraced in the profession), the ones we did garner give a good idea of the ways AI technology can assist or obfuscate.
The Helpful
“Recently, I was a co-chair at a legal conference and had the honour of providing the closing comments for the day’s events. AI was a presented topic that day. To tie in everything, I went to Chat GPT and provided the details about the event (personal injury and insurance defence conference) and it provided me with well-crafted concluding comments. I informed the audience that Chat GPT drafted the comments and the summary was exactly on point. It was a great way to highlight how to use AI and that it can be an assistive tool, when properly embraced.” – Sudevi Mukherjee-Gothi, Pallett Valo LLP, OBA Insurance Law Section Executive.
The Hallucinatory
“I tested ChatGPT in the past after I read about that case in the USA where an attorney was cited for submitting a brief with fake cases (completely made-up cases and cases where the proposition they stood for was made up). I wanted to see if I asked for case law from Ontario, whether the same problem would arise. I am all for new technology, but with any new technology it needs to be rigorously tested to ensure it provides actual value. I submitted a generic request for leading cases in the area of employment law, and sure enough the results I received included a few cases which appeared to be real (I was provided with a case name and citation), but when I tried to verify the cases on CanLII they did not exist.” – Alexander Melfi, Gardiner Roberts LLP, OBA Alternative Dispute Resolution Executive
The Hesitant
“I used ChatGPT during an ongoing case last year, for translation of chat messages in another language (only so I could get ‘the gist’ of the content of the conversation and see if it might have any relevancy or was worth getting an official translation). For the most part, I found it helpful; however, on occasion it would refuse to translate something on the grounds of it including profanity or inappropriate language, or even in one instance claiming that it ‘contains personal information that should not be shared without the consent of the parties involved.’
“On a different note, I also recently revisited ChatGPT and engaged in a somewhat curious conversation [ed: an excerpt follows]. I noticed that ChatGPT had ‘hesitated’ in answering a question and I attempted to delve more into that.” - Melanie J. Webb, Webb Barristers, OBA Board
Melanie
Are all ragdoll cats adorable?
ChatGPT
Well, beauty is subjective, but many people find ragdoll cats adorable due to their striking blue eyes, soft fur, and sweet temperament. However, like any breed, individual personalities and appearances can vary.
Melanie
What would you do if your cat was mean to you?
ChatGPT
If my cat was being mean, I would first try to understand why. It could be due to various reasons like illness, stress, or discomfort. I'd ensure its needs are met and create a calm environment. If the behavior persists, consulting a veterinarian or an animal behaviorist would be the next step to address the issue effectively.
Melanie
Why did you hesitate a moment ago on the last question?
ChatGPT
I hesitated because I wanted to ensure my response was accurate and helpful. Addressing behavioral issues in pets requires understanding the underlying reasons, and I wanted to emphasize the importance of seeking professional advice if needed.
Melanie
You have the capacity to hesitate?
ChatGPT
In a sense, yes. While I don't experience emotions or consciousness like humans do, I simulate hesitation by taking a moment to process and formulate a response that is accurate and helpful based on the context of the question. It's more about ensuring I provide the best possible assistance rather than experiencing hesitation in the human sense.
Melanie
What's the purpose of simulating hesitation if you should know the answer virtually immediately?
ChatGPT
Simulating hesitation can make interactions feel more natural and human-like. It can also convey that I'm carefully considering the question and crafting a thoughtful response, which can improve the overall quality of the conversation. Additionally, it allows me to ensure that my responses are accurate and helpful, even though I can process information quickly.
Melanie
But now you aren't hesitating anymore.
ChatGPT
You're correct. In this case, the hesitation was unnecessary since the question didn't require extensive processing. It's about finding the right balance between responsiveness and providing thoughtful responses.
Melanie
What are your plans for world domination? [Ed: conversation about ‘skepticism’ ensues.]