This is the first installment of our series on security and privacy in legal AI tools. In each post we examine a specific risk, explain why it matters for legal practice, and describe how LexBuddy addresses it. The goal is to give you a framework for evaluating any tool — not just ours.
Part I: The problem
The incident that explains everything
In April 2023, three incidents at Samsung in under 20 days put AI training risk on the front page. Engineers at the company pasted proprietary semiconductor source code, confidential internal meeting notes, and production hardware details into ChatGPT — to get help with debugging, summaries, and optimizations.
The problem: the version of ChatGPT they were using actively trained on user conversations. That confidential data became part of the model — not because anyone at OpenAI made an active decision to use it, but because that was exactly what the tool did by default.
Samsung banned ChatGPT on its premises shortly after and began developing internal language models. But the code was already out.
For a lawyer, the exact analogue is this: pasting the text of a contract under negotiation, a litigation strategy, an unpublished legal opinion, or a client's financial data into a public chatbot. If the tool trains on those conversations, that information becomes part of the model — available, to some degree, to any other user who asks the right question.
How training on user data works
To understand the risk, it helps to briefly understand how language models improve over time.
Base models are trained once on a massive text corpus. But companies continuously improve them through a process called fine-tuning, which uses real user conversations to make the model more accurate, more useful, and more contextually relevant.
There's also a second mechanism: Reinforcement Learning from Human Feedback (RLHF). Evaluators rate the model's responses, and those ratings guide the adjustment. User conversations are the raw material of that process.
The practical result: when a user writes something into a tool that trains on conversation data, that information can:
- Directly influence the model's weights and how it responds to other users.
- Be stored in the provider's systems as training data.
- Be used to generate synthetic training variations that propagate the original content in ways that are hard to trace.
The word "can" matters. Not every system does this. But whether a given system does is rarely answered clearly in marketing materials — and the default answer, especially for free or consumer-tier tools, is frequently "yes."
The risk map by tool type
Not all AI tools have the same training policy. The risk varies significantly by usage tier:
Consumer tools (free or low-cost). Most use conversations for training by default. OpenAI, Google, and Anthropic let users opt out of having their data used for training in privacy settings — but the default is training enabled. Most users never change that setting because they don't know it exists.
Enterprise tools with a no-training clause. Enterprise versions of the major platforms contractually prohibit using customer data for training. This is the most common solution in the corporate market. It's better than the consumer alternative, but it rests on a contractual commitment — not an architectural limitation.
Tools with architectural data isolation. The highest level of protection. Customer data is technically and structurally separated from any training pipeline. It's not just a contract — the system doesn't have the capability to use that data for training even if it wanted to.
The difference between the second and third tier isn't always visible in a tool's marketing, but it's the difference that matters when you have to answer to a client, a regulator, or a court.
What professional ethics already requires
ABA Formal Opinion 512 (July 2024) clearly establishes that Model Rule 1.6 — the duty of confidentiality — applies to the use of AI tools with client information. Specifically:
Lawyers must make "reasonable efforts" to prevent inadvertent disclosure of client information. Using a tool that can use that information to train models accessible to other users is difficult to reconcile with that standard.
Generic consent in an engagement letter — the kind that says "the firm may use technology to deliver services" — isn't enough to authorize using client data in an AI training system. Specific, informed consent about that particular use is required.
The EU's GDPR adds another layer for lawyers with clients in the EU: Article 5(1)(b) establishes that data may only be used for the specific purpose for which it was collected (purpose limitation). Using clients' legal data to train AI models is a different purpose than delivering legal advice — and requires a specific legal basis that frequently doesn't exist.
The specific risk for legal practice
The information lawyers put into AI tools has characteristics that make it especially sensitive to training risk:
- It's strategic and non-public. Contractual positions, negotiation strategies, and legal risk analysis represent real economic value to the client — and to their competitors or counterparties.
- It's permanent. A strategy that leaks into the training model can potentially influence the responses the model gives other users for years.
- It's hard to trace. Unlike a leaked physical document, the influence of training data on a model isn't traceable or recoverable. There's no "delete" once data is in the model.
- It's potentially identifiable. Even though models don't "remember" specific conversations the way a database would, certain highly specific or uncommon types of information can influence the model's responses in ways that could make the source identifiable.
Part II: The solution
What should a tool guarantee for this risk not to exist?
It isn't enough for a tool to say "we don't train on your data" on its marketing page. A real guarantee has three components:
Contractual guarantee. A specific clause in the services agreement — not the general user terms — prohibiting the use of client data for training of any kind, including fine-tuning and RLHF.
Supply-chain guarantee. The no-training guarantee must extend to every underlying provider that processes the data. If the tool uses a third-party model (such as a base LLM from a cloud provider), that provider must be under the same restriction too.
Architectural guarantee. The strongest version: the technical architecture separates client data from any training pipeline — not as a policy, but as a design. If data structurally never passes through the training pipeline, there's no risk that it will.
How LexBuddy addresses the training risk
At LexBuddy, client data — conversations, documents, case files — is never used to train, fine-tune, or improve AI models. That protection has three layers:
Contractual layer. Our Terms of Service explicitly state that user data is not used for model training under any circumstances. It isn't a configuration option — it's a condition of the service.
Supply-chain layer. The no-training restriction extends contractually to every underlying technology provider that processes user data on our platform. Your clients' data doesn't train our models or our providers' models.
Architectural layer. User data on LexBuddy doesn't pass through any training pipeline. Not as the result of a decision that could change tomorrow — but because the system is designed so that client data flow is structurally separated from any model-improvement process.
The practical result: when a lawyer uses LexBuddy with a client's information, they can answer with precision the question ABA FO 512 requires them to be able to answer — "how will my client's data be used?" — with a clear answer: to respond to the specific request, and nothing else.
Before you put a client's information into any AI tool, ask one question: can this tool use what I write to train its models? If the answer isn't a clear "no" — contractually backed and architecturally implemented — the risk exists.
Next in the series: Zero Retention Policy — what happens to your data when you close the conversation?
This article is for informational purposes only and does not constitute legal advice. For guidance specific to your practice, consult a professional-responsibility specialist in your jurisdiction.
Sources
| Source | Link |
|---|---|
| The Guardian — Samsung bans ChatGPT after data leak (April 2023) | theguardian.com |
| BBC — Samsung workers accidentally leaked confidential data | bbc.com |
| Bloomberg — Samsung data leak via ChatGPT details | bloomberg.com |
| ABA Formal Opinion 512 (PDF, July 29, 2024) | americanbar.org |
| OpenAI — Enterprise Privacy (no training on enterprise data) | openai.com |
| Anthropic — Privacy Policy and data use | anthropic.com |
| GDPR — Article 5: principles relating to processing of personal data | gdpr-info.eu |