Key Points

• A new Penn State study tested five popular AI chatbots on patent searches, including ChatGPT, Claude and Gemini.
• Results varied widely and every tool made up citations, so inventors can't rely on any one chatbot.
• AI tools can be helpful, especially tools specifically designed for patent research.
• Sharing invention details with a public AI tool could put confidentiality, and possibly patent rights, at risk.

Inventors are increasingly turning to free, general- purpose generative artificial intelligence (AI) chatbots to answer a basic question: Has someone already patented my idea? A new study suggests that these tools can help in limited ways, but inventors should not rely on them to serve as a primary patent search tool. Commercial tools that are designed for patent research, as well as database searches by experienced professionals, remain the most reliable avenues for patent research.

In an article titled "Hallucinations and Hits: Generative AI Patent Searching Performance, Pitfalls, and Library Best Practices," published in the Journal of the Patent and Trademark Resource Center Association, Aidan J. Whitlock and Denise A. Wetzel of Penn State University tested how five well-known AI tools (ChatGPT, Claude, Gemini, Copilot and Perplexity) perform when used for a prior art search. The researchers ran a standardized set of prompts through each tool, using free or basic-tier access.

Key Findings

Results varied widely from tool to tool.

The five tools returned a variety of different results, with very little overlap. According to the report, "Only three patent citations appeared in more than one tool's output … [and] only one patent application was actually cited by all five tools." The authors concluded that an inventor who relies on any single tool will get an incomplete picture of the prior art.

The tools returned false results.

All five tools yielded hallucinations. The researchers found "patent links that redirected to incorrect documents, non-existent pages, or copies of previously cited patents.... This issue occurred across all tools and required manual verification of every cited patent."

The authors warn that the confident, authoritative tone of these tools makes such errors especially risky for inventors who may not be able to spot them.

The tools can still be useful in some situations.

The study did find that standard AI tools do provide value in certain tasks. For example, all five tools did a good job identifying relevant Cooperative Patent Classification (CPC) codes from a description of the invention, which can save time for inventors unfamiliar with the classification system, or for patent professionals who may use those codes for database searching. The tools also performed well when finding patents that are relevant to existing commercial products. In practical terms, this suggests the tools can help inventors and patent professionals understand the patent landscape around existing products in a market.

Cautions for Inventors

The takeaway for inventors and other non-patent professionals is clear: Use caution before relying on general-purpose AI as your primary tool for patent research. These tools can miss important references, incorrectly list patent details and give different answers to the same question, all while sounding confident.

In addition, inventors should be mindful that public AI tools provide no guarantee of confidentiality. An inventor who provides a public AI tool details about their invention risks the invention becoming public information, as the AI tool may share those details with other users, or use the details to train itself on what is known prior art. Although courts have yet to definitively rule on whether disclosure to AI can trigger a loss of patent rights, recent U.S. District Court rulings in other contexts (including attorney-client privilege and trade secret rights) have considered disclosure to a public AI tool to be a public disclosure.

When Used With Care, AI Can Add Value

Despite these cautions, AI tools do have a place in patent research. As the authors note, AI can help "as a supplementary brainstorming and searching tool” in a comprehensive research strategy. In addition, many commercial AI tools are available that have been trained on verified patent data sets, or that include guardrails and deterministic functions that can yield more reliable search results.

When conducting in-depth patent research, inventors should rely on professionals who use a combination of traditional database searching and specialized AI tools that are built for legal and/or patent professionals. At Fox Rothschild, we employ several of these tools, and we also invest in training our team on how to effectively use them. With proper use and training, AI can be a valuable supplement to professional expertise.

For more information, please contact James M. Singer at jsinger@foxrothschild.com or another member of Fox Rothschild’s Intellectual Property Practice Group.

This information is intended to inform firm clients and friends about legal developments, including the decisions of courts and administrative bodies. Nothing in this alert should be construed as legal advice or a legal opinion. Readers should not act upon the information contained in this alert without seeking the advice of legal counsel. Views expressed are those of the authors and not necessarily this law firm or its clients.