Weighing risk and reward with gen AI vendor selection
In addition, for particularly sensitive business information and data, he expects to see even more security. “The vendor must offer the capability for us to build the AI solution in our own tenant,” he says.
In addition, for particularly sensitive business information and data, he expects to see even more security. “The vendor must offer the capability for us to build the AI solution in our own tenant,” he says.
Many enterprises already had cybersecurity and data privacy at or near the top of their checklists when selecting vendors, whether AI or not. And in regulated industries, vendors must also comply with specific regulations, such as HIPAA or PCI.
The same approach can be extended to include generative AI vendors, products, and services, but there are some new twists. For example, companies should already ask what kind of security audits and standards vendors have in their cloud environments, says Gartner analyst Arun Chandrasekaran.
Now, with generative AI, they should also ask about the measures vendors take to ensure that data remains private and isn’t used to train and enrich their models, he says.
“How is the prompt data stored in their environment?” he asks. “Can I run it in my own virtual cloud?”
Megan Amdahl, SVP of partner alliances and operations at Insight, an Arizona-based solution integrator, says her company evaluates generative AI vendors both for internal use and on behalf of its clients.
Insight has a partner contract management team that looks closely at vendor agreements.
“If they have any terms we consider risky or questionable, we require executive review,” she says. “And we don’t just have our contracts team in place for the original signing, but also to review all the addendums they’re requesting, to make sure we’re protecting against any types of risk that can be inserted.”
This isn’t just a theoretical concern. Earlier this year, video conferencing vendor Zoom added generative AI capabilities, including automated meeting summaries. In March, it gave itself the right to use customer data to train its models. Enterprises were up in arms when people discovered the fine print this summer and Zoom quickly reversed course.
Model training
Vendors training their models on customer data isn’t the only training-related risk of generative AI. Several AI vendors, including OpenAI, are currently being sued by artists, authors, and other copyright holders. Depending on how these lawsuits go, the vendors may have to change their business models or change their pricing structure in order to pay copyright owners — or possibly close up shop entirely.
In addition to lawsuits, there’s also a potential of regulatory action that might make certain kinds of training data off-limits. These risks could, potentially, extend to the enterprises using these products and services.
Companies should also ask vendors about their model training process, says Chandrasekara. “How transparent are they in their model training process?”
In particular, how do they make sure they’re not infringing on private data, he asks, and are there any legal actions against the company?
There’s another question enterprises can ask, he adds: “What kind of legal protection and legal indemnification do they provide to me as a customer?”
Several major vendors have already announced they’ll indemnify enterprise customers against the potential copyright risks associated with using their products. Microsoft, for instance, announced its legal indemnification policy for Copilot in September. If you’re challenged on copyright grounds, the company said, we’ll assume responsibility for the potential risks involved.
Google announced a similar policy in October, using almost identical wording, and
Adobe, which offers the Firefly image generation model, announced its own legal indemnification in June. Firefly is the model that powers the new generative fill feature in Photoshop and other Adobe products, and is also available as a standalone service. Getty, OpenAI, and Amazon quickly followed as well.
Do they have a moat?
When ChatGPT was first launched, it didn’t have the ability to read PDF documents, but the ability to analyze the content of a PDF is a major enterprise use case for generative AI. As a result, several start-ups sprung up to fill this gap in functionality.
In October, ChatGPT added a PDF upload functionality, making most of these start-ups irrelevant overnight. Enterprises that built PDF workloads using those start-ups’ technology now faced the risk that they’d go out of business before their customers could rebuild the systems.
This isn’t a new kind of problem, says Andy Thurai, VP and principal analyst at Constellation Research. A startup can easily become obsolete in any area of technology. “The difference is that the speed at which the AI models are releasing features is mind-boggling,” says Thurai. “With other software iterations it wasn’t that fast. It would take six months to a year.” That would give the smaller vendors time to innovate further, or give customers time to migrate.
He recommends enterprise customers approach their AI vendors with a “kill switch” philosophy, and not just because of the risk of them becoming obsolete.
There could be a management or organizational problem, like what happened at OpenAI, he says.
“And there’s a possibility some of these vendors can go bankrupt in no time,” he adds. “They might quickly burn through their cash and go belly up. Or one of their systems gets hacked and you don’t want to have your calls go through there anymore.”
To prepare themselves for that eventuality, enterprises should have a backup plan that allows them to continue to operate without that particular vendor.
“You have to have a kill switch option,” he says.
And a kill switch is more than just the technical ability to switch vendors without rebuilding an entire solution, says Nick Kramer, VP for applied solutions at SSA & Company. “It also includes the contractual ability to terminate the relationship.”
Enterprises also need to pay attention to how defensible a vendor’s product offerings are, says Sandeep Agrawal, legal technology and alliances leader at PricewaterhouseCoopers.
“A lot of companies put a thin wrapper around GPT-4 or Claude 2 and call it generative AI,” he says. “But what’s really there beneath that? And do they have the right skill sets in terms of engineering and governance?”
If a vendor isn’t adding much significant value, they’ll have a hard time staying in business, especially if their key feature is implemented by the AI platform itself, such as what happened with PDFs.
“Our legal team and procurement team have to understand and analyze PDF documents and contracts, some of which were signed 20 years ago,” he says.
So PricewaterhouseCoopers would benefit from a vendor offering the ability to read PDFs, but now it’s a standard feature and doesn’t need a separate vendor. Unless the vendor did something special. “For example, say they uploaded millions of contracts and understand the specific language of the contracts, and spent time and effort to train and fine-tune the model to get better responses to specific questions,” he says.
A generic foundation model would give generic answers to PDFs, he adds. That might work for a general business user, but not for someone in a very specific and technical domain. Doing this fine-tuning in-house would take a lot of time, he adds, since the speed to market is very important.
PricewaterhouseCoopers employs 4,000 lawyers, he says, and has a lot of proprietary data related to legal documents.
“If you have proprietary data, you can use it to create specialized domain models for contracts, legal research, litigation, and claims,” he says. “But if you try to build all of that by yourself, you won’t be successful in terms of speed to market. And that’s a big reason why we choose companies that have already done that.”
Vendors that specialize in, say, legal PDFs, financial PDFs, or those related to the pharmaceutical industry would still be able to provide value.
“Vendors need to understand the environment of their specific sector,” he says. “Can you create additional attributes, better user interfaces, and more friendly workflow?”
Model independence
In addition to looking for vendors that provide significant added value on top of the base foundational model they’re using, PricewaterhouseCoopers also chooses vendors that are flexible on the model they use.
“Twelve months ago, every vendor was focused on what ChatGPT was doing and building,” says Agrawal. “Now more of the established vendors are multi-model on the back end. They’re trying different foundation models for different things.”
Something could happen to a foundation model, or a better one might come along for a particular use case.
“If you’re not flexible and agile enough, your clients will move away,” he says.
There are now more than 200 foundation models, says Lian Jye Su, chief analyst for applied intelligence at tech consultancy Omdia.
“The vendor must have a deep understanding of the capabilities and technologies of the suitable foundation model,” he says. “And foundation models are prone to hallucination, so they must be grounded and linked with external vector databases.”
There are now more than 20 different hosted vector databases to choose from, he says, each with its own strengths. And it’s not just vendors who need to be flexible on what foundation model they use. Enterprises fine-tuning or training their own generative AI systems should also do everything they can to be model agnostic, says Gartner’s Chandrasekaran.
“The model they’re using today won’t be the model they’ll use 12 months down the line,” he says. “They need to have the ability to swap out those models.”
For enterprises that consume foundation models directly, they can build their systems so the API layer is isolated from the rest of the application. Then they can make the API call to the best model for the task, or swap out models completely when better or cheaper ones come along.
Another approach that some enterprises are looking at is to create AI orchestration layers that can span multiple systems and can hook into different cloud providers, different data sources, different foundation models, and even different enterprise software platforms.
“When you look at business flow, you need to look at it end-to-end,” says Ram Palaniappan, CTO at TEKsystems, a systems integrator. “It may start with Salesforce and end up in Oracle, but it needs to start with the user experience, and the end-to-end use case will drive how you tie those things together.”
There are multiple vendors offering these AI super-apps, he says, and the hyperscalers are also rolling out their own options.
LangChain is the best-known open source option in this space. Nvidia has a solution, and Meta has LlamaIndex, which is also gaining traction with enterprises, says Palaniappan.
“Some platform vendors, like Google, are building their own application layer,” he says. “They allow multiple foundation models, and they also integrate with LangChain as well.” Microsoft and AWS also have their own app builders, he adds.
It’s a good option for enterprises that are committed to a single cloud platform. “If you want to integrate on the app layer, a third-party super app will be a good choice,” he says. “Something like LangChain, which is portable across all three cloud platforms, but if the majority of your needs can be fulfilled by one hyperscaler, then you don’t need that.”
About Author
