Conversational AI security: Why distributors need a built-in trust layer

Conversational AI security is how you control what an AI assistant can see, how your business data is used and whether you can check its answers. For distributors, that means keeping existing user permissions, ruling out model training on your data, guarding against prompt-based leaks and verifying every output.

Distributors have good reason to be cautious about opening their business data to AI. Customer pricing agreements and supplier terms are commercially sensitive, and exposing them through a public AI tool could weaken your position in the market. Before you bring AI into the business, you need to know how it protects your data, who can access it and whether anyone can use it to train shared models.

That's why an AI trust layer matters. For distributors, the strongest starting point is software that already combines those controls with an understanding of your business. Choosing AI shouldn't mean taking on a separate project to build the foundations that make it usable.

What is an AI trust layer?

An AI trust layer is the framework around an AI model that controls what it can access, how it uses your information and how people check its outputs. It connects security and permissions with data governance, business context and ways to verify results.

Security is only one part of trust. The National Institute of Standards and Technology (NIST), a US government agency, released its AI Risk Management Framework in 2023. It describes trustworthy AI as valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced and fair with harmful bias managed. NIST's AI RMF FAQs are clear that addressing these characteristics one at a time won't make an AI system trustworthy, you need them all.

Is your business data used to train Claude or Chat GPT?

If you use business and API plans, public AI don't train on your business data by default. OpenAI says it doesn't train on data from ChatGPT Enterprise, ChatGPT Business, ChatGPT Edu or its API platform by default. Anthropic says it won't use inputs or outputs from its commercial products, such as Claude for Work and the Anthropic API, to train its models by default. Consumer plans follow different policies, so the plan your team uses matters as much as the tool.

Processing data to answer a question and using it for model training are different activities. A provider's commitments should make that distinction clear. Ask these questions before anyone connects ERP data to an AI tool:

Which plan will your team use, and does its training policy apply to every user?

How long are prompts and responses kept, and can you shorten that?

Which third parties process the data, and are they bound by the same terms?

Where is the data hosted?

What's the safest way to connect ChatGPT or Claude to ERP data?

It's probably not a good idea to give a public assistant direct access to ERP tables or bulk exports. We recommend puting a governed layer between the two that checks who's asking, applies that person's permissions before any data reaches the model, sends only the fields a question needs and records what was accessed. Then make sure people can check the answer against the source.

Use this checklist to protect customer pricing:

1. Use business or enterprise accounts as personal accounts fall under consumer terms.
2. Apply least privilege for the AI so it has read-only access, and limit each user to the customers, branches and measures their role needs.
3. Restrict sensitive fields like customer-specific pricing, supplier terms and rebates should only reach the model when the user is allowed to see them and the question needs them. Masking can hide values the AI doesn't need.
4. Keep credentials out of prompts. Passwords and API keys shouldn't appear in chats or connected documents.
5. Log access and record who asked what and which data the AI used, so you can review it later.
6. Make answers checkable so users can see the data and calculation behind an answer.

Customer pricing, supplier terms and financial results deserve protection wherever employees access them. Introducing conversational AI should preserve the access boundaries the business has already set. For example, a rep asking for a customer summary should only receive information permitted for their role. A conversational interface shouldn't open a route to restricted accounts or financial records.

Many distributors already set up role-based access for reps, managers and finance partners and conversational AI should honor the same rules. In Phocas, administrators control which databases each user can open and can restrict the measures, streams, dimensions and specific dimension values they see. Here's how database access and restrictions work in Phocas. If you're comparing options, our guide on how to choose a conversational analytics tool covers the access and accuracy questions to ask.

Building the trust layer versus buying it built in

You can assemble your own controls, but your team then owns the work of maintaining them. Buying software with the trust layer included moves that work to the provider, so you can focus on users, data and decisions.

Businesses building their own AI applications may work with technology providers to configure safeguards across their technology stack. Services such as Amazon Bedrock Guardrails offer configurable controls for sensitive information, denied topics and checking whether responses are grounded in source material. AWS also documents how to test a guardrail before you rely on it.

For a distributor building custom applications, that flexibility can be useful. It also creates work like connecting systems, maintaining permissions, defining business measures and deciding who checks the results. It's one reason generic AI tools connected straight to ERP tables get numbers wrong: they often skip the shared definitions, connected data and permissions that make answers reliable.

Buying software with these foundations included is a more practical route for distributors whose priority is applying AI to the everyday work of growing sales. The provider maintains the product's underlying controls, while you manage your users, data and business decisions.

Conversational AI risks that distributors should know

The OWASP Top 10 for LLM Applications 2025 lists the main security risks for AI assistants. Three matter most when a chat tool can reach your business data: prompt injection (LLM01), sensitive information disclosure (LLM02) and excessive agency (LLM06).

The common thread is simple. Don't rely on the model to police itself. Controls should sit around it, so adding AI never creates a separate route around the access rules already protecting your data.

How does the NIST AI Risk Management Framework apply?

NIST's framework is voluntary, but it gives you a shared checklist for judging any AI tool. Under its secure and resilient characteristic, AI systems are secure when they maintain confidentiality, integrity and availability through protection mechanisms that prevent unauthorized access and use. On July 26, 2024, NIST added a Generative AI Profile (NIST AI 600-1) for risks specific to generative AI.

In practice, two principles do most of the work:

• Least privilege so each person and each AI tool gets only the access they need.
• Data minimization ensures the model receives only the data a question requires.

Both depend on good data governance, meaning agreed owners, definitions and access rules for your data. NIST is revising AI RMF 1.0, so check its site for the latest version before you build a policy around it.

Adoption is moving faster than governance

Research shows confidence in AI hasn't caught up with adoption. An IDC InfoBrief surveyed more than 2,000 supply chain leaders across nine global markets. As DC Velocity reported, 52% cited trust in AI-driven decisions as a top barrier to faster adoption. Only 12% said AI planning governance was fully embedded in their operations.

The research covers supply chains broadly, but it raises a useful question for distributors. Are your AI controls as strong as the tool's capabilities?

Introducing AI into sales, purchasing or finance creates decisions about access, accountability and review. Those decisions get harder to manage when teams use different data from the ERP and a range of tools to understand it, each applying its own business rules and definitions.

Distribution context remains in the foundation

A distributor should expect AI to understand the relationships that make its data meaningful.

Imagine asking which customers are buying less. Does 'less' mean lower revenue, fewer units or fewer product categories? Does the comparison include returns? Are branches using different codes for the same customer?

These are questions a distributor should resolve through shared definitions and connected records. Asking each employee to explain them afresh leaves too much room for inconsistent analysis when using generic conversational AI.

The same complexity applies to inventory. Stock on hand and stock available to sell can tell different stories when orders, allocations or incoming purchases are involved. A specific AI experience needs the relevant measures and context for the question.

That's the argument for choosing software designed around distribution data and workflows. Business understanding should support the answer from the start.

Fresh data needs a clear refresh process

Trust also depends on knowing how current the information is. AI needs current business data to give useful answers. Embedding AI in a BI platform that refreshes from the ERP on a schedule gives it a regularly updated view of sales, inventory and financial performance, without relying on people to export and upload files. In Phocas, data commonly syncs overnight, and database builds can be scheduled hourly, daily, weekly or monthly.

Look for connected data, an appropriate refresh schedule and a clear way to see when information was last updated. A daily refresh may suit one analysis, while another decision may need more frequent updates.

Built-in controls should sit alongside a reliable data process. They can't make an old extract current or correct transactions that haven't reached the source system.

How to secure MCP connections to your business data

Model Context Protocol (MCP) is an open standard that connects AI applications to external data sources and tools. Each MCP connection is a new way into your data, so treat it like any other integration. Authorize each tool, scope access tokens to the user, apply the same permissions the user has elsewhere and limit which actions a tool can take.

The Phocas MCP Server, currently in beta, connects approved AI tools to the governed, modeled data already in Phocas. Your existing Phocas permissions and governance still apply, so connecting an AI tool doesn't grant access to data you couldn't otherwise see.

How Phocas builds in the trust layer?

The Phocas Trust Layer is the shared framework beneath every AI feature in Phocas. It brings together existing user permissions, business context, security and governance rather than operating as a separate product.

It works in four parts:

• Phocas knows who's asking (identity) and applies existing permissions and access controls.
• Phocas supplies the relevant data, definitions and context from your environment.
• A suitable AI model applies the Phocas security and governance controls, built on Amazon Bedrock's enterprise-grade security.
• You get a verifiable answer, summary or action and where supported, a way to check it.

For conversational AI, Workspaces, powered by PACE grounds responses in the Analytics data and business context already in Phocas. It applies each user's access rights, and you can review the data behind a response in the underlying Analytics grid.

Customer and transaction data used by Phocas AI isn't used to train shared or public models, and it stays hosted in your region. Where we use third-party AI providers, we apply enterprise configurations that don't permit data retention for model training. AI features also sit within the wider Phocas platform's security standards, including SOC 2 Type II certification, encryption by default and ongoing security testing.

A built-in trust layer gives distributors a stronger foundation for adopting AI because it connects safeguards to the data and business rules you already use. For distributors, the benefit is continuity. As AI becomes part of more workflows, the same governance principles carry through.