Evaluating regulatory compliance

The first question is not whether a vendor is compliant. It is which regime the product falls under, because the answer determines every question that follows, and it varies more than buyers expect across a single shortlist.

A product that interprets clinical data and drives a clinical decision may be a medical device, in which case it is cleared, authorized, or approved for a specific indication, or it operates under enforcement discretion. A product that surfaces information to a clinician who can independently review the basis of the recommendation may sit outside the device definition entirely under the clinical decision support criteria. A product that reads a chart and drafts a document usually falls outside device regulation and inside privacy and contract law instead.

The important case is the one buyers handle worst: many of the highest spending categories in healthcare AI have no vendor level regulator at all. Revenue cycle, prior authorization submission, coding, scheduling, document intake, workforce, and back office automation are governed by contract, by payer and clearinghouse rules, and by the general privacy regime, not by a device pathway. Asking a vendor in one of those categories whether it is cleared produces an honest not applicable and no information.

In an unregulated category the questions that carry weight are different: who is accountable when an output is wrong, what the contract says about that accountability, what error rate the vendor will disclose and how it is measured, what human review sits between the output and the consequence, and which downstream counterparty relationships the vendor depends on to deliver at all. Ask those instead of forcing a device question onto a product that was never a device.

Of the fifteen axes this index applies to every vendor, four are regulatory and compliance axes and two more sit adjacent to them. They are kept separate because vendors are routinely strong on one and weak on another, and a single compliance verdict hides exactly that.

FDA and Regulatory Status

Whether the product is cleared or authorized as software as a medical device, operates under enforcement discretion, or is positioned as non device clinical decision support. What is graded is the clarity and appropriateness of the positioning, not the possession of a clearance. Evidence: the public device databases, the clearance number, the indication for use statement. Not evidence: a compliance page saying the product is designed to meet device standards.

HIPAA and BAA Posture

Whether a business associate agreement is available, at which tier or plan it becomes available, and whether subprocessors are disclosed. Evidence: the agreement itself, the order form, a published subprocessor list with change notification terms. Not evidence: the phrase HIPAA compliant on a marketing page, which describes an intention rather than an instrument.

Security Certifications and Trust Center

Which certifications are held, by which legal entity, over which systems, and current as of when. Evidence: a trust center that will release the report under a nondisclosure agreement, the certificate, the scope statement, the audit period. Not evidence: a logo strip.

AI Governance and Bias Disclosure

Whether the vendor publishes substantive artifacts: model cards, transparency attributes for tools embedded in an electronic health record, fairness evaluations with a stated methodology and a stated population, and third party audits. Evidence: the published artifact. Not evidence: a responsible AI statement of values.

AI Safety and PHI Stewardship (adjacent)

How protected health information moves through the model lifecycle: whether it is used in training, how long it is retained, how de identification is performed and verified, and what safety engineering and incident reporting exist behind the product.

Deployment Model and Data Residency (adjacent)

Which deployment options are documented, where data physically rests, and how tenants are isolated. This axis frequently decides whether the other five can be satisfied at all in a given environment.

Grading them separately is what makes the result usable. A vendor with an excellent trust center and no published fairness evaluation is a different risk from a vendor with the reverse, and both would score identically against a single compliance rating.

Across the roster the same failure recurs in four distinct forms, and in every one of them the vendor is making a true statement that does not mean what a buyer will read it to mean.

Parent and group certificates

An acquired company or a subsidiary citing the certificate held by the group. A certificate names a legal entity. If it does not name the entity on your contract, it does not cover your contract.

Cloud inheritance

The hosting provider offers a business associate agreement and holds security certifications, so the application built on it is described as covered. The infrastructure being eligible says nothing about the application layer, its access controls, its logging, or its own agreement with you.

Partner platform inheritance

A product distributed inside a certified marketplace, an electronic health record platform, or a workflow platform, describing the platform posture as its own. Distribution is not a control environment.

Adjacent domain accreditation

An accreditation earned for a different line of business, a different product, or a different market, presented as coverage for the AI product being sold. Accreditations are scoped, and the scope is written on them.

The test is short and it works on all four. Every certificate names an entity, a scope, and a period. Ask for all three in writing. If the scope does not name the product you are buying and the entity is not the one signing your agreement, the certificate is context rather than coverage. A vendor that answers this cleanly and quickly is also telling you something useful about how it is run.

A clearance is evidence about a specific version, on a specific date, for a specific indication. It is not a statement about the model running in production today, and treating it as one is how a diligence process gets comfortable with something it has not examined.

The indication for use is the part that matters and it is almost always narrower than the marketing. Read it directly from the regulatory record. It will name the population, the modality, the setting, and often the reading or workflow context in which the product was evaluated. Deployment outside that envelope is not prohibited, but it is unevaluated, and the responsibility for evaluating it has moved to you.

Clearance is also becoming a process rather than only an event. A submission can carry a predetermined change control plan, which authorizes a defined envelope of future modification without a new submission. This is a genuine improvement and it changes the buyer question. Ask whether the clearance carries such a plan, what sits inside the envelope, what would fall outside it, and how the vendor will notify you when the model moves within it.

Whatever the answer, keep your own record of the version you validated and the date you validated it. The clearance is the agency’s record of the vendor. The version history in your environment is yours to keep, and nobody else will keep it for you.

Compliance pages are written to reassure, and the verbs carry the meaning. Aligned with, supports, designed to meet, built on, and mapped to describe an intention or an architecture. Certified against, audited by, attested, and cleared describe a completed external process with an artifact behind it. The two categories look alike in a summary table and are not alike in a contract.

The disclosures that answer real questions are rarely on the compliance page. Trust centers, security pages, privacy policies, terms of service, subprocessor lists, data processing addenda, and pricing pages routinely carry specifics that product marketing omits, including the tier at which an agreement becomes available and the default position on training.

One search is not a finding. A single unsuccessful look for a disclosure means the search was incomplete, not that the control is absent. That distinction is load bearing for anyone recording results: an absence you can describe precisely, naming what was searched and what was not found, is a usable finding that a vendor can answer. A blank cell is not.

A vendor with no business associate agreement is not automatically a weaker custodian of data. It may be governed by a different instrument that is equal or stronger, and recording it as a gap misdescribes the situation. Name the regime that applies instead: the European and United Kingdom data protection regimes and the European AI Act, Canadian federal and provincial health privacy law, Brazilian data protection law, and so on. Where the vendor sells into United States health systems, the sharper question is what changes contractually when it does.

Conformity assessment in Europe is per jurisdiction and per risk class, and it is a different exercise from a United States clearance. Neither substitutes for the other, and a vendor holding one while describing it as satisfying the other has made an error worth noticing. The reverse case is equally common and equally worth catching.

Scoping an axis honestly is more useful to a buyer than penalizing a vendor for failing a test that was never relevant to it. That principle applies across the whole framework, not only to geography.

The question is not simply whether a vendor trains on customer data. Nearly every vendor says the right thing when asked directly. The questions that determine what actually happens are: what is the default, is the default set in the contract or in a settings panel, who can change it and by what process, does it survive a renewal or a plan change, and what happens to derived artifacts such as embeddings, fine tuned weights, and evaluation sets that are not themselves protected health information.

De identification deserves the same treatment. Ask which standard is used, whether the determination is documented, who performed it, and whether re identification risk was assessed against the specific data flow rather than in general.

A default expressed only in a help center article is a policy decision wearing a form. If it matters to your risk position, it belongs in the agreement.

These are written to be pasted into a diligence questionnaire. Each one has a document behind it, which is the point: the answers should arrive as artifacts rather than as assurances.

1. 01Which regulatory pathway applies to this product, and if none does, what governs it instead?
2. 02If cleared or authorized, what is the submission number and what does the indication for use say verbatim?
3. 03Does the clearance carry a predetermined change control plan, and what is inside its envelope?
4. 04Which legal entity holds each certification you list, what is the scope statement, and what is the audit period?
5. 05Will you release the current report under a nondisclosure agreement, and when does it next expire?
6. 06At which tier does a business associate agreement become available, and may we read it before signing?
7. 07Where is your subprocessor list published, and how are we notified of changes to it?
8. 08What is the default position on using our data for training, and where is that default recorded?
9. 09What is the product permitted to do without human review, and where is that boundary documented?
10. 10What is the escalation and override path when the model is uncertain, and who monitors it?
11. 11What performance evaluation exists, on which population, and was any of it conducted independently?
12. 12What is your process and timeline for notifying customers of a safety event or a security incident?

Every vendor record here carries a grade on all fifteen axes, including the four regulatory ones, with a source basis attached to each data point and a verification date on the record. Records containing regulatory status are re reviewed on a shorter cycle than the rest, because regulatory status changes more often than anything else on a vendor record.

A low grade on a compliance axis is a statement about what a counterparty can currently verify from published sources. It is not a finding that a control is absent. Where a vendor publishes more, or publishes it somewhere findable, the grade moves. That is the whole mechanism, and it is the same mechanism a buyer runs internally when a vendor sends over a document that changes the picture.

If you work for a vendor and something on your record is out of date, the correction path takes a source and a specific axis, and it is the same path for everyone.