Methodology

Evaluation Framework

The AI Health Index assesses each vendor across thirteen structured capability axes organized in four groups: AI capability (AI centrality, autonomy and oversight model, model and technology transparency, clinical and operational evidence, AI safety and PHI stewardship), regulatory and compliance (HIPAA and BAA posture, security certifications, FDA and regulatory status, AI governance and bias disclosure), integration and deployment (EHR and interoperability depth, deployment model and data residency), and commercial (commercial transparency, setting and specialty coverage). These axes were selected based on the decision criteria most frequently cited by CMIOs, CIOs, and health system operations leaders during vendor shortlisting, with particular weight on the questions AI products raise that conventional software does not.

Each axis receives a letter grade (A through F) or a "Not Rated" designation where evidence is insufficient. A grade reflects the assessed strength of the vendor’s offering on that axis, not a comparison to other vendors. Grades are not aggregated into a composite score; the index holds that no single number can responsibly summarize a healthcare AI vendor’s suitability across disparate operational and clinical contexts.

Vendors are added to the index when they meet a minimum threshold of publicly available information. Vendors for which insufficient evidence exists across multiple axes are listed but marked accordingly, rather than excluded. The absence of a vendor from the index does not constitute a negative assessment.

What Gets Indexed

The index covers companies that sell healthcare technology to organizations, where the product has a real artificial intelligence capability. Consumer products, drug and biotechnology developers, investment firms, care delivery organizations, and general purpose technology companies that merely have healthcare customers are outside its scope. Where a company applies AI internally to develop its own drug pipeline rather than selling software, it is treated as a drug developer and excluded.

Inclusion is by product, not by company. Where a company sells several lines of business, the record covers the AI products relevant to healthcare and states plainly what is excluded. A precision medicine company is indexed for its clinical decision support software and not its sequencing services; a company with both healthcare and non healthcare divisions is indexed only for the healthcare products; a company that pairs software with its own care delivery operation is indexed for the software, with the care operation disclosed.

The same product scoping applies to electronic health record vendors and other established platform companies. An EHR is not itself an AI product, and the presence of AI features inside a platform does not by itself create a record. Where such a company sells a distinct AI product, that product is indexed on the same terms as any other, with the system of record treated as context rather than as the subject of the record. The AI Centrality axis then reports honestly on how central the model is to what a buyer is actually purchasing, which is the material question when weighing an incumbent bundled capability against a specialist tool.

A grade of C or lower on AI Centrality is not a criticism. It is a factual statement that the AI is a layer on a product whose value stands substantially without it, which is information a buyer needs when comparing options that are priced and procured very differently.

A record exists to support a buying decision, so a product must be far enough along that a reader can act on it. Commercially available products qualify. So do products in active regulatory review where validation is complete and the regulatory pathway is disclosed, since an organization planning a program needs visibility into what is coming and from whom. Products at prototype stage, or where the company itself projects first revenue in a future year, are not indexed. This is not a judgment about the technology or the team. It is a judgment about whether publishing a record would imply to a reader that something can be evaluated and purchased when it cannot.

The Thirteen Capability Axes

AI Centrality assesses whether artificial intelligence is the product itself, the engine of a core module, or a feature layer on a platform whose value stands without it. The index includes companies across this full range; this axis is how readers distinguish an AI-native product from a platform with AI capabilities.

Autonomy and Oversight Model assesses what the AI is permitted to do (draft, decide, or act) and how rigorously the vendor discloses its human oversight structure, including escalation thresholds and override paths. The axis grades disclosure rigor, not autonomy itself: high autonomy with a documented oversight model can grade well, while any autonomy with no disclosed oversight grades poorly.

Model and Technology Transparency assesses disclosure of what is under the hood: proprietary models versus fine-tuned foundation models, training data claims, model cards, and update practices.

Clinical and Operational Evidence assesses the strength of evidence behind performance claims, from peer-reviewed prospective multi-site validation at the top of the scale down to outcome percentages published with no methodology. Vendor-reported statistics are always recorded as vendor-reported and are never restated as independent results.

AI Safety and PHI Stewardship assesses how the vendor handles protected health information across the AI lifecycle, including use in training, retention, and de-identification, along with safety engineering disclosures such as guardrails and safety event reporting.

HIPAA and BAA Posture assesses Business Associate Agreement availability, the tier at which the BAA applies, and subprocessor disclosure. The specific BAA tier fact is recorded in the vendor’s pricing record; this axis grades the overall posture and quality of disclosure.

Security Certifications and Trust Center assesses SOC 2 Type II, HITRUST, and ISO 27001 status, verified through public trust centers wherever possible, along with security incident disclosure practices.

FDA and Regulatory Status records whether the product is FDA cleared or authorized as a Software as a Medical Device, operating under enforcement discretion, or positioned as non-device clinical decision support, verified against the FDA’s public databases. The axis grades the clarity and appropriateness of the vendor’s regulatory positioning, not the possession of clearance; products for which clearance is not applicable are not penalized.

AI Governance and Bias Disclosure assesses substantive responsible AI commitments: published model cards, decision support transparency attributes for EHR-embedded tools, bias and fairness evaluations with stated methodology, and third-party AI audits.

EHR and Interoperability Depth evaluates integration maturity with major EHR systems, covering native workflow embedding, SMART on FHIR applications, HL7 interfaces, or standalone operation, verified against EHR vendor marketplace listings and integration documentation where available. For lanes where EHR integration is not the relevant surface, the axis is assessed against the relevant integration surface or marked not applicable.

Deployment Model and Data Residency assesses documented deployment options (cloud, virtual private cloud, on-premises) along with United States data residency commitments and tenant isolation disclosures.

Commercial Transparency assesses whether a buyer can learn what the product costs and how it is priced without a sales engagement: published tiers, published pricing basis, and self-serve trial availability. Specific figures are recorded in the vendor’s pricing record, with estimates labeled as estimates. Vendors that publish no pricing are recorded as Contact the vendor and left ungraded on this axis; a failing grade is reserved for published pricing claims contradicted by evidence.

Setting and Specialty Coverage assesses clarity about where the product is validated to operate (ambulatory, inpatient, emergency, behavioral health, post-acute, home) and specialty coverage where relevant. Narrow coverage clearly stated grades well; the measure is clarity and validation, not breadth.

Source Basis and Grading

Every data point in the index carries a source basis. There are four categories: Vendor Published, Health System Interview, Third Party Estimated, and Regulatory Filing. The source basis is displayed alongside every grade, price, and claim.

Vendor Published indicates data sourced from the vendor’s own documentation, website, or public statements. This is the most common source basis for descriptions, pricing models, and EHR integration claims. Vendor-published data is recorded as stated but is not independently verified unless corroborated by another source.

Health System Interview indicates data gathered through structured interviews with health system staff who have deployed or evaluated the vendor’s product. These sources are anonymized, and interview-sourced data is treated as primary evidence, carrying greater weight than vendor-published claims where the two conflict. This source category is defined in the index’s data model but is not yet in use: no interview-sourced data is currently published.

Third Party Estimated indicates figures derived from third-party sources such as industry analyses, procurement records, or comparable deployments, none of which have been confirmed by the vendor. These figures are always labeled "Estimated" in the interface and accompanied by disclosure notes describing the estimation basis.

Regulatory Filing indicates data sourced from public regulatory databases, primarily the FDA’s device classification and 510(k) databases. Regulatory filing data is factual and independently verifiable.

What "Estimated" Means

When a pricing figure or capability assessment is labeled "Estimated," it means the AI Health Index has derived this figure from sources other than the vendor’s own disclosure. Estimated figures are presented to provide directional guidance and are never represented as confirmed.

Estimated pricing is typically derived from one of three methods: analysis of publicly available procurement records, extrapolation from comparable vendors with published pricing in the same category and care setting, or information provided by health system procurement staff under condition of anonymity.

Estimated figures carry an inherent margin of uncertainty. The index recommends treating any estimated price as accurate within a range of plus or minus thirty percent of the stated figure. Where the estimation basis is particularly thin, the figure is withheld rather than published with a wide confidence interval.

Vendors are invited to confirm or correct estimated figures. When a vendor confirms an estimated figure, the source basis is updated to "Vendor Published" and the "Estimated" label is removed. When a vendor disputes an estimated figure but does not provide a confirmed alternative, both the estimate and the dispute are noted.

Verification Standards

Every vendor record in the index carries a "Last Verified" date. This date indicates when the record was most recently reviewed against its sources. A record may be re-verified without any change to its content, confirming that the existing data remains accurate.

Verification follows a documented cadence: records are reviewed at minimum every one hundred eighty days. Records containing FDA clearance status are reviewed every ninety days, as regulatory status can change. Records where all data is vendor-published and no third-party corroboration exists are flagged with a reduced confidence indicator.

When a material change is detected, such as a new FDA clearance, a change in pricing model, a security incident, or a corporate acquisition, the record is updated outside the normal cadence and the "Last Verified" date is reset. The previous version of the record is not retained in the public interface. Product and capability changes at indexed vendors are published separately in the change log, each with a cited source and an assessment of buyer impact.

The "Last Verified" date on a record does not guarantee that every data point within the record was independently confirmed on that date. It indicates that the record was reviewed and that no material changes were identified. The source basis for each individual data point indicates the strength of evidence for that specific claim.

Editorial Independence

The AI Health Index accepts no payment from vendors for inclusion in the index, for placement within the index, for grades or assessments, or for expedited review. No vendor can purchase a favorable comparison outcome or the removal of a negative assessment.

The index does not currently sell any product or service to the vendors it covers. If commercial products are introduced in the future, they will be kept structurally separate from editorial assessment, and this methodology will be updated to disclose the funding model before any such product launches.

Vendors may submit corrections, additional sources, or requests for re-assessment. These are reviewed against the same standards applied to all data. A vendor submission that is accepted as a source is labeled accordingly. A vendor submission that is rejected is not acted upon, and the rejection is not published.

The editorial team reserves the right to decline or remove a vendor listing where a vendor is found to have misrepresented its capabilities in communications with the index, or where a vendor attempts to influence assessments through channels other than the submission of verifiable evidence.

Limitations

The AI Health Index is a reference tool, not a regulatory body, procurement advisor, or legal authority. Assessments reflect the editorial team’s best judgment based on available evidence at the time of verification. They do not constitute an endorsement, a recommendation, or a guarantee of performance.

The index cannot evaluate a vendor’s performance within a specific health system’s environment. Capability grades reflect the product’s design and stated capabilities, not its realized performance in deployment. Health systems should treat the index as a starting point for due diligence, not a substitute for it.

The index does not assess contractual terms, service level agreements, or vendor financial stability. These are critical procurement considerations that fall outside the scope of this reference. Health systems should engage qualified legal and financial advisors for these dimensions.

AI in healthcare is a rapidly evolving field. A vendor’s capabilities, regulatory status, and pricing may change between verification cycles. The index recommends that users always check the "Last Verified" date and consult primary sources before making procurement decisions.