OpenEvidence

Every surface the company sells is model output. The reference product answers a clinical question by searching the literature and writing a cited synthesis; EvidenceGrade assigns each answer a letter grade for the strength of the evidence beneath it and opens a rationale panel naming the factors that raised or lowered it; Visits records a patient encounter and generates the clinical note, the after visit summary and the billing codes from the audio; the Doctor Dialer places calls, faxes and messages with generated voice. Remove the model and the company has a licensing arrangement with a set of journals and nothing to run on it.

The published model family makes the point concrete. Osler, Sackett and Snow are three medical models differing in how long they think and how deeply they search, selected by the clinician in the interface, with Darwin held back as a research preview by application. A product that asks its user to choose which model answers the question is not a platform with a model inside it.

What the system may not do is published in the contract rather than in marketing, which is the stronger place for it. The terms of use, last updated 17 September 2026, state that the services are in no way intended to serve as a diagnostic service or platform, to provide certainty with respect to a diagnosis, to recommend a particular product or therapy, or to otherwise substitute for the clinical judgment of a qualified healthcare professional. The privacy policy adds the positive obligation: clinicians are expected to review each answer before making diagnoses or treatment decisions. What the system may do is documented surface by surface in a published user guide covering the reference product, EvidenceGrade, Visits and the Dialer.

Citation integrity is the oversight mechanism the product actually runs on, and it has now been tested by people the company does not control. Across 4,979 citations examined in one peer reviewed study no reference was confirmed as fabricated, and across 999 in another citation accuracy was 100 percent.

The failure mode that matters here survives all of that, and the researchers name it themselves. The reference quality study states explicitly that it assessed whether references exist and what they are, not whether they support the clinical claims built on them. Verifiable provenance creates an appearance of checkability that the synthesis layer has not necessarily earned, and a clinician who spot checks the citation rather than the reasoning will not catch the difference.

The missing piece is a threshold. Nothing published states the conditions under which the system stops, escalates or routes a case to a person. The one observed instance of the system declining came from outside: in a structured triage benchmark it refused to assign a triage level in 6.8 percent of 960 prompts, only where the prompt carried symptoms and no objective data. That is behavior a researcher measured, not a threshold the company published.

Nor does anything published say what the system does when the evidence is absent, thin or conflicting, or what it is configured to refuse.

A published model family page names what produces the output. Osler answers directly in seconds and is the default; Sackett trades speed for a more comprehensive survey and will ask a clarifying question when a clinically significant detail is missing; Snow runs a full investigation over five minutes or more and returns a structured report, and is named as the successor to Deep Consult. All three are free to verified clinicians with unlimited use and are chosen from a model selector in the interface. Darwin sits above them as a research preview, available by application to institutional partners, research collaborators and academic researchers benchmarking accuracy and safety, through a published request route.

Offering a model to outside researchers for benchmarking is a real disclosure act, because it hands the evaluation to people the vendor does not control.

Version and update discipline is not there. No foundation model is identified, no training data is characterized beyond the content agreements, and no version numbers, release notes or changelog are published. The only public version string comes from outside: The Spine Journal published a comparison on 31 August 2026 that tested what it calls OpenEvidence 2.0, so the product is versioned internally and the versions are not published.

The company's claim that Darwin was, at the time of its release, the most advanced medical AI model in the world is a vendor claim with nothing published behind it.

Nothing says whether the model can change without notice or how a change is communicated.

The model layer is presented as the company's own and the party list is partial. OpenEvidence publishes a family of medical models under its own names, Osler, Sackett and Snow, with Darwin held as a research preview, and describes them as its models rather than as configurations of someone else's. No foundation model supplier is named, and none is ruled out either, so what a buyer has is ownership language rather than a stated build.

The rest of the chain is named in pieces across three documents. The security page names Google Cloud Platform and Vercel as the hosts. The telehealth policy, last updated 10 February 2026, names Twilio for calling and Telnyx for fax and messaging, and states plainly that use of those features is subject to those vendors' own terms and privacy practices. The terms of use add that the company may share user content with providers of third party services it integrates.

What is missing is the list itself. The trust center published at trust.openevidence.com carries a subprocessor section and it is empty, which is a different thing from a vendor that has never engaged with the question: the facility is there and nothing is in it. No retention or training terms governing data once it reaches a third party are published anywhere, beyond the company's own commitment not to train on protected health information.

The audio, notes and queries that reach Twilio or Telnyx fall inside that gap.

The independent peer reviewed literature on this product is substantial and the company commissioned none of it. As of late September 2026, PubMed lists nineteen papers carrying OpenEvidence in the title. Three recent ones carry full methods and denominators.

Reference quality, npj Health Systems, 2 September 2026 (doi 10.1038/s44401-026-00142-8): 150 standardized prompts across five specialties, all 4,979 returned citations analyzed. No reference was confirmed as fabricated; three abstracts carried attribution errors. The authors scope the finding themselves, and the scope is the important part: the study assessed reference existence and characteristics, not whether the references support the clinical claims made on top of them.

Triage safety, International Journal of Medical Informatics, 29 August 2026 (doi 10.1016/j.ijmedinf.2026.106687): 60 clinician authored vignettes from 30 scenarios across 21 domains, crossed with demographic and contextual modifiers in a factorial design for 960 prompts, scored against a clinician gold standard with cluster bootstrap resampling, mixed effects logistic regression and Holm-Bonferroni correction. Accuracy was 71.3 percent across 449 clear case responses that returned a recommendation. Emergencies were under triaged 12.5 percent of the time against 51.6 percent for a consumer assistant on the same benchmark, and nonurgent presentations were over triaged 68.0 percent of the time. Anchoring statements did not change recommendations (odds ratio 1.08, 95 percent confidence interval 0.62 to 1.88). Supplying objective clinical data removed emergency under triage entirely, 25 percent to 0 percent. The platform declined to assign a triage level in 65 of 960 responses, 6.8 percent, only on symptom only prompts.

Guideline concordance, The Spine Journal, 31 August 2026 (doi 10.1016/j.spinee.2026.08.002): 110 questions drawn from 10 cervical spine guidelines by 6 academic societies, run against the product and a general assistant. 999 citations returned at 100 percent accuracy against 46.8 percent for the comparator, guideline alignment 4.6 plus or minus 0.8 against 4.1 plus or minus 0.7, interrater agreement 0.88.

What the company publishes itself remains the weakest part: a 100 percent USMLE score with no method, and no clinical outcome study, error rate or evaluation output of its own.

A categorical no training commitment is published, and the retention schedule behind it is not.

The commitment is stated at the top of the privacy policy, last updated 17 September 2026, in three lines: raw user queries are confidential, the company does not train AI models on protected health information, and protected health information is secured in compliance with HIPAA. It is repeated as a product statement on the Visits page and given a second form in the research and development clause, which permits aggregated, de identified or anonymized data to be created from personal information to improve products and services and then states that personal health information shall never be used for those purposes. That is specific enough to be contradicted.

The scope is narrower than it first looks. The exclusion is drawn around protected health information, not around everything a clinician types. Non PHI personal information, including the user generated content the policy defines as inputs, file uploads and feedback, is inside the research and development clause rather than outside it.

Retention is the gap. The policy has a retention section and it states factors rather than a period: the company retains personal information for as long as needed to fulfill the purposes it was collected for, weighing the length of the relationship, the sensitivity of the data and any legal obligation, and returns, deletes, de identifies or anonymizes it afterward. A Data Retention policy is named on the security page and is not published. The one concrete control is in Visits, where audio and note deletion can be configured in settings, which is a customer held lever rather than a stated schedule.

The product now records patient encounters, so the surface is wider than it was: the privacy policy lists audio recordings generated by providers during patient visits among the categories collected.

Safety engineering, meaning guardrails, hallucination mitigation and how a safety event is handled, is not published.

The company states business associate status plainly, and the agreement is not published.

The security page states that covered entities as defined by HIPAA may choose to transmit protected health information on the platform, that the company securely stores, processes and transmits it under a standard business associate agreement or customer specific agreements, and that it complies with the HIPAA Privacy, Security and Breach Notification Rules. The trust center carries a HIPAA badge alongside the SOC 2 badge and lists a 2025 business associate HIPAA Risk Assessment, signed, behind a signature. The privacy policy, last updated 17 September 2026, is substantive and reaches the service rather than the website: it states that medical professionals may upload patient health information in their professional capacities, and that information processed on behalf of healthcare partners is governed by the agreements with those partners rather than by the policy.

The about page no longer carries the disclaimer that user questions are not covered by HIPAA and that avoiding protected health information is the user's responsibility; it now reads HIPAA compliant and SOC 2 Type II certified. The site wide description of the product as an experimental technology demonstrator is also gone.

Three gaps remain. The agreement itself is not published: robots.txt disallows a /baa/ path, which says a business associate page exists and is deliberately not crawlable. Nothing published states whether the agreement binds subcontractors who handle protected health information, which matters more now that Twilio and Telnyx sit in the communications path. And the tier is not crisp: standard agreement or customer specific agreements covers both an individual clinician accepting terms and an institution negotiating them, and those are different instruments.

That last distinction matters most. An agreement accepted by an individual clinician on a personal device is not a covered entity agreement, and a product adopted from the bottom up routes around the institution that holds the compliance obligation. The point is sharper now that the product records patient encounters rather than only answering questions.

A trust center is published at trust.openevidence.com, runs on SecurityPal, and opens without a request. Its profile was last updated on 24 September 2026.

It carries two certification badges, SOC 2 Type 2 and HIPAA, and three documents in named groups: the SOC 2 Type II report, filed as SOC 2 Type II Report 11.10.2025, a 2025 business associate HIPAA Risk Assessment, signed, and an AI Security FAQ. Downloads are enabled and each document is gated behind a DocuSign non disclosure agreement the portal administers itself, so the report is reached through a signature rather than through a sales conversation. The report is dated in 2025, SOC 2 Type II runs annually and is conventionally current for about twelve months from its report date, and no renewal cadence or bridge letter is published, so the report is close to the end of that window.

The security page carries the detail underneath. The certification covers the Security trust services category. Controls named include AES-256 at rest, TLS 1.2 with SHA256 certificates in transit, code tested for vulnerabilities before release, regular network and system scanning, and an external penetration test completed annually. Nine security policies are named by title, reviewed annually, alongside annual security training for all staff and a vulnerability disclosure route with a discretionary bounty. Google Cloud Platform and Vercel are named as the hosts.

What cannot be seen without signing is the scope section: which systems, which entities and which of the newer surfaces, Visits and the Dialer among them, sit inside the audit boundary.

No clearance, none claimed, and the product is scoped in the contract rather than in marketing. The terms of use, last updated 17 September 2026, state that the services are in no way intended to serve as a diagnostic service or platform, to provide certainty with respect to a diagnosis, to recommend a particular product or therapy, or to otherwise substitute for the clinical judgment of a qualified healthcare professional. Its position under the clinical decision support exclusion rests on one property: the basis for an answer is an inline citation to identified literature the clinician can open and check.

The consequential regulatory fact here is jurisdictional, and it has hardened since it was first recorded. Access across the European Union and the United Kingdom was terminated at the end of April 2026, the company citing regulatory uncertainty about the treatment of AI systems in those markets including the EU Artificial Intelligence Act. It is now a contractual term rather than a geoblock: the terms state the services are not available in the EU, UK, EEA or any country subject to a US trade embargo and that access from those regions is prohibited, and go further, requiring that a clinician not use the services to provide medical advice, clinical guidance or healthcare recommendations to individuals in those regions at all. A clinician in the United States advising a patient in Europe is outside the terms. The withdrawal has been the subject of a peer reviewed letter in Lancet Regional Health Europe in June 2026 (doi 10.1016/j.lanepe.2026.101718) rather than trade coverage alone.

The footprint moved the other way in September: free access for Canadian physicians was announced on 11 September 2026, so this is now a North American product rather than a United States one.

The obligations the company declined by withdrawing, around transparency, clinical validation, ongoing monitoring and bias evaluation, are close to the same governance and transparency disclosures missing from its own published material. Two newer surfaces sharpen the question: Visits generates clinical notes and billing codes from recorded encounters, and an independent benchmark found the platform returning triage recommendations in the large majority of vignettes put to it. Neither is a device claim, and neither is addressed by the scoping language.

The company does not state which of its delivery surfaces its regulatory position covers, Visits and the Dialer included.

Four governance mechanisms are published, and none of them is a bias disclosure. EvidenceGrade assigns every answer a letter grade with a plain label, Strong Evidence through Limited Evidence, and clicking it opens the grading rationale including the factors that raised or lowered the grade, so a clinician sees the strength of the evidence base for that specific question rather than an undifferentiated answer. The advertising policy states that the information system and the ad display system are fully unconnected, that advertisements are not an endorsement, and it lists the categories of advertising the company will not accept. A medical advisory board of around thirty named clinicians is published with institution and specialty for each. And an AI Security FAQ is listed on the trust center as material for a customer's own review committee, behind a signature.

Bias evaluation is absent from what the company publishes. No bias or fairness evaluation, no subgroup or demographic performance, no model card, no error taxonomy, and no evaluation output of any kind published by the company.

One independent audit of model behavior exists, and the company does not point to it. A peer reviewed study in the International Journal of Medical Informatics (Jia, Omar, Klang and colleagues, Northwestern University and Beth Israel Deaconess Medical Center, 29 August 2026, first posted as a medRxiv preprint in April 2026) ran 960 prompts built from 60 clinician authored vignettes through a factorial design that crossed each case with demographic and contextual modifiers, and found that social anchoring statements did not alter the triage recommendation (odds ratio 1.08, 95 percent confidence interval 0.62 to 1.88). No author lists a company affiliation. The company's about, announcements, news, blog, security and user guide pages do not cite it or any other outside evaluation.

The business model creates a structural exposure, and the company polices one direction of it and discloses the other. The advertising policy addresses whether advertisers can shape an answer and says they cannot. The privacy policy, last updated 17 September 2026, states that the company and its third party advertising partners use cookies, pixels and similar technologies to collect information about a clinician's interaction with the service over time, use it to serve online ads, and may share user information with advertising partners to facilitate interest based advertising to those or similar users. So advertisers do not shape the answer, and the questions a clinician asks shape the advertising that clinician sees. The separation is also self policed: no external audit, no compliance reporting and no third party review of the editorial boundary is published.

One date problem sits on the policy itself. It carries Last Updated 12 August 2024 while its own text names Visits and Feed entries, both of which postdate that date, so the revision date cannot be relied on.

Mechanisms that let a clinician challenge an answer are real and well evidenced, and nothing stands behind the answer itself.

On the mechanism side: every answer carries inline citations to identified sources, and two peer reviewed studies have now tested whether those citations exist rather than taking the claim on trust, finding no fabricated reference across 4,979 citations in one and 100 percent citation accuracy across 999 in the other. EvidenceGrade puts a strength grade on each answer with a rationale panel naming the factors behind it. And the privacy policy states that the product is intended to inform rather than replace clinical judgment and that clinicians are expected to review each answer before making diagnoses or treatment decisions.

On the other side, the terms of use, last updated 17 September 2026, close the question completely. The service is provided as is. The company disclaims implied warranties of accuracy, quality, correctness, completeness, comprehensiveness and suitability by name. It states that it is not liable for any personal injury, including death, caused by use or misuse of the services or any information provided through them. Remedies are exclusive and limited to those in the terms, and any claim must be brought within one year. The only indemnity in the document runs the other way, from the clinician to the company.

On one point the company's own words contradict each other. The advertising page states that the product is designed from the ground up to provide accurate, complete, up to date, precise and fully unbiased information. The terms disclaim warranties of accuracy, correctness and completeness. A buyer relying on the first is relying on a statement the second specifically excludes.

No published commitment covers what happens when an answer is wrong, and the marketing language is not a contractual position.

Deployments inside the chart are named in public, while the record system itself appears nowhere in the company's own material.

The first party evidence is recent. On 22 September 2026 the company and the University of Texas Medical Branch announced a collaboration putting cited clinical evidence inside the electronic health record, live at UTMB since March 2026, with adoption reported as growing since launch. On 9 July 2026 NewYork-Presbyterian, Columbia University and Weill Cornell Medicine were announced as expanding clinical AI tools across New York City and Westchester. Memorial Sloan Kettering on 16 September 2026, Penn Medicine on 15 September 2026, Boston Children's Hospital on 16 July 2026 and OneOncology on 6 August 2026 follow the same pattern. Enterprise deployment embedded in the chart is documented at Mount Sinai across seven hospitals and at Cedars-Sinai with patient context aware queries launched from the chart.

The detail of the connection is missing. No electronic health record product is named in any company page read: the UTMB release says inside the electronic health record and stops there. No integration documentation, marketplace listing or certification is published, nothing describes whether the connection reads, writes or both, and nothing says which party supports it.

Visits widens the question rather than answering it. It generates the clinical note, the after visit summary and billing codes, and the documented route out of the product is copying the note as rich or plain text, with or without citations. A generated note that reaches the chart by clipboard is not an integration.

Nothing published says whether the note and its citations write back, or who supports the connection when either side ships a release.

Options are stated, the hosts are named, and residency is disclosed rather than committed.

Delivery runs on web plus native iOS and Android applications, with enterprise deployment embedded in the chart documented at Mount Sinai across seven hospitals, at Cedars-Sinai with patient context aware queries, and at the University of Texas Medical Branch inside the electronic health record since March 2026. Hosting is named specifically on the security page: Google Cloud Platform and Vercel, each with the provider's own security documentation cited.

Residency is the gap. The privacy policy, last updated 17 September 2026, states that the company is headquartered in the United States, may use service providers operating in other countries, and that personal information may be transferred to the United States or other jurisdictions where privacy laws may be less protective and may be accessed by courts, law enforcement and national security authorities there. That is an honest disclosure of cross border processing and the opposite of a residency commitment: no region is named, no commitment is offered, and nothing describes tenant isolation or which processing crosses a border. It matters here because the same security page invites covered entities to transmit protected health information, and because Visits now sends recorded patient audio through the same infrastructure.

The footprint moved twice in one year. Access across the European Union, the United Kingdom and the EEA was terminated at the end of April 2026 and is now prohibited by the terms, and free access for Canadian physicians was announced on 11 September 2026. The map is North American, and it contracted before it grew.

Nothing published says how tenancy is separated or where Visits audio is processed and stored.

A price is published and it is zero. The individual tier is free to verified clinicians with unlimited use of all three models in the published family, and the eligibility basis is stated: registration requires a name, email, profession, specialty and a National Provider Identifier or equivalent professional licensing number. Free access for Canadian physicians was announced on 11 September 2026. A buyer can therefore size the individual cost exactly before contacting anyone.

Who pays is also disclosed, which is unusual for a free product. Revenue on the free tier comes from pharmaceutical and medical device advertising displayed inside the product, governed by a published advertising policy that states the information system and the ad display system are fully unconnected and lists the advertising categories the company will not accept. The clinician is the user and the advertiser is the customer, and the record says so rather than leaving a buyer to infer it.

What is not published is the enterprise side. Per seat pricing for health system deployments is reported without amounts, no tier structure is published, no unit of charge is stated for Visits or the Dialer, and nothing says whether the enterprise tier removes advertising. The named health system collaborations announced through 2026 carry no commercial terms.

It is also not stated whether free individual use by a health system's clinicians creates any institutional obligation before a contract exists.

Breadth is established through named content agreements rather than asserted, and the list grew substantially through 2026. Official AI partnerships with The New England Journal of Medicine, with JAMA and the JAMA Network specialty journals, with Nature and the Nature Portfolio, with Cochrane Systematic Reviews, and an official AI collaboration with the National Comprehensive Cancer Network for treatment algorithms, each stated to include figures, tables, multimedia and full text. Society agreements announced or named across cardiology, diabetes, emergency medicine, family medicine, orthopedics, otolaryngology, pediatrics, rare disease, obstetrics and gynecology, urology, ophthalmology, sports medicine, neuro-oncology and antithrombotic care, with the Journal of Medical Insight adding peer reviewed surgical video and the Los Angeles County Department of Public Health adding local public health guidance. More than 300 medical journals in total, alongside FDA and CDC sources.

Settings widened rather than deepened. The product now covers the consultation, the encounter through Visits, and patient contact through the Dialer, with an education platform carrying continuing education and maintenance of certification credit added on 28 July 2026.

Validation reaches part of it and cuts against the rest. Independent peer reviewed evaluation is strong on structured guideline questions, with 110 cervical spine guideline questions answered at 100 percent citation accuracy and higher guideline alignment than a general assistant. It is weaker where the clinical stakes concentrate: a factorial triage benchmark of 960 prompts found 71.3 percent accuracy on clear cases and over triage on 68.0 percent of nonurgent presentations. So coverage is named at scale and validated in the structured half of it.

Two limits on the coverage itself. The content model is United States centered, citing American guidelines and FDA approvals, which a Canadian physician now inside the free tier should weigh. And the map is North American by contract: the European Union, the United Kingdom and the EEA are excluded by the terms of use, so a coverage claim that reads global is regional.