AgileMD

The model is the cleared device and the filing says so directly: the primary functions of the system are imparted by the Gradient Boosted Machine learning algorithm that takes input from the EHR in real time. Remove the model and there is no eCART.

Scoping note, stated so the grade is not read wider than it is. AgileMD also sells a separate Clinical Pathways product, a content and workflow tool with more than 4,800 pathways live, whose asset is curated clinical logic rather than a model. This record is scoped to eCART, following the same product scoping used for Luminare and VirtuSense VSTOne. A buyer evaluating the pathways line alone is looking at a different centrality picture, closer to the content led vendors in clinical decision support.

The second A on this axis in the category, and it rests on a structural property nothing else here has. The cleared indication restricts the model to data a human has already validated. eCART calculates risk only from validated EHR data such as vitals confirmed by a registered nurse, and unvalidated data streaming from monitors and devices is not used until a healthcare professional confirms it. That places a licensed human between the raw signal and the model, which is the inverse of the automation bias problem and a direct answer to the substrate quality question Sickbay raises.

Three further supports. The indication is explicitly adjunctive, stating that predictions are for reference only and that no therapeutic decision should be made solely on an eCART score. The user is shown the observed event rate at their score rather than only a colour, so the clinician calibrates their own response. And there is no low risk or de escalation output, so eCART avoids the invisible failure surface carried by CLEWLR, by Etiometry step down and by Healthplus.ai discharge support, in the same way PeriGen does.

Held honestly rather than uncritically. The moderate threshold fires on 7.5 percent of observations, which is a real volume, and nothing is published about what clinicians actually do with those alerts or how many are dismissed.

Best in this category, and it earns the grade by a different route from Etiometry. The model class is named in the filing, a Gradient Boosted Machine, where the Epic Sepsis Model discloses nothing and CLEW discloses everything except the model class. The score construction is published: a 0 to 100 composite scaled on specificity rather than raw predicted probability. The default alerting thresholds are published, 93 for moderate risk and 97 for high risk.

The decisive property is what the clinician actually sees. The observed rate of deterioration at each score threshold is displayed as odds of deterioration in the next 24 hours, alongside the score itself. That is calibration surfaced at the point of care, and nothing else in this index does it. A risk number a clinician can read as an actual event rate is the practical form of the falsifiability test applied to every model here.

Supporting candour: the group published Less is More in Resuscitation 2021, showing that age, heart rate and respiratory rate alone recover much of the performance, which complicates its own 97 variable pitch. Held short of a clean sweep by two gaps stated on the record. There is no model card, and the 97 input variables are never enumerated.

The architecture is described honestly, including the part most vendors would leave to inference, which is the basis for the grade. The engine is hosted by the vendor and requires no on site hardware, operating behind stated layers of protection with encryption, network security and audit logging.

The material difference from its peers follows directly from that and is worth naming plainly: three comparable vendors in this lane run on infrastructure the hospital provides, so patient data can stay inside the institution, and this one is vendor hosted, so patient data leaves the hospital by design. That is a legitimate choice and it is what makes a rapid deployment credible, and it is a different risk conversation rather than a worse one.

A privacy policy, terms of service and separate organisational terms are published, which is better than most of this category. What is absent is everything that would bound the offsite path: no cloud provider named, no sub processor list, no data residency commitment or region statement, and no retention limit.

The retraining question is live rather than theoretical because the product is on its fifth version, so models have demonstrably been rebuilt and nothing states what they were rebuilt from. Ask which provider hosts it, in which region, for a sub processor list, for retention, and whether customer data develops the model.

The deepest evidence base in this category and one of the deepest in the index, spanning more than fifty peer reviewed publications from 2011 to 2025 in Critical Care Medicine, the American Journal of Respiratory and Critical Care Medicine, JAMA Internal Medicine, JAMA Network Open, Chest, Resuscitation and Annals of Surgery.

Three items carry the grade. First, an outcome study rather than a validation: The Impact of a Machine Learning Early Warning Score on Hospital Mortality, A Multicenter Clinical Intervention Trial, Crit Care Med 2022;50(9):1339-1347. Nothing else in this category has a multicentre intervention trial with a mortality endpoint. Second, prospective external validation, published as Multicenter Development and Prospective Validation of eCARTv5, Crit Care Explor 2025;7(4):e1232, and independently visible in the 510(k) on 205,946 prospective encounters. Third, the score has been used as a comparator by other groups, including Australian evaluations against Between the Flags, Q-ADDS, NEWS and MEWS in Resuscitation 2018;123:86-91 and Resuscitation 2020;153:28-34.

Four caveats, stated rather than implied. The literature is overwhelmingly authored by the founders, who are co inventors. The intervention trial is not randomised. The item the company features first is a conference abstract, Am J Respir Crit Care Med 2025;211:A5678, which is the same imprecision Etiometry and PeriGen show when they lead with abstracts rather than papers. And the retrospective cohort spans 2009 to 2023, fourteen years across which ward practice changed materially.

A privacy policy, terms of service and separate terms and conditions for organisations are all published and linked, which is better than most of this category. None was opened on this pass, and that is recorded explicitly rather than left implicit.

Architecture is described at a useful level: a hosted cloud clinical decision support engine requiring no on site hardware, operating behind multiple layers of protection, encryption and network security, with advanced audit logging and monitoring.

The material difference from its peers is where the PHI travels. CLEW, Etiometry and Sickbay all run on infrastructure the hospital provides, so data can stay inside the institution. eCART is vendor hosted, so patient data leaves the hospital by design. That is a legitimate architecture and it is what makes the days to weeks deployment claim credible, but it is a different risk conversation, and no data residency commitment, region statement or retention limit is published. Open and unaddressed: whether customer data is used to develop or retrain the model, which matters because the product is on its fifth version.

The company states plainly that the AgileMD Cloud CDS engine is HIPAA compliant, which is more than CLEW, AITRICS or Etiometry publish, so this is not Not Rated. But the claim stops at the assertion. No business associate agreement terms, no tier, no execution path, no statement of which entity signs, and no detail on subprocessors for a cloud hosted product that necessarily has them. A bare compliance assertion from a vendor holding PHI for hundreds of hospitals is the floor rather than the standard, and the gap is easy to close.

Something is published, and it is unverifiable in exactly the way this index asks about. The SOC 2 claim does not specify the type. The site displays a SOC 2 logo and states the engine is SOC-II compliant, without saying Type 1 or Type 2, which is the single question this index puts to every SOC 2 claim and which Etiometry answered to earn an A on this axis. A Type 1 report describes controls at a point in time; a Type 2 tests them over a period. They are not interchangeable. No ISO 27001, no HITRUST, no trust centre and no report request path was located.

Two genuine credits, both rare in this category. The company publishes a vulnerability disclosure policy, which almost nothing in this index does and which signals a real security function rather than a compliance badge, and it operates a public system status page. Specifying the SOC 2 type would move this grade immediately.

K233253, decided 21 June 2024, 21 CFR 870.2210 Adjunctive Predictive Cardiovascular Indicator, Class II, product code QNL, the same regulation and product code as CLEW. The predicate device is CLEWICU, K200717, so this product was cleared against another vendor in this same category, and the reference device is PeraServer and PeraTrend, K172959, the Rothman Index.

The filing answers all three questions this index puts to a clearance, which is why the grade is A and why it reads as the mirror image of AITRICS. Does the device description match the marketing? Yes, the cleared device is the gradient boosted machine learning model the company sells, not a rule based dashboard sitting behind it. What settings are excluded? Hospitalised ward patients aged 18 and over, which is a precise and genuinely narrow indication rather than an open ended deterioration claim. Was clinical data required? Yes, and extensively, with retrospective validation on 1,769,461 encounters and prospective validation on 205,946 further encounters across three geographically distinct health systems.

One thing a buyer should ask about rather than assume. The 510(k) summary describes no Predetermined Change Control Plan, so unlike CLEW there is no published, regulator reviewed commitment governing how the model may be updated after purchase.

The first race stratified performance disclosure in this category, and the benchmark every other vendor here should now be asked to match. Table D of the 510(k) summary reports AUROC, sensitivity, specificity, PPV and NPV at both alerting thresholds for five race categories, with unique patient counts: American Indian or Alaska Native, Asian or Mideast Indian, Black or African American, Native Hawaiian or Other Pacific Islander, and White or Caucasian. Comorbidity subgroups for congestive heart failure, COVID-19, chronic pulmonary disease and sepsis are reported the same way. AITRICS previously set the category benchmark with a subgroup floor across age and sex only and explicitly no race or ethnicity, while Epic, CLEW and Etiometry publish none at all.

The grade is for publishing, not for the result, and the result deserves reading. That is the same separation applied to the Epic record. The vendor states that all five groups met the performance thresholds, and that is accurate. It is also true that sensitivity at the moderate threshold runs from 46.6 percent for American Indian or Alaska Native patients and 48.1 percent for Black or African American patients up to 56.5 percent for Native Hawaiian or Other Pacific Islander patients, so the two lowest values fall on groups already subject to access disparities, and the smallest groups rest on 1,427 and 3,282 unique patients, which leaves their intervals wide.

A second exposure is unaddressed anywhere. The outcome is death or ICU transfer, and transfer is a decision rather than a physiological fact, so differential willingness to transfer particular patients is learned as signal. Healthplus.ai and Kinometrix carry the same action defined label problem. Credit also for a candid published limitation, that comparative effectiveness against existing or future SARS-CoV-2 strains is unknown.

This is the strongest record in the index on one specific property: calibration surfaced at the point of care. The observed rate of deterioration at each score threshold is displayed to the clinician as odds of deterioration in the next twenty four hours, alongside the score itself. Nothing else assessed here does that.

A number a clinician can read as an actual event rate rather than as a rank is the practical form of the falsifiability this axis tests for, and it turns a score into a claim the bedside can check against what it subsequently sees.

Around it: the model class is named in the regulatory filing, the score construction is published as a bounded composite scaled on specificity rather than raw predicted probability, and the default alerting thresholds are published as specific numbers, so a hospital knows exactly where the line sits before it deploys. One further disclosure is candour of a rare kind.

The group published work showing that a handful of variables recover much of the model's performance, which complicates its own multi variable pitch and gives a buyer an argument against its own product. Held below the top grade because no warranty, indemnity or remediation commitment attaches, no model card exists, and the input variables are never enumerated. Ask for the variable list, and whether the published thresholds and observed rates are re measured per site.

Functionally deep and evidentially thin, which is the same shape that held Sickbay at B. The product is embedded in native EHR workflows rather than presented as a separate application, and the pathways layer does real bidirectional work: ordering directly from within a pathway, aligning order sets and other decision support, embedding live labs and vitals from the record, generating clinical documentation and integrating clinical calculators. Deployment is claimed in days to weeks without heavy IT involvement, which is only credible with genuine integration depth.

The gap is that no EHR vendor is named anywhere. No Epic or Oracle Health partnership is stated, no FHIR or SMART on FHIR support is described, and no marketplace or Toolbox listing was located, which matters more since the Connexall record established Epic Toolbox designation as the stronger credential. The 510(k) states only that the health system must supply an EHR connection and data interfaces.

Naming the integrations would move this to A immediately, given the install base.

The broadest named install base in this category by a wide margin. Twenty health systems are named on the site including Advocate Aurora, BJC, Children's Hospital Colorado, Children's Nebraska, Corewell Health, Johns Hopkins Medicine, Mercy, MUSC, Penn Medicine, UCHealth, UChicago Medicine, UC Health Cincinnati, UF Health, UNC, UVA Health, UW Health, Vanderbilt and Yale. It is cloud hosted with no on site hardware, and deployment is claimed at scale in days to weeks rather than months to years, which is a real differentiator in a category where every competitor needs a bespoke real time data feed built first. Reporting is offered at health system, facility, unit and provider level.

Two things hold it at B. The install base count is internally inconsistent across the vendor's own material: the site meta description says more than 200 US hospitals while the statistics block on the same page says 400 plus, and the 2024 clearance announcement said over 250. Publication counts run 80, 85 plus and 88 plus across the same sources. Inconsistent self reported figures bite hardest on a vendor whose pitch is evidentiary rigour, and Etiometry shows the same problem.

Second, nothing is published on implementation shape, readiness gates, training or data residency, which is what earned Etiometry an A here.

No pricing published at any level. There is no rate card, no stated unit of pricing such as per bed, per monitored patient day or per hospital, no indicative band, and no implementation or integration fee. Every route leads to a demo request form.

The asymmetry is the finding. The company presented a conference abstract titled Measuring the Return on Investment of an Artificial Intelligence Early Warning System, Circulation 2023;148(Suppl 1: Abstract 350), and markets decreased mortality, reduced length of stay and lower direct variable costs. It has quantified the customer's return in public while publishing nothing at all about the cost side of that same calculation. Droxi, Healthplus.ai, Kinometrix and HealthLeap take the same posture, so a buyer working across this category should expect to price all five through a sales process.

Two cost factors worth raising directly. Whether the eCART licence and the Clinical Pathways licence are separate lines, and whether pathway authoring and content maintenance are included or billed as services.

The cleared scope is precise and narrow, which this index credits: hospitalised ward patients aged 18 years or older. That is an explicit negative scope statement ruling out paediatric use and, like AITRICS, it places the product on the general ward rather than in the ICU, which is the harder problem because ward patients are measured intermittently rather than monitored continuously. Comorbidity coverage is evidenced rather than asserted, with published subgroup performance for sepsis, congestive heart failure, chronic pulmonary disease and COVID-19, and the wider literature adds validations in postoperative surgical inpatients and long term acute care hospitals.

A question a buyer should put directly, and it is a milder relative of the cleared versus marketed gap found at AITRICS and Biobeat. The FDA indication covers adults only, yet the company publishes a paediatric implementation outcome study in Pediatric Critical Care Medicine 2025;26(2):e146-e154, names two children's hospitals among its customers, and its pathways product covers neonatal topics. Establish which product a paediatric site is actually running and on what regulatory basis.

Held at B rather than A because validation sits in three health systems, all of which appear to be academic, so community and safety net performance is unestablished, which is exactly the gap the Epic county emergency department study exposed.