I’ve evaluated a lot of healthcare AI vendors over the years. Some have been excellent partners. Others have been, diplomatically, learning experiences.

The healthcare AI vendor market is maturing, but it still includes companies with amazing demos and questionable substance. Here’s how to tell the difference.

Start with TGA Registration

This should be non-negotiable for any AI making clinical recommendations or diagnoses.

Ask: “What’s your TGA registration status?”

Acceptable answers:

• “We have ARTG registration number [specific number]”
• “We’re registered under the [specific classification]”
• “We’re in final stages with expected registration by [specific date]”

Red flag answers:

• “We’re working on it”
• “Our US FDA clearance covers Australia”
• “We don’t need registration for our application”
• “We’re operating under [vague exemption]”

If a vendor claims they don’t need TGA registration for a clinical AI product, they’re either wrong or describing a product that’s narrower than what you think you’re buying.

Performance Claims Need Context

Every vendor will tell you their AI performs brilliantly. The numbers in marketing materials are almost always accurate—just misleading.

Questions to ask:

“What population was this validated on?” AI trained on US academic medical centre data performs differently on Australian regional hospital data. Ask specifically about validation on populations similar to yours.

“What’s the false positive rate?” Sensitivity (catching true positives) is what vendors highlight. Specificity (not flagging false positives) is what matters for workflow. An AI that catches 99% of cancers but flags 30% of non-cancer cases as suspicious will overwhelm your clinicians.

“How does performance vary by subgroup?” Overall accuracy might mask significant variation. Does the AI perform equally well across age groups, ethnicities, and clinical presentations? If the vendor hasn’t analysed this, that’s concerning.

“What’s the failure mode?” When the AI makes mistakes, what kind of mistakes? Failing to detect (false negative) versus over-detecting (false positive) have different clinical implications.

Reference Sites Are Essential

Don’t accept curated reference lists. Any vendor will give you their happiest customers.

Instead:

• Ask your professional networks who’s using the product
• Contact reference sites the vendor doesn’t suggest
• Ask specifically about implementation challenges, not just outcomes
• Talk to end users (clinicians), not just executives who approved the purchase

Questions for reference sites:

• “What surprised you during implementation?”
• “If you could do it again, what would you do differently?”
• “What’s the ongoing effort required to maintain the system?”
• “Has the vendor been responsive when issues arise?”
• “Would you recommend this product to a colleague?”

The answers to these questions are more valuable than any demo.

Understand the Business Model

AI pricing models vary significantly:

Per-study/per-case licensing. You pay based on usage. Predictable per-case cost, but total cost scales with volume. Good for organisations with uncertain or variable volumes.

Fixed license fee. Annual fee regardless of usage. Better economics at high volumes. Risk if volumes don’t meet expectations.

Infrastructure bundled. Some vendors include required hardware or cloud infrastructure. Understand what’s included and what’s additional.

Implementation costs. These are often underestimated. Get detailed quotes for integration, training, and go-live support.

Ongoing costs. Support, updates, and monitoring have costs. A cheap upfront price with expensive ongoing fees is worse than it looks.

Model out five-year total cost of ownership under different volume scenarios. The cheapest option in year one is often not the cheapest over five years.

Integration Complexity

Healthcare IT is complex. Integration is usually harder than vendors suggest.

Questions to ask:

“What integration interfaces do you support?” HL7 v2, FHIR, direct API—these matter for your architecture. If the vendor doesn’t support your standard interfaces, custom development will be required.

“Have you integrated with our EMR/PACS vendor before?” Direct experience with your specific systems reduces integration risk. Ask for specifics about previous implementations.

“Who’s responsible for integration?” Clear delineation between vendor and customer responsibilities. Get it in writing.

“What’s the typical integration timeline?” Compare their estimate to what reference sites actually experienced.

Data Governance Questions

AI creates data governance obligations. Vendors should be clear about these.

“Where is patient data processed?” Cloud-based? On-premises? Hybrid? If cloud, which jurisdiction? This affects data sovereignty and privacy compliance.

“Do you use customer data for model training?” Some vendors improve their models using customer data. This might be acceptable with appropriate consent and anonymisation, but you need to know.

“What happens to inference data?” AI generates data about its own recommendations. Who owns this? Who can access it? Can it be used for research?

“How do you handle data breaches?” Notification procedures, liability allocation, remediation processes. Get specifics.

Contract Terms to Negotiate

Don’t accept standard contracts without negotiation:

Performance guarantees. Can you include minimum performance thresholds with remedies if not met?

Exit provisions. If you discontinue use, what happens to your data? What’s the transition process?

Update control. Can you control when algorithm updates are deployed? You don’t want surprise changes to clinical systems.

Liability allocation. Who’s responsible when AI contributes to adverse outcomes? This should be clear.

Audit rights. Can you audit the vendor’s security and data handling practices?

Warning Signs

Red flags I’ve learned to watch for:

Evasive about limitations. Good vendors are honest about where their AI doesn’t work well. Vendors who claim their AI works perfectly everywhere are either lying or don’t understand their product.

Minimal local presence. Implementation and support require people. International vendors without Australian presence create risk.

Unusually aggressive pricing. If it seems too cheap, it probably is. Unsustainably low pricing means the vendor may not survive or may increase prices dramatically after you’re committed.

Recent market entry. Healthcare AI is not a space for learning. Prefer vendors with track records over promising startups.

High staff turnover. If the people who sold you the product leave before implementation, that’s concerning.

My Recommendation Process

When I advise organisations on vendor selection:

1. Define requirements precisely before talking to vendors
2. Long-list vendors meeting basic requirements (TGA, integration, geography)
3. Reference check before detailed evaluation
4. Structured demo against specific clinical scenarios you define
5. Site visit to at least one reference site
6. Contract negotiation with appropriate legal review
7. Pilot before full commitment if possible

This takes time—typically four to six months for a significant AI procurement. Rushing leads to regret.

The right vendor makes AI implementation dramatically easier. The wrong vendor can set you back years. Take the time to choose well.

---

Dr. Rebecca Liu is a health informatics specialist and former Chief Clinical Information Officer. She advises healthcare organisations on clinical AI strategy and implementation.