Customer Story
Vital predicts hospital wait times with Chalk’s feature platform
Example screenshot of the ERAdvisor mobile application notifying a patient that their wait time to see a Triage Nurse is 15 to 30 minutes.

Client

Use Case

ERAdvisor: Real-time hospital wait times

Industry

Healthcare

Cloud

AWS

Challenges

  • Experimentation slow, tedious, unreliable
  • Feature changes required deep, specialized knowledge limited to a handful of engineers
  • Third-party data infrastructure

Solutions

  • Now update model 2-4 times / month
  • Up to nine engineers working concurrently
  • Patient data never leaves the cloud

Overview

Vital is redefining patient experience with software that gives more control, clarity, and predictability to emergency department visits and hospital stays. Using advanced AI, Vital transforms complex health record data into easy-to-use, personalized interfaces that inform and engage over one million patients per year.

Hospitals across the U.S. use Vital to improve patient satisfaction, drive growth and patient loyalty, achieve better clinical outcomes, and reduce workload for care teams. However, Vital found that its previous solution’s data infrastructure was not up to the task —they struggled to make updates to their AI models, and spent more time managing data pipelines than innovating on their core product. After deploying Chalk, Vital was able to stop wrangling infrastructure and focus on improving its models, launching new products, and delivering a world-class patient experience.

We probably write about half or a third of the code that we did with our previous solution — and get twice as much done. It’s a magical developer experience. The code is also much simpler. We’ve had developers from other teams come in and immediately understand stuff, which was 100% not the case before. Do more with less code.
Mack Delaney Director of Machine Learning, Vital

Challenges

Vital began their machine learning journey with another managed feature platform which ultimately did not serve their needs. Heavily dependent on Spark and Databricks, the previous solution created challenges for Vital due to its poor developer experience and architectural limitations:

  • Experimentation was a slow, tedious process filled with guess work and unreliable results, which made them unable to re-train their models.
  • Changes to feature pipelines required deep knowledge of Spark and Databricks in addition to the feature store architecture, which was limited to a small number of engineers.
  • Data processing occurred on third-party infrastructure.

Solutions

Vital selected Chalk to address each of these challenges. With Chalk, Vital dramatically increased the pace of its product development.

Rapid iteration and experimentation

One of Vital’s early models was trained on hospital data collected during COVID-19. Unsurprisingly, hospital activity during the early pandemic was vastly different from hospital activity post-COVID-19 vaccine, so it was crucial to re-train models to deliver accurate predictions. Vital recognized the importance of updating their models, but they were unable to release updates with the feature engineering solution they had.

There were several reasons why model releases were challenging. To start, Vital’s engineers needed to manage Spark, Databricks, various custom data pipelines, and the feature store itself. Modifying a single feature required full re-computation of the entire feature view, so data scientists needed to coordinate with infrastructure engineers throughout experimentation.

Even after data scientists were ready to bring an idea to production, they were unable to reuse training feature code for production feature generation, leading to training-serving skew. Re-implementing feature definitions led to inconsistency between training and production, which made it difficult for the team to be confident in the accuracy of production results.

Chalk solved these problems. It unlocked rapid iteration cycles and empowered a broad range of engineers to contribute code. Critically, Chalk enabled users to reuse the same feature code between notebooks, training, and serving. Today, when Vital engineers want to experiment with new feature pipelines, they create branch deployments with a single Chalk command. Later, they can deploy the exact same code to production. Engineers no longer have to manually update Spark and Databricks — updating features is as simple as editing a few lines of code. “Anyone who can write Python can work with Chalk,” says Mack Delany, Director of Machine Learning at Vital.

With Chalk, we can quickly add or remove a feature, test across all models, and then roll out quite quickly. For a team of nine that covers research and engineering for three different product lines, being able to iterate quickly and get improvements out and then move back to our roadmap was like the big shining light.
Te Riu Warren CTO, Vital

Full control of customer data

Vital handles sensitive healthcare information, which is regulated by HIPAA and requires strict data control. Their previous solution required data processing to occur on third-party cloud infrastructure.

When they set out to replace their machine learning partner, they searched for an enterprise-grade solution that could guarantee data would never leave their own cloud infrastructure. Chalk was a natural fit because it was designed with data privacy and security in mind from day one.

  • Chalk is deployed in Vital’s own cloud infrastructure and no production traffic exits Vital’s virtual private cloud (VPC).
  • Real-time data processing, feature caching, and query serving all happen from within Vital’s infrastructure.

By deploying Chalk into a customer’s own cloud infrastructure, customers are able to retain control of their data while still benefiting from Chalk’s best-in-class developer experience.

Outcomes

Vital now uses Chalk to process and serve features powering seven different models in production. These models are used in real-time applications, such as Vital’s ERAdvisor software, which guides patients through emergency room visits with personalized wait times and next steps.

ERAdvisor uses Chalk to provide accurate wait time estimates based on real-time, hospital-specific factors, including each hospital’s latest wait time data. Vital has seen improvements in:

Execution Speed
The team now deploys updated models 2-4 times a month.
Model accuracy
Vital is able to deploy model updates with confidence.
Performance & cost efficiency
Vital is able to serve upwards of 200 model predictions per patient visit with millisecond latency and low costs by leveraging Chalk’s real-time feature resolution and caching.
Data privacy and security
All of Vital’s data storage, feature processing, and serving happens in its own cloud infrastructure, ensuring sensitive patient and healthcare data never leaves its environment.
Developer happiness
Vital’s machine learning team has grown to support 9 engineers concurrently working on changes. Vital engineers from across the company are genuinely excited when they get to work on the Chalk codebase.

Looking ahead

Vital is looking forward to expanding its product offerings with machine learning powered by Chalk. They aim to continue guiding patients through their healthcare journeys even after the emergency room. Vital’s latest offering, AccessAdvisor, helps patients connect with specialized doctors for follow-up care while considering insurance matching, proximity, and relevant experience. Vital will use Chalk to predict the best doctor matches for patients. Additionally, they plan to increase the granularity of their existing guided products, with new predictions such as wait time estimates for lab results.

Build faster with Chalk
See what Chalk can do for your team.