From 805 ms to 23 ms: Accelerating State-Space Models for Real-Time ICU Monitoring

Real-time ICU early-warning systems operate under low-latency constraints, with sub-50 ms targets enabling timely clinical intervention. However, irregular sampling and missing values exceeding 30% in physiological time series force deep learning pipelines into sequential preprocessing routines that consume over 85% of total wall-clock time. We introduce Triton-accelerated state-space models (SSMs) with a time-aware formulation that fuses interpolation and inference into a single GPU kernel, eliminating this bottleneck. Our system reduces end-to-end inference latency by 35.7× (from 805 ms to 23 ms) compared to a PyTorch SSM implementation, trains 2.29× faster than GRU-D baselines, and scales to 10,316 samples per second. Across 5 seeds on PhysioNet Challenge 2012, the Triton SSM improves test AUROC over GRU-D by +0.037 (paired bootstrap 95% CI [0.018, 0.058], p < 0.01); the same kernel ports to MIMIC-III 25-task phenotyping with a +0.024 macro-AUROC improvement. The implementation is hardware-portable across NVIDIA and AMD GPUs.

1The bottleneck is the preprocessing, not the model

ICU data has two properties that sabotage standard GPU pipelines: vital signs are recorded at irregular intervals, and 30% or more of values are missing. Filling the gaps needs a conditional neighbour search and an interpolation that, even vectorized in PyTorch and compiled, degenerates into many small sequential GPU operations. Profiling shows this step alone takes more than 85% of inference time, which makes even a fast model clinically unusable.

For each missing value, we find the nearest valid observations within K = 10 timesteps and interpolate weighted by time. Unlike forward-fill or mean imputation, this preserves local clinical trends, and because every output depends only on a bounded window, it is fully parallel.

2One fused kernel

The whole interpolation runs as a single Triton kernel with one thread block per (batch, time, feature-tile). Each block coalesces a 128-feature load, runs a bidirectional neighbour search of up to K steps with early-exit flags, applies the time-delta weighting, and writes once. Nothing round-trips through global memory in between. The gap-filled tensor then feeds a standard 4-layer state-space model (827K parameters); the contribution is the interpolation front-end, not the SSM scan.

3Speed at every level

The kernel stays nearly flat as sequences grow, while the optimized PyTorch baseline (cuDNN + torch.compile, the most realistic implementation a practitioner would write) grows with length. Every speedup below is against that compiled baseline, not the naive loop.

4Faster, and more accurate than GRU-D

Speed is the point, but a fast model has to hold up. Under matched budgets over 5 seeds, with paired bootstrap confidence intervals and Wilcoxon tests, the Triton SSM beats GRU-D on PhysioNet 2012 while training faster and using fewer parameters.

5Near the hardware ceiling, on both vendors

The kernel is bandwidth-bound (0.30 FLOP/byte) and reaches 1,344 GB/s of effective HBM bandwidth, 75% of peak, against 143 GB/s (8%) for the PyTorch baseline. The remaining gap comes from divergence in the bidirectional search and masked writes at irregular sampling boundaries. The identical Triton source runs on AMD MI300X with no source changes.

6Limitations

1. A test AUROC of 0.66 on PhysioNet 2012 trails state-of-the-art feature-engineered ensembles. This work optimizes the systems substrate, not the predictive model.
2. Linear interpolation assumes local smoothness and can soften abrupt clinical transitions such as acute desaturation or arrhythmia onset. A discontinuity-preserving kernel is future work.
3. The headline interpolation is bidirectional, standard for retrospective benchmarks. A causal variant for streaming uses no future samples at identical cost, for −0.006 AUROC (0.653 vs 0.659, within seed variance).
4. Cross-vendor results use one AMD SKU, and energy figures are steady-state averages from NVML / ROCm-SMI sampling.

BibTeX

@inproceedings{singh2026from,
  title     = {From 805 ms to 23 ms: Accelerating State-Space Models
               for Real-Time {ICU} Monitoring with Fused Triton Kernels},
  author    = {Manpreet Singh},
  booktitle = {ICML 2026 Workshop on Structured Data for Health},
  year      = {2026},
  url       = {https://openreview.net/forum?id=QSJGOuySuH}
}

More research

This is one piece of a broader line of work at Embedded LLM on portable, fused GPU kernels for biological and clinical foundation models, measured on both NVIDIA and AMD.

• COLM 2026Deploying Clinical Language and Vision-Language Models Where the Data Lives
• MOML 2026From 13 Hours to 4.6: Profiling-Driven Kernel Fusion for TensorNet in Molecular Simulation
• ICPP 2026Error-Bounded Fused Attention Compression for Long-Context Genomic Foundation Models Across Heterogeneous GPUs
• ISCA 2026When the LLM-Tuned Stack Misses: An Infrastructure View of Biological Foundation Model Inference Across NVIDIA and AMD
• ISC 2026Portable GPU Kernel Acceleration for Biological Foundation Models & Algorithms using OpenAI Triton
• RECOMB 2026Hardware-Portable Fused GPU Kernels for High-Throughput Biological Foundation Models
• MLSys 2026BioTriton: Portable Cross-Vendor GPU Kernels for High-Throughput Bioinformatics via OpenAI Triton

All research →