16–20 Jun 2025
THotel, Cagliari, Sardinia, Italy
Europe/Rome timezone

🎙️ RoMAE: A Framework for Irregular and Sparse Time-Series Analysis

19 Jun 2025, 16:15
20m
T1c

T1c

Parallel talk Inference & Uncertainty 🔀 Inference & Uncertainty

Speaker

Uros Zivanovic (University of Trieste)

Description

Analyzing irregular and sparse time-series is a widespread problem in fundamental physics, astronomy, climate science and many other fields. This talk presents the Rotary Masked Autoencoder (RoMAE), a novel Transformer-based architecture for multi-dimensional irregular time-series and sparse data, designed as a foundation model for general time-series interpolation, object classification, and representation learning. Our method consists of a pre-training phase, where the model is trained on interpolation, and a subsequent fine-tuning phase where the learned representation is adapted to down-stream tasks, such as classification. We highlight the performance of our algorithm on a variety of physics datasets including the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC) and simulated recoil events in a liquid xenon time projection chamber for direct dark matter detection. We compare our method to other popular models for irregular time-series such as S5, RoFormer and ATAT, showing that our approach can out-compete the current state-of-the-art.

AI keywords transformer, irregular time-series, sparse data

Primary author

Uros Zivanovic (University of Trieste)

Co-authors

Presentation materials

There are no materials yet.