Speaker
Description
Analyzing irregular and sparse time-series is a widespread problem in fundamental physics, astronomy, climate science and many other fields. This talk presents the Rotary Masked Autoencoder (RoMAE), a novel Transformer-based architecture for multi-dimensional irregular time-series and sparse data, designed as a foundation model for general time-series interpolation, object classification, and representation learning. Our method consists of a pre-training phase, where the model is trained on interpolation, and a subsequent fine-tuning phase where the learned representation is adapted to down-stream tasks, such as classification. We highlight the performance of our algorithm on a variety of physics datasets including the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC) and simulated recoil events in a liquid xenon time projection chamber for direct dark matter detection. We compare our method to other popular models for irregular time-series such as S5, RoFormer and ATAT, showing that our approach can out-compete the current state-of-the-art.
| AI keywords | transformer, irregular time-series, sparse data |
|---|
