TF-JEPA: Predictive Alignment of Time–Frequency Representations Without Contrastive Pairs

Cross-View Prediction in Latent Space as an Alternative to Contrastive Learning

Learning generalizable representations from multivariate time series is challenging due to complex temporal dynamics, distribution shifts, and the difficulty of designing effective contrastive pairs. TF-JEPA is a non-contrastive self-supervised method that leverages predictive alignment to integrate representations from the time and frequency domains without relying on negative sampling.

TF-JEPA utilizes dual online time and frequency encoders, each paired with its own momentum-updated target encoder, embedding both views into a stable and unified latent space. Experiments on sleep EEG, gesture recognition, mechanical fault detection, and EMG classification demonstrate that TF-JEPA matches or surpasses contrastive and time–frequency consistency baselines.

Predictive Alignment Across Time and Frequency

TF-JEPA couples an online time encoder with a momentum-updated frequency encoder and trains them with a lightweight cosine loss. The architecture rests on three design choices:

Dual EMA Targets

Frozen time and frequency target encoders are updated after every step by an exponential moving average (EMA, momentum m = 0.995) of the online weights, providing stable target representations with no gradient overhead.

Lightweight Predictors

Two small MLPs (128→256→128) map each online embedding to predict the corresponding target view. A BYOL-style cosine loss aligns the two domains without negative pairs.

End-to-End Fine-Tuning

Because the objective avoids contrastive collapse, all encoder weights can be unfrozen during downstream training, allowing full adaptation to the target distribution.

Architecture diagram not found — see paper for details.

Cross-Dataset Transfer Performance

Each model is pre-trained on the source dataset and fine-tuned on the corresponding target dataset with identical classifier heads. All experiments were conducted on a single NVIDIA A10 GPU.

Results figure not found.

Foundation Model Comparisons

TF-JEPA is also compared against large-scale foundation models (NormWear, CBraMod) pre-trained on diverse physiological corpora. TF-JEPA uses only a 13.3% subset of TUH EEG v2.0.1, yet remains competitive across all benchmarks.

Foundation model comparison figure not found.

EMA Momentum Analysis

An ablation across six momentum values (0.9–0.9995) shows that higher EMA momentum consistently improves transfer metrics. The HAR→Gesture task shows +11.3 pp gain at m = 0.9995 compared to the lowest momentum, confirming that target network stability is critical for non-contrastive time-series learning.

EMA ablation figure not found.

Resource Usage

Because TF-JEPA replaces the contrastive NT-Xent objective with a non-contrastive BYOL-style cosine loss, it eliminates the quadratic 2B×2B similarity matrix that NT-Xent must materialize and back-propagate through at every pre-training step.

TF-JEPA's forward pass is marginally slower (12.7 ms vs. 11.0 ms) because the momentum-updated target encoder doubles the stored parameters. However, the backward pass is 1.8× faster (14.1 ms vs. 25.0 ms) because the cosine loss produces a compact O(B) gradient graph in place of the O(B²) graph generated by the contrastive similarity matrix — more than compensating for the extra forward-pass cost.

GPU comparison figure not found.

ICLR 2026 — TSALM Workshop Poster

Presented at the 1st ICLR Workshop on Time Series in the Age of Large Models (TSALM). Click to view full resolution.

Poster image not found.

BibTeX

If you find this work useful, please cite our paper:

@inproceedings{chaykowsky2026tfjepa,
  title     = {{TF-JEPA}: Predictive Alignment of Time--Frequency
               Representations Without Contrastive Pairs},
  author    = {Chaykowsky, Michael},
  booktitle = {1st ICLR Workshop on Time Series in the Age of
               Large Models (TSALM)},
  year      = {2026},
  url       = {https://iclr.cc/virtual/2026/10013864}
}