Yann LeCun’s New LeWorldModel (LeWM) Research Targets JEPA Collapse in Pixel-Based Predictive World Modeling

LeWorldModel was introduced by a research team including Yann LeCun researchers including Yann LeCun and collaborators introduced LeWorldModel (LeWM).

LeWM trains end-to-end from raw pixels without stop-gradient or frozen encoders, leWM trains end-to-end from raw pixels using two loss terms.

LeWM objective uses two loss terms: a next-embedding prediction loss and the SIGReg regularizer, leWM objective uses prediction loss plus SIGReg regularizer.

Model architecture uses a ViT-Tiny encoder (~5M parameters) and a transformer predictor (~10M parameters), encoder is implemented with a ViT-Tiny and Predictor is a transformer.

Prediction loss computes mean-squared error between predicted and consecutive actual embeddings prediction loss computes MSE between predicted and actual embeddings.