What Remains in Sight? Autoregressive Video Decoding as Representation-Guided Context Rewriting

Published in NeurIPS, 2026

Lixuan He, Jihyeon Je, Yang You, Leonidas Guibas

arXiv PDF Code BibTeX

Abstract

ReCR is a training-free method for long-horizon autoregressive video generation. Under a fixed visible-token budget, it uses internal decoder representations to select context from the generated history, balancing global relevance, redundancy, temporal boundary coverage, and spatial-layout diversity. The selected context is temporally reindexed before decoding the next block, improving long-video consistency across multiple backbones without updating model weights.

BibTeX

@inproceedings{he2026recr,
  title     = {What Remains in Sight? Autoregressive Video Decoding as Representation-Guided Context Rewriting},
  author    = {He, Lixuan and Je, Jihyeon and You, Yang and Guibas, Leonidas},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026}
}

Recommended citation: He, L., Je, J., You, Y., & Guibas, L. (2026). What Remains in Sight? Autoregressive Video Decoding as Representation-Guided Context Rewriting. Advances in Neural Information Processing Systems.