OpenLongTail

Generative Scaling of Long-Tail Driving Data

1Texas A&M University   2NVIDIA   3University of Wisconsin–Madison   4The University of Texas at Austin   5Yale University   6Adobe   7Meta   8University of Delaware   9Stanford University
*Equal contribution    Corresponding author

Abstract

TL;DR OpenLongTail is an open-scaling generative data engine that converts heterogeneous long-tail driving videos into pose-grounded, multi-view data, enabling scalable VLA policy learning from heterogeneous sources.

Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing context. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Plücker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.

From a Single View to a Full Surround Rig

InputFront
GeneratedCross-Left
GeneratedCross-Right
GeneratedRear-Left
GeneratedRear-Right
GeneratedRear-Tele

OpenLongTail Improves VLA Driving Policies

Closed-Loop Evaluation in AlpaSim

Fine-tuning Alpamayo-R1 with OpenLongTail-synthesized multi-view data improves closed-loop driving robustness on 53 long-tail events, on par with training on ground-truth multi-camera capture. AS = AlpaSim Score (higher is better); CR = Collision Rate (lower is better).

Model / Training data Complex Int.Cyclists Uncommon Veh.Work Zone Average
AS↑CR↓ AS↑CR↓ AS↑CR↓ AS↑CR↓ AS↑CR↓
Alpamayo R10.45771.4%0.53450.0%0.57550.0%0.68350.0%0.53458.8%
Alpamayo 1.50.46975.0%0.347100.0%0.51733.3%0.73150.0%0.50161.1%
+ SFT (10K Rand)0.65925.0%0.6700.0%0.7330.0%0.9360.0%0.71611.1%
+ SFT (10K + NV-OOD GT)0.7430.0%0.6880.0%0.7580.0%0.9380.0%0.7640.0%
+ SFT (10K + NV-OOD Syn) Ours0.7470.0%0.6620.0%0.7170.0%0.9350.0%0.7480.0%
+ SFT (10K + NV-OOD GT + Waymo-E2E GT)0.69112.5%0.6910.0%0.7350.0%0.9580.0%0.7365.6%
+ SFT (10K + NV-OOD Syn + Waymo-E2E Syn) Ours0.69912.5%0.6890.0%0.7650.0%0.9500.0%0.7485.6%

OpenLongTail-synthesized data (Ours) lifts average AS to 0.748 and reduces the collision rate to 0.0%, comparable to training on ground-truth multi-view data (0.764 AS).
SFT: supervised fine-tuning  ·  10K Rand: 10K randomly sampled trajectories  ·  NV-OOD: NVIDIA PAV out-of-distribution subset  ·  GT: ground-truth multi-view  ·  Syn: OpenLongTail-synthesized multi-view.

Generative Scaling Across Diverse Sources

OpenLongTail converts external monocular videos (e.g., Waymo E2E) into target-rig multi-view assets, expanding the long-tail training pool. Combining in-house (NVIDIA PAV) and external synthesized data yields more scene-consistent closed-loop rollouts across uncommon vehicles, cyclists, and complex intersections.

Closed-loop benefits of generative scaling across sources

Closed-loop bird's-eye-view rollouts of Alpamayo-R1 versus variants fine-tuned with OpenLongTail-synthesized assets from NVIDIA PAV data and from PAV combined with an external source (Waymo E2E). Generative scaling expands useful long-tail coverage rather than merely increasing the number of training samples.

Method Overview

OpenLongTail pipeline overview

Given a long-tail front-view driving video, OpenLongTail first recovers a motion-consistent metric ego-trajectory and uses it to construct pose-aware geometric conditions. The generation model combines Plücker-ray geometry, temporal depth warping, and a cross-view memory bank within a Wan 2.1-VACE backbone to synthesize synchronized non-front views under the target camera rig.

Metric Pose Recovery

MapAnything recovers a metric-scale ego-trajectory, stabilized by Kalman + RTS smoothing for jitter-free pose conditioning.

Plücker Ray Geometry

Per-token camera rays encode target-camera geometry so every branch operates in one consistent 3D ray frame.

Temporal Depth Warp

Depth-based reprojection propagates visible front-view evidence into side and rear targets, even with zero overlap.

Cross-View Memory

A directed memory graph lets each target view condition on generated neighbors for seam-level consistency.

BibTeX

@misc{liu2026openlongtailgenerativescalinglongtail,
      title={OpenLongTail: Generative Scaling of Long-Tail Driving Data},
      author={Lulin Liu and Nuo Chen and Yan Wang and Bangya Liu and Wenyan Cong and Hezhen Hu and Boris Ivanovic and Hao Wang and Ziyao Zeng and Xinyu Gong and Yang Zhou and Zixiang Xiong and Dilin Wang and Zhangyang Wang and Weisong Shi and Ruohan Zhang and Marco Pavone and Zhiwen Fan},
      year={2026},
      eprint={2607.09655},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.09655},
}