TL;DR OpenLongTail is an open-scaling generative data engine that converts heterogeneous long-tail driving videos into pose-grounded, multi-view data, enabling scalable VLA policy learning from heterogeneous sources.
Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing context. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Plücker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.
From an observed front-view video, OpenLongTail synthesizes five non-front views under a target camera rig. Toggle each card between Generated and Ground Truth. The front input view is always the original observation.
Closed-Loop Evaluation in AlpaSim
Fine-tuning Alpamayo-R1 with OpenLongTail-synthesized multi-view data improves closed-loop driving robustness on 53 long-tail events, on par with training on ground-truth multi-camera capture. AS = AlpaSim Score (higher is better); CR = Collision Rate (lower is better).
| Model / Training data | Complex Int. | Cyclists | Uncommon Veh. | Work Zone | Average | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| AS↑ | CR↓ | AS↑ | CR↓ | AS↑ | CR↓ | AS↑ | CR↓ | AS↑ | CR↓ | |
| Alpamayo R1 | 0.457 | 71.4% | 0.534 | 50.0% | 0.575 | 50.0% | 0.683 | 50.0% | 0.534 | 58.8% |
| Alpamayo 1.5 | 0.469 | 75.0% | 0.347 | 100.0% | 0.517 | 33.3% | 0.731 | 50.0% | 0.501 | 61.1% |
| + SFT (10K Rand) | 0.659 | 25.0% | 0.670 | 0.0% | 0.733 | 0.0% | 0.936 | 0.0% | 0.716 | 11.1% |
| + SFT (10K + NV-OOD GT) | 0.743 | 0.0% | 0.688 | 0.0% | 0.758 | 0.0% | 0.938 | 0.0% | 0.764 | 0.0% |
| + SFT (10K + NV-OOD Syn) Ours | 0.747 | 0.0% | 0.662 | 0.0% | 0.717 | 0.0% | 0.935 | 0.0% | 0.748 | 0.0% |
| + SFT (10K + NV-OOD GT + Waymo-E2E GT) | 0.691 | 12.5% | 0.691 | 0.0% | 0.735 | 0.0% | 0.958 | 0.0% | 0.736 | 5.6% |
| + SFT (10K + NV-OOD Syn + Waymo-E2E Syn) Ours | 0.699 | 12.5% | 0.689 | 0.0% | 0.765 | 0.0% | 0.950 | 0.0% | 0.748 | 5.6% |
OpenLongTail-synthesized data (Ours) lifts average AS to 0.748 and reduces the collision rate
to 0.0%, comparable to training on ground-truth multi-view data (0.764 AS).
SFT: supervised fine-tuning · 10K Rand: 10K randomly sampled trajectories ·
NV-OOD: NVIDIA PAV out-of-distribution subset · GT: ground-truth multi-view ·
Syn: OpenLongTail-synthesized multi-view.
OpenLongTail converts external monocular videos (e.g., Waymo E2E) into target-rig multi-view assets, expanding the long-tail training pool. Combining in-house (NVIDIA PAV) and external synthesized data yields more scene-consistent closed-loop rollouts across uncommon vehicles, cyclists, and complex intersections.
Closed-loop bird's-eye-view rollouts of Alpamayo-R1 versus variants fine-tuned with OpenLongTail-synthesized assets from NVIDIA PAV data and from PAV combined with an external source (Waymo E2E). Generative scaling expands useful long-tail coverage rather than merely increasing the number of training samples.
Given a long-tail front-view driving video, OpenLongTail first recovers a motion-consistent metric ego-trajectory and uses it to construct pose-aware geometric conditions. The generation model combines Plücker-ray geometry, temporal depth warping, and a cross-view memory bank within a Wan 2.1-VACE backbone to synthesize synchronized non-front views under the target camera rig.
MapAnything recovers a metric-scale ego-trajectory, stabilized by Kalman + RTS smoothing for jitter-free pose conditioning.
Per-token camera rays encode target-camera geometry so every branch operates in one consistent 3D ray frame.
Depth-based reprojection propagates visible front-view evidence into side and rear targets, even with zero overlap.
A directed memory graph lets each target view condition on generated neighbors for seam-level consistency.
@misc{liu2026openlongtailgenerativescalinglongtail,
title={OpenLongTail: Generative Scaling of Long-Tail Driving Data},
author={Lulin Liu and Nuo Chen and Yan Wang and Bangya Liu and Wenyan Cong and Hezhen Hu and Boris Ivanovic and Hao Wang and Ziyao Zeng and Xinyu Gong and Yang Zhou and Zixiang Xiong and Dilin Wang and Zhangyang Wang and Weisong Shi and Ruohan Zhang and Marco Pavone and Zhiwen Fan},
year={2026},
eprint={2607.09655},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2607.09655},
}