Full Release of the Autonomous Driving Multimodal “CoVLA-Dataset” Open Source Release of the “Terra” Model, Source Code, and World Model Evaluation Benchmark “ACT-Bench”
Turing Inc. (Headquartered in Shinagawa, Tokyo; CEO: Issei Yamamoto, hereinafter “Turing”) has publicly released the CoVLA-Dataset, the world’s largest multimodal dataset for autonomous driving. This comprehensive dataset is now available to support innovation and research in the field of autonomous driving. Turing has also released the model files and source code for the generative world model “Terra,” which allows vehicles to follow designated trajectories, and “ACT-Bench,” which can evaluate how accurately a world model can follow given action directives.
Release Background
Multimodal large language models (MLLMs) are a crucial technology in the pursuit of fully autonomous driving systems capable of responding to complex and unforeseen situations. These models provide advanced decision-making by leveraging data such as images and text.
A significant bottleneck in this field is the lack of large-scale annotated datasets for AI training and the scarcity of available research on the application of End-to-End (E2E) autonomous driving systems for route planning. To address these challenges, Turing has released the CoVLA-Dataset, the world’s largest autonomous driving multimodal dataset with over 6 million frames, as open data to accelerate research and innovation. Turing has also introduced “ACT-Bench,” the world’s first benchmark based on the generative world model “Terra,” which enhances the ability to follow designated future trajectories. This release will allow researchers and developers to advance autonomous driving technologies and provide an objective metric to evaluate the performance of world models in following action directives.
ACT-Bench and Terra are explained in detail in the newly-released paper “ACT-Bench: Towards Action Controllable World Models for Autonomous Driving.” The model files and source code are also publicly available.
| CoVLA-Dataset | https://huggingface.co/datasets/turing-motors/CoVLA-Dataset |
|---|---|
| Terra | https://huggingface.co/turing-motors/Terra |
| ACT-Bench | https://turingmotors.github.io/actbench/ |
About the “CoVLA-Dataset”
The CoVLA-Dataset is the world’s largest VLA (Vision-Language-Action) dataset, integrating sensor data from vehicles, control information, scene language captions, and future vehicle trajectories for each frame. The research paper describing this dataset has been accepted for presentation at the WACV 2025 international computer vision conference. We are publicly releasing the entire 6-million frame dataset to advance technological innovation in autonomous driving.
About “Terra”
“Terra” is a generative world model capable of understanding complex situations, such as real-world physical laws and interactions between objects, and generating realistic driving scenes in video format. Additional training with the CoVLA-Dataset enhanced its ability to generate videos in response to specified future trajectories (action directive adherence). The release of the model files and source code will make it easier for companies and research institutions to develop world models that are adaptable enough to follow action directives.
About “ACT-Bench”
ACT-Bench is a benchmark designed to quantitatively evaluate a world model’s ability to follow action directives. Traditional evaluations of these technologies focused on video realism or performance at specified tasks, with no objective criteria for measuring fidelity to the provided action directives. This benchmark will assist researchers and developers in obtaining objective metrics to assess the action-following performance of world models.
Future Developments: Tokyo30 and Public Road Testing
As part of the “Tokyo30” project, Turing is developing an autonomous driving system capable of navigating Tokyo’s streets for 30 minutes without human intervention by 2025. Turing initiated public road testing for its autonomous driving model “TD-1” in December 2024. We will continue to drive innovation in autonomous driving with technologies like the CoVLA-Dataset and Terra, with the goal of developing fully autonomous vehicles without steering wheels by 2030.
Company Overview: Turing Inc.
Company Name: Turing Inc.
Location: 4th Floor, East Tower, Gate City Osaki, 1-11-2 Osaki, Shinagawa-ku, Tokyo
Representative: CEO Issei Yamamoto
Founded: August 2021
Business: Development of fully autonomous driving technology
URL: https://tur.ing/
Career Opportunities
Turing is seeking individuals who are eager to change the world by making fully autonomous driving a reality. We frequently host company introduction events and autonomous driving experience sessions, so feel free to reach out to us.
Careers Page: https://tur.ing/jobs
Event information: Connpass
Media Inquiries
PR Contact (Hiraku Abe): pr@turing-motors.com