2,000 Servers and a Proxy Network. LAION Got 10 Million Hours.

TL;DR

LAION-BVD is an open video dataset of 80 million downloaded videos totaling 10 million hours. It was built by pulling 4.7 billion candidate URLs out of CommonCrawl, filtering to 1.3 billion links on YouTube, Vimeo, and Dailymotion, and then actually downloading a subset of them with 2,000 virtual servers and a residential proxy network. The paper hit arXiv on August 25 from the University of Tuebingen, LAION, the Juelich Supercomputing Centre, and MPI for Intelligent Systems. The previous largest open video-text corpus, InternVid, holds 760,300 hours. This is roughly a 13x jump. URL lists and captioned subsets are live on Hugging Face; the 80M raw videos require an access request, and the whole thing is research-only with no commercial use.

The funnel, honestly reported

Image datasets are easy to scale because the web hands you the caption for free: alt-text is right there in the markup. Video gives you a URL and a bandwidth bill. That asymmetry is why LAION-5B holds 2.3 billion English image-text pairs while the biggest open video corpus until this week held 7.1 million videos.

The download infrastructure is the part worth reading twice. LAION used all CommonCrawl WAT dumps available as of March 2024, extracted platform links with yt-dlp extractors, and ran the fetch across 2,000 virtual servers coordinated by Celery. To keep the platforms answering, they routed the whole thing through a residential proxy network.

That sentence sits in the methods section under the heading "Video download." Plenty of people scrape at this scale. Very few write down how they got past the rate limiter and then put it in a preprint funded by the German research ministry.

What is actually in the 10 million hours

94 percent of the videos come from YouTube, 4 percent from Vimeo, 2 percent from Dailymotion. Median resolution is 720p. Mean video length is 7.7 minutes, median 3.7 minutes, and upload dates span almost two decades. English is 57 percent of the corpus, followed by Russian at 9 percent and Spanish at 8 percent, so nearly half of it is not English.

From that pool the team randomly sampled 2.4 million videos, threw out anything under 10 seconds or over 30 minutes, and split the rest with PySceneDetect at a threshold of 30, discarding segments with no real frame-to-frame motion. That produced BVD-V-55M: 55 million scene-level clips with aligned audio. A separate pass through ffmpeg scene detection at threshold 0.1 yielded BVD-I-300M, 300 million scene-changing frames for image-text training.

The frames are the sleeper release

Web image corpora are everyone's best shot: the photo somebody chose to publish and write alt-text for. Scene-change frames are the whole contact sheet, including the blurry middle of a turn. The paper reports that these frames carry a visual distribution distinct from standard web image datasets, and CLIP models trained on them still land strong retrieval numbers. If you have been fighting the sameness of alt-text data, that is a new axis.

Every caption came from a small model

None of the 55 million clip captions were written by a human. Video captions come from Qwen3-VL-2B-Instruct served through vLLM, sampling up to 32 frames per clip with the prompt "Describe the video in 20 words or less." Audio captions come from Audio Flamingo 3 with a 10-word budget. Frames were recaptioned with DeepSeek-VL2-tiny.

A 2B captioner with a 20-word cap is the cheap end of the market, and the authors say so. They ran a human audit of 134 captions: 79.1 percent accurate, 18.7 percent containing a localized mistake (wrong action, object, or color), and 2.2 percent substantially wrong. Wrong actions were the most common failure at 17 cases.

It beats InternVid, on the benchmarks they ran

To prove the data trains something, the team pretrained ViCLIP (video-text), CLAP (audio-text), and CLIP (image-text) at several scales. The headline video result is a ViCLIP L-14 average across Kinetics-400, UCF-101, HMDB51, MSR-VTT, and MSVD.

The team also reproduced InternVid-10M-FLT themselves and got a lower number than the original authors reported, then traced it to checkpoint merging. Applying WiSE-FT lifts InternVid-10M-FLT from 58.0 to 60.2 and narrows the gap. BVD-V-50M with the same treatment goes to 62.6. The ordering holds either way, and reporting the reproduction gap instead of quietly using the friendlier number is a small act of integrity worth noting.

Audio is the more interesting result

For CLAP, BVD-A-10M matches or beats LAION-Audio across four model scales. At 431M parameters it scores 46.8 against LAION-Audio's 44.8. But adding curated AudioSet data to LAION-Audio scores 56.6, a full ten points clear of anything BVD produced alone. Ten million hours of ambient web audio does not replace a curated audio corpus. It scales predictably, which is a different and cheaper virtue.

The caveats the paper puts in writing

• Contamination exists and was measured. Unique YouTube IDs in BVD-V-55M overlap MSR-VTT at 5.5 percent (49 of 885) and MSVD at 6.3 percent (33 of 522), K400 at 0.26 percent. Re-evaluating on decontaminated test sets left results comparable.
• No generative evaluation. Everything here is contrastive encoder training. Nobody trained a diffusion video model on it yet, so its value for the thing most people want video data for is unproven.
• No joint audio-visual training. Video-text, audio-text, and image-text were evaluated separately.
• No safety filtering at collection. The authors relied on the platforms' own moderation and say the data may still carry biases and uneven representation across languages and regions.

What you can actually do with it

Right now, without asking anyone: pull laion/BVD-URLs (1.35B rows) and the captioned subsets from Hugging Face, including BVD-V-55M, BVD-A-10M, and BVD-I-300M. The full 80M-video BVD-RAW collection sits behind a request-access gate aimed at research institutions, and the terms of use restrict the dataset to research purposes with no commercial use.

So this is not a free video corpus for your startup. It is a reproducibility unlock for labs that could previously only read papers about proprietary video data. Given that the crawl bottleneck was always compute and engineering rather than availability, having someone publish the pipeline (four stages, all in the GitHub repo) may matter as much as the bytes.

Key Takeaways

• LAION-BVD holds 80M videos and 10 million hours, about 13x the 760,300 hours in InternVid, the previous largest open video-text dataset.
• The pipeline is fully documented: 4.7B CommonCrawl URLs filtered to 1.3B platform links, 130M download attempts at roughly 60 percent success, run on 2,000 servers behind a residential proxy network.
• 94 percent of the corpus is YouTube, English is only 57 percent of it, and Music is 17 percent of all videos by category.
• Captions are machine-generated by small models (Qwen3-VL-2B, Audio Flamingo 3, DeepSeek-VL2-tiny) and a 134-caption human audit found 79.1 percent fully accurate.
• ViCLIP trained on BVD subsets beats filtered InternVid by 3.3 to 4.0 points on a five-benchmark average, but curated AudioSet still beats BVD audio by about ten points.
• URL lists and captioned subsets are open on Hugging Face; the raw video collection is gated, and the license is research-only.

Sources: arXiv:2608.24845 (LAION-BVD paper), LAION-BVD project page, LAION-AI/BVD on GitHub, BVD collection on Hugging Face, LAION-BVD download portal, InternVid (arXiv:2307.06942)