Open3DLab Says a Roblox AI Crawler Took 145 GB of 3D Models and Ignored robots.txt

TL;DR

On October 15 and 16, a crawler identifying itself as trellis2-data-sourcing/1.0 (3D foundation model research; [redacted]@roblox.com) read 18,926 project pages across Open3DLab, SFMLab and SmutBase, then downloaded 1,027 files from 712 projects, 145.5 GB in all. It fetched robots.txt and ignored it, the sites' terms forbid scraping, 423 of the 712 projects are licensed NonCommercial-only, and 345 of them are adult content. The operators blocked it on October 18, emailed Roblox twice, and heard nothing. The label fits Roblox's published 3D foundation model work, but Roblox has not confirmed it ran the crawler.

What the logs show

The operators, a small team that has run SFMLab since 2014, SmutBase since 2016 and Open3DLab since 2020, published their account on October 26. Every request carried the same user agent and came from a single Amazon Web Services address. Over 34 hours the crawler made 26,429 requests and read the main page of 18,926 projects, which the operators call practically their whole catalogue.

Then it started pulling files. The only thing that slowed it down was the sites' free download tier, which caps speed and concurrent downloads. The crawler saturated those servers for a couple of days and real users felt it. The haul was 1,027 complete files from 712 projects, and 364 of those files (54.8 GB) came from SmutBase, which hosts adult content only.

The crawler went quiet over the weekend. The operators spotted it on Sunday October 18, blocked it that morning, and emailed the Roblox address in the user agent. It came back on October 19, every request was refused, and it did not slow down. It kept what the operators describe as CEST office hours, with an hour off around lunch, which is more work-life balance than most scrapers get. On October 20 it returned at 08:00, tapered its request rate from midday in what the operators read as a probe of their rate limiting, and stopped around 15:00. A second email went to Roblox. Neither got a reply. Tom's Hardware picked the story up on October 27 with the same numbers and the same unanswered emails.

Why the label points at Cube

The operators are careful to say the user agent is a self-description. Anyone can type a Roblox email into a header, and the crawler ran from an AWS address instead of Roblox's own network. They add that nothing suggests impersonation and that the label fits Roblox's published research closely, which is a fair reading.

Roblox announced Cube in March 2025 as its core generative AI system for 3D and 4D, and open-sourced the first shape model on GitHub and Hugging Face. The Cube technical report, 46 authors from Roblox's Foundation AI team, says the shape tokenizer was trained on roughly 1.5 million 3D assets drawn from licensed and public datasets such as Objaverse, plus free Creator Store assets whose creators opted in to training. Character rigging is listed as a target capability, and generating characters and avatars as future work. In February 2026 Roblox put 4D generation into beta on top of Cube: players type a prompt and get a working car with wheels that spin, and the first game to use it, Wish Master, logged over 160,000 generated objects during early access.

The email in the user agent belongs to one of those 46 authors, whose public profile says they build a 3D foundation model at Roblox. The operators redacted the name and say the responsibility sits with the company, not with one person running a job.

Public datasets like Objaverse are a warehouse of loose mannequin parts, mostly untextured and unrigged. A site like SFMLab is the costume department: complete, textured, often rigged characters, in the formats artists open every day, each with a title, a description and preview images attached. That is the paired data a text-to-shape or image-to-3D model is short of, and what a character and avatar generator would want next.

The "trellis2" prefix probably nods at TRELLIS.2, Microsoft's MIT-licensed 4B-parameter image-to-3D model from December 2025. The operators checked its published data toolkit and found no web crawler and no user agent in it, so whoever ran this wrote the crawler and its label for the job. Whether the data was meant to tune a TRELLIS-style model, or "trellis2" is just a project codename, nobody outside Roblox knows.

The adult-site problem

SmutBase describes itself as exclusively for adult content, and all three sites are for adults only. The crawler took adult projects at the same rate as everything else: 224 projects from SmutBase, 121 more marked adult on SFMLab, 345 of the 712 in total, 62.6 GB. Examples the operators list include an Eve model from Stellar Blade, a Mei model from Overwatch, and a sex toy.

The operators do not think Roblox set out to collect adult content, and they have no evidence any of it reached a model. Their point is that nobody checked. The operators note that more than a third of Roblox's daily users were under 13 in 2025 by the company's own figures, that its rules ban sexual content, and the only thing standing between an unfiltered scrape and a model that generates assets for that audience is whatever filter gets built afterwards. One project the crawler took has since been disabled on SFMLab. If Roblox kept its copy, the deletion never reached it.

robots.txt was read, then ignored

The crawler requested the sites' robots.txt and then did the opposite of what it said, and the terms of service that forbid scraping got the same treatment. Compare the other AI crawler in the same logs: in the month before the Roblox hits, Meta's Meta-ExternalAgent, which Meta's own docs say crawls to train foundation models or index content, made about 380,000 requests and loaded 211,000 project pages. The operators have since blocked it too. But the Roblox-labeled crawler was the first one they saw pull the uploaded asset files. Meta's had only read the pages around them.

The Hacker News thread, 455 points at the time of writing, spent most of its energy on defence. One camp blocks every cloud provider's address ranges outright, since the big bots live in AWS, Azure and DigitalOcean. Another answered that most of their bot traffic now arrives from residential proxies pretending to be the same Chrome build, and that blocking cloud ranges locks out real customers behind corporate VPN gateways. Both camps are right, which is why defence is hard: a single-IP, honestly labeled crawler is the easy case, and the operators still needed until Sunday to notice it.

The license fight is messier than the headline

423 of the 712 projects carry a NonCommercial licence, and a Roblox model that ends up in a product is a commercial use by any ordinary reading. The HN thread pushed back on the licence point, and some of it lands. Several commenters pointed out that a large share of what these sites host is fan ports: character meshes extracted from commercial games, uploaded under whatever licence the extractor picked. A licence you had no right to grant is hard to enforce, and if a court calls training fair use the licence terms stop mattering entirely. The operators say as much themselves: they dislike copyright as it exists, comply with studio takedowns, and are not asking for stronger enforcement. Their complaint is narrower, and their example is Jammie Thomas-Rasset, ordered to pay $222,000 for sharing 24 songs, set against companies for which infringement at scale becomes a cost to negotiate after the fact.

One HN commenter clicked a random model and found it licensed as beerware. By that reading Roblox now owes somebody a bar tab.

What Roblox has said

Nothing, as of publication. The operators have put six questions to the company in public: did Roblox run the crawler, what was the data for and has any of it trained or evaluated a model, why were robots.txt and the terms ignored, did anyone review the target list or exclude adult content, will Roblox delete the files and anything derived from them and confirm it in writing, and will it respect NonCommercial licences from now on. They are holding the full log export for an outside audit.

What to take from it

If you run a crawler for a model team, this is a checklist of how not to. Label it honestly, which this one did, then honour the robots.txt you fetched, which it did not. Do not point a bulk downloader at a free-tier mirror and let it saturate for days. Review the target list before the run, not after someone publishes your user agent. And answer the email, because the alternative is 455 points on HN and a Tom's Hardware headline.

If you host creator uploads, the operators' playbook is the useful part: log user agents and source IPs per request, alert on single-source request volume, rate-limit the free download tier (it was the only thing that capped this run), keep the raw logs for an audit, and know that robots.txt is a request, not a lock. Whether a licence on a fan-ported mesh will hold up is a question for a court. Whether your servers can take 26,429 requests in 34 hours is one you can answer today.

Key Takeaways

• A crawler labeled trellis2-data-sourcing/1.0 (3D foundation model research; [email protected]) downloaded 1,027 files (145.5 GB) from 712 projects across Open3DLab, SFMLab and SmutBase on October 15 and 16, after reading 18,926 project pages in 26,429 requests over 34 hours.
• It fetched robots.txt and ignored it, the sites' terms forbid scraping, and 423 of the 712 projects are licensed NonCommercial-only.
• 345 of the 712 projects (62.6 GB) are adult content, taken at the same rate as everything else; the operators say nobody appears to have filtered the target list.
• The label fits Roblox's Cube 3D foundation model work and its February 2026 4D generation beta, and the email belongs to a Cube paper co-author, but Roblox has not confirmed it ran the crawler and has not answered two emails or six public questions.
• Meta-ExternalAgent hit the same sites with about 380,000 requests in the prior month but, per the operators, never pulled asset files; both crawlers are now blocked.
• The operators are not asking for stronger copyright. They are asking Roblox to credit people, respect the chosen licence, and ask first.

Sources: Open3DLab: Roblox downloaded 145 GB of models from our sites, Tom's Hardware, Hacker News discussion, Roblox: Introducing Cube (March 2025), Cube: A Roblox View of 3D Intelligence (arXiv), Roblox: 4D generation beta (February 2026), Roblox/cube on GitHub, Microsoft TRELLIS.2, TRELLIS.2 paper (arXiv), Meta web crawlers documentation