On Friday, Roblox announced plans to introduce an open-source generative AI tool that will allow game creators to create 3D environments and objects using text prompts, MIT Tech Review reports. The feature, which is still in development, could streamline the process of creating game worlds on the popular online platform, potentially opening up more aspects of game creation to those without strong 3D design skills.
The system will also reportedly allow users to make modifications, such as changing the time of day or swapping entire landscapes, and Roblox says the multimodal AI model will eventually accept 3D videos and directions, not just text.
A video showing Roblox’s generative AI model in action.
The 3D environment generator is part of Roblox’s broader AI integration strategy. The company reportedly uses around 250 AI models across its platform, including one that monitors voice chat in real time to enforce content moderation, which isn’t always popular with players.
Predicting the next token in 3D
Roblox’s fundamental 3D model approach involves a custom next-token prediction model, a foundation similar to the large language models (LLMs) that power ChatGPT. Tokens are chunks of text data that LLMs use to process information. Roblox’s system “tokenizes” 3D blocks by treating each block as a numerical unit, allowing the AI model to predict the most likely next 3D structural element in a sequence. Together, the technique can build entire objects or scenarios.
Anupam Singh, vice president of AI and growth engineering at Roblox, told MIT Tech Review about the challenges in developing the technology. “Finding high-quality 3D information is hard,” Singh said. “Even if you get every dataset you can imagine, to be able to predict the next cube it needs to literally have three dimensions: X, Y, and Z.”
According to Singh, the lack of 3D training data can lead to flaws in the results, such as a dog with too many legs. To address this, Roblox is using a second AI model as a sort of visual moderator to spot errors and reject them until the right 3D element comes along. Through iteration and trial and error, the first AI model can create the right 3D structure.
Notably, Roblox plans to open-source its 3D base model, allowing developers and even competitors to use and modify it. But it’s not just about giving something back – open source can be a two-way street. Choosing an open-source approach could also allow the company to use the knowledge of AI developers if they contribute to the project and improve it over time.
The constant search to capture gaming revenue
News of the new 3D base model came at Roblox’s 10th annual developer conference in San Jose, California, where the company also announced an ambitious goal of capturing 10 percent of global gaming content revenue through the Roblox ecosystem, and the introduction of “Party,” a new feature designed to facilitate group play among friends.
In March 2023, we detailed Roblox’s early foray into AI-powered game development tools, as revealed at the Game Developers Conference. The tools included a beta version of Code Assist for generating simple Lua functions from text descriptions and a material generator for creating 2D surfaces with associated texture maps.
At the time, Roblox Studio Head Stef Corazza described these as initial steps toward “democratising” game creation with plans for AI systems now coming to fruition. The 2023 tools focused on discrete tasks like code snippets and 2D texturing, laying the groundwork for the more comprehensive 3D foundational model announced at this year’s Roblox Developer Conference.
The upcoming AI tool could streamline content creation on the platform, possibly accelerating Roblox's path to its revenue goal. “We see a powerful future where Roblox experiences will have extensive generative AI capabilities to power real-time creation integrated with gameplay,” Roblox said in a statement. “We will deliver these capabilities in a resource-efficient way, so we can make them available to everyone on the platform.”