3D-GPT can create any digital world from text

Researchers from various universities created an artificial intelligence that can generate 3D environments from text prompts. They designed 3D-GPT as a more efficient and intuitive way to create 3D graphics than 3D modeling methods. As a result, it could revolutionize architecture, city planning, game design, and other industries that involve creating digital spaces.

Imagine the possibilities of having a refined version of 3D-GPT. You can command this AI program to produce any three-dimensional space your mind can conjure. Consequently, you might be able to create your first video game much easier and quicker than ever. It could facilitate making city plans so that engineers can refine a draft instead of making a fresh one.

This article will discuss how 3D-GPT functions and explain its potential. Later, I will cover other AI projects similar to this 3D environment creator.

How does 3D-GPT work?

This three-dimensional environment generator is a collaborative project of the Australian National University, the University of Oxford, and the Beijing Academy of Artificial Intelligence. They published their findings on arXiv.

That online paper says 3D-GPT offers a more intuitive and efficient method of producing 3D assets than conventional workflows. It can “dissect procedural 3D modeling tasks into accessible segments and appoint the apt agent for each task.”

The program uses multiple AI agents to understand text prompts and turn them into three-dimensional objects. Here are 3D-GPT’s artificial intelligence components:

• Task dispatch agent: analyzes text instructions to determine appropriate AI functions
• Conceptualization agent: adds details missing from the initial description
• Modeling agent: sets parameters and generates code for 3D software

The AI program works with popular 3D modeling software like Blender. “It enhances concise initial scene descriptions, evolving them into detailed forms while dynamically adapting the text based on subsequent instructions,” the paper explained.

Researchers tested it with prompts like “a misty spring morning, where dew-kissed flowers dot a lush meadow surrounded by budding trees.” In response, the program generated 3D scenes with nearly realistic graphics that accurately reflected descriptions.

I would describe the environments as slightly polished Minecraft worlds. It generates blocky trees with leaves convincing from afar. However, closer inspection reveals low-resolution pixels and unnatural leaf clumps.

You may also like: The 10 best games online to play with friends

As mentioned, 3D-GPT could facilitate 3D modeling. Previously, you had to build models from scratch, taking time to make your models’ initial shape and size and painstakingly adding details.

This program could create a rough draft of your model, so you only need to refine it. As a result, you could significantly reduce work time to boost productivity.

VentureBeat says researchers admitted that 3D-GPT is still in its early stages. Nevertheless, they continue to improve the tool to open new possibilities.

Other AI projects

A fledgling video game company created a project similar to 3D-GPT. In April 2023, AI company Charisma created a title called Project Electric Sheep.

Both produce 3D environments, but the firm designed Electric Sheep to generate dream depictions. The game starts in a grey void with a red bed in the middle. 

Interact with the bed, and it will ask, “What do you wish to dream about?” Type a description of your dream, and the game will produce a digital world using ChatGPT, the popular AI bot. Soon, AI programs similar to 3D-GPT will enable the following game development features:

You may also like: New 3D printing method uses coffee

• User-generated content: Players could help produce more content for a game so they can enjoy it further. For example, Roblox will release a generative AI asset creator soon.
• Live ops: Bain says generative AI could improve community management and player support.
• Story generation and nonplayable characters (NPCs): Generative AI could enable limitless interactive stories instead of assigning canned dialogue to NPCs. For example, Nvidia demonstrated how its AI model enabled a chef character to engage the player in spontaneous conversation.
• Game assets: Generative AI could generate whole game worlds instead of several artists making them. For example, TrustedReviews said the latest Bethesda title, Starfield, used AI procedural generation for its planets.

Conclusion

Researchers from various universities produce an artificial intelligence that produces three-dimensional spaces. As a result, it could greatly facilitate 3D asset creation.

It could help game developers and city planners create their models faster. Also, it might be appropriate to encourage people to start their first 3D project.

Learn more about the 3D-GPT artificial intelligence software from the arXiv research paper. Also, check out more about digital tips and trends at Inquirer Tech.