This AI developed into an expert Minecraft player using GPT-4

A Minecraft bot that is able to explore and expand its capabilities in the game’s open world has been developed by AI researchers. However, in contrast to other bots, this one basically wrote its own code through trial and error and a lot of GPT-4 queries.

This experimental system, which is called Voyager, is an example of an “embodied agent,” which is an artificial intelligence that can move and act freely and purposefully in a real or simulated environment. AIs and chatbots that resemble personal assistants don’t have to actually do anything, let alone navigate a complicated world, to complete tasks. However, because that is the very thing that a household robot might be expected to do in the future, there has been a lot of research into how they might accomplish that.

Because it is a very (very) close representation of the real world, Minecraft is a good place to test such things. It has simple rules and physics, but it is also complex and open enough that there is a lot to do or try. Even though purpose-built simulators are excellent, they have their own restrictions.

Because you can’t just put any AI in Minecraft and expect it to understand what all these blocks and pigs are doing, MineDojo is a simulation framework based on Minecraft. Its creators, many of whom are also on the Voyager team, compiled a variety of data, including transcripts from YouTube videos about the game, wiki articles, Reddit posts from r/minecraft, and a great deal of other data, so that users could construct or refine an AI model based on them. It additionally allows those models to be assessed pretty much dispassionately by perceiving how well they do things like form a wall around a llama or find and mine a precious stone.

Voyager outperforms Auto-GPT, the only other model that comes close, in these endeavors. However, they take the same approach: making use of GPT-4 to develop their own code on the fly.

Normally, you would simply train a model on the excellent Minecraft data and hope that it would learn how to fight skeletons at night. However, Voyager has a brief internal conversation with GPT-4 about what it ought to do and how it should do it as it encounters new challenges throughout the game.

For example, when night falls, those skeletons emerge. The agent has a general idea of what’s going on, but it asks itself, “What would a good player of this game do when monsters are nearby?” According to GPT-4, if you want to safely explore the world, you should make and equip a sword, then use it to hit the skeleton without being hit. And the general direction of action is translated into specific objectives: Build a sword at the crafting table out of stone and wood, equip it, and fight a skeleton.

When it completes those tasks, it adds them to a general skill library so that when the task is to “go deep into a cave to find iron ore,” it doesn’t have to learn fighting all over again. It really does in any case utilize GPT, yet it utilizes the less expensive and quicker GPT-3.5, which tells it the abilities generally pertinent to a given circumstance — so it doesn’t attempt to mine the skeleton and battle the metal.

It is comparable to an agent like Auto-GPT that must learn how to navigate an interface it does not yet know in order to achieve its goal. However, a specialty agent like Voyager performs much better because Minecraft is a much more complex environment than it is accustomed to working in. Compared to the other bots, it finds more things, learns more skills, and explores a much larger area.

When it comes to producing code that is useful, GPT-4 outperforms GPT-3.5 (also known as ChatGPT), which is interesting but perhaps not surprising. The agent hit a stumbling block early on, perhaps even literally, and failed to improve in a test that substituted the latter. Although it may not be apparent from speaking with the two models that one is significantly smarter, the truth is that it is not necessary to be particularly smart to engage in conversation that appears to be intelligent (ask me how I know). Coding is a lot harder, and GPT-4 was a big improvement there.

The goal of this study isn’t to make Minecraft players obsolete; rather, it is to find ways for relatively straightforward AI models to improve themselves based on their “experiences,” for want of a better term. Assuming we will have robots helping us in our homes, clinics, and workplaces, they should learn and apply those illustrations to future activities.