Minecraft:Project Malmö

File:Projectmalmoimage1.png

Malmö and a human perspective

Project Malmö is an open-source platform developed by Template:W for experimenting with and researching artificial intelligence in Minecraft: Java Edition. Rather than being a conventional Minecraft mod for players, Malmo provides an environment in which AI agents can interact with Minecraft, receive observations and rewards, and perform actions in specially designed missions.<ref>Template:Cite</ref>

Project Malmö was designed to use the open-ended nature of Minecraft as an environment for studying challenging AI problems. Agents can be programmed to navigate environments, collect items, fight mobs, craft objects, build structures, and cooperate with other agents. The project includes both the Minecraft modification and APIs and tools for controlling Minecraft from external programs.

History

Project Malmö was developed by Microsoft Research and publicly released in 2016. It was intended to provide researchers with a flexible environment for artificial intelligence experimentation using Minecraft.

The project evolved through numerous releases. Early versions focused on basic mission control, observations and agent commands, while later releases added features such as improved recording, human-in-the-loop experiments, additional observation types, multi-agent functionality and Python environments.

The project eventually reached version 0.37.0. The repository also contains the separate MalmoEnv implementation, which provides a Python environment with an OpenAI Gym-like interface and communicates directly with the Minecraft client.<ref>Template:Cite</ref><ref>Template:Cite</ref>

How it works

The central concept of Project Malmö is the Mission. A mission describes an AI task and its Minecraft environment. Missions are commonly defined using XML.

A mission can specify:

• the Minecraft environment and world generation;
• the starting position and inventory of an agent;
• blocks, items, entities and structures in the world;
• observations available to the agent;
• commands the agent is allowed to perform;
• reward handlers that determine positive or negative rewards;
• conditions that end the mission;
• multiple agents and their roles; and
• recording and other experiment settings.<ref>Template:Cite</ref>

After a mission has been created, an agent connects to Minecraft and interacts with the environment. The agent sends commands to Minecraft and receives observations and rewards in return.

The Minecraft side of Malmo is implemented using Minecraft Forge, while communication between the agent and Minecraft uses network sockets and JSON-formatted messages.<ref>Template:Cite</ref>

Agents

Agents are external programs that control a Minecraft player. Project Malmö supports several programming languages through its APIs and bindings. The repository contains examples and tools primarily focused on Template:W, Template:W, Template:W and Template:W in later versions.

An agent can send commands such as movement, turning, attacking, using items and interacting with the Minecraft environment. The available commands depend on the command handlers enabled by the mission.

For example, the MalmoEnv implementation supports basic actions such as:

• `move`
• `turn`
• `use`
• `attack`

Other command handlers can expose additional actions, including strafing, jumping, inventory interaction and human-level keyboard and mouse controls.<ref>Template:Cite</ref>

Observations

Agents do not have to rely solely on the rendered Minecraft screen. Missions can provide structured observations describing the environment.

Available observation types include:

• the agent's position and orientation;
• nearby blocks;
• the agent's inventory and hotbar;
• entities;
• chat messages;
• distances to locations;
• ray-tracing information;
• recent commands;
• navigation information;
• video frames;
• depth maps; and
• Minecraft system information.

The observation system allows researchers to decide exactly what information an AI agent receives, which makes it possible to create controlled experiments.<ref>Template:Cite</ref>

Rewards

Rewards are used to tell an agent whether its behaviour is desirable.

For example, a mission can reward an agent for reaching a particular location, collecting an item, defeating a mob or completing a construction task. Negative rewards can also be used to discourage unwanted behaviour.

This allows Malmo to be used for reinforcement-learning experiments, where an agent learns which actions lead to higher rewards.

Missions

Missions can range from simple navigation tasks to complex multi-step objectives.

Examples included with Project Malmö demonstrate tasks such as:

• navigating around obstacles;
• avoiding hostile mobs;
• collecting resources;
• crafting items;
• following routes;
• solving mazes;
• building structures;
• copying structures;
• interacting with other agents; and
• human-controlled or human-in-the-loop experiments.

One example mission requires an agent to gather ingredients and craft rabbit stew. The agent receives rewards for collecting the required items and a larger reward after successfully creating the final item.<ref>Template:Cite</ref>

World generation

Mission XML can create or modify the Minecraft environment before an experiment starts. Malmo supports features such as custom world generation, fixed or random seeds, mazes, structures and other decorators.

For example, the mission schema contains support for a default world generator, maze generation and build-battle environments. World settings can also control whether a generated world is reset or destroyed after use.<ref>Template:Cite</ref>

Multi-agent experiments

Project Malmö supports missions involving multiple agents. Different Minecraft client instances can be associated with different agent roles, allowing researchers to investigate cooperation and competition between agents.

MalmoEnv uses the first agent's Minecraft instance as a coordinator for multi-agent missions. Additional agents can connect using separate ports.<ref>Template:Cite</ref>

MalmoEnv

MalmoEnv is a Python implementation of a Gym-like environment for Project Malmö. Unlike the native Malmo implementation, MalmoEnv is written in Python and communicates directly with the Java Minecraft client.

It provides an interface suitable for reinforcement-learning environments and supports both single-agent and multi-agent missions.

MalmoEnv can be installed as a Python package using:

<syntaxhighlight lang="sh"> pip3 install malmoenv </syntaxhighlight>

The repository also provides example missions and scripts for running single-agent and multi-agent experiments.<ref>Template:Cite</ref>

Recording and datasets

Project Malmö includes functionality for recording experiments. Recorded data can contain video frames, observations and other information useful for analysing an agent's behaviour or creating datasets.

Later versions added the ability to save video streams as individual image frames, including colour and depth data. These features were particularly useful for machine-learning and imitation-learning experiments.<ref>Template:Cite</ref>

Malmo also contains tools for working with recordings and for creating datasets from human-controlled Minecraft sessions.

Human-in-the-loop experiments

Project Malmö includes support for human-controlled experiments. The human action space can record mouse and keyboard events performed through the Minecraft window.

A researcher can therefore combine human behaviour with AI agents, allowing experiments involving demonstrations, imitation learning and human-agent interaction.<ref>Template:Cite</ref>

Setup

Downloading

1. Visit the Project Malmö GitHub repository.

1. Download a compatible pre-built release from the releases page.

1. Alternatively, clone the repository if building Malmo from source.

The source-code archive downloaded directly from GitHub is not equivalent to a pre-built release. Users who download the source repository may need to build the project and its dependencies themselves.<ref>Template:Cite</ref>

Running

The repository also provides a Python environment and scripts for starting Minecraft and running sample missions.<ref>Template:Cite</ref>

Development

Project Malmö is open source and licensed under the Template:W.

The repository contains several major components, including:

• Minecraft – the Minecraft/Forge side of the platform;
• Malmo – the native Malmo agent API;
• MalmoEnv – the Python environment implementation;
• Schemas – XML schemas describing mission configuration;
• sample_missions – example missions;
• scripts – development and utility scripts; and
• ALE_ROMS – additional experiment-related resources.<ref>Template:Cite</ref>

Compatibility

Project Malmö is tied to specific Minecraft and Forge versions rather than being a modern version-independent Minecraft mod. For example, release 0.30.0 updated the project from Minecraft 1.8 to Minecraft 1.11, requiring corresponding changes to Minecraft and Forge.<ref>Template:Cite</ref>

Because of these dependencies, modern Minecraft versions are not automatically compatible with older Malmo releases.

See also

• Template:W
• Minecraft: Java Edition
• Reinforcement learning
• Minecraft Forge
• Minecraft mods

Gallery

• Malmö first-person perspective and symbolic view
  Malmö first-person perspective and symbolic view