MineWorld is an open-source project from Microsoft that brings real-time world modeling to the Minecraft universe. Designed specifically for research and experimentation, MineWorld predicts and generates the next frame of a Minecraft scene based on an initial image and a selected in-game action.
Built entirely for the Minecraft environment, this tool offers a fresh way to simulate interactive gameplay scenes using a trained world model. It’s perfect for researchers and developers exploring how visual context and player choices can drive realistic scene generation within a controlled domain.
Easy to set up with Python and Conda, MineWorld is a step toward more immersive and responsive virtual environments — all rooted in one of the most iconic sandbox games ever made.
Resource
Hugging Face
Link: https://huggingface.co/microsoft/mineworld
GitHub
Link: https://github.com/microsoft/mineworld
Step-by-Step Process to Install Microsoft Mineworld Locally
For the purpose of this tutorial, we will use a GPU-powered Virtual Machine offered by NodeShift; however, you can replicate the same steps with any other cloud provider of your choice. NodeShift provides the most affordable Virtual Machines at a scale that meets GDPR, SOC2, and ISO27001 requirements.
Step 1: Sign Up and Set Up a NodeShift Cloud Account
Visit the NodeShift Platform and create an account. Once you’ve signed up, log into your account.
Follow the account setup process and provide the necessary details and information.
Step 2: Create a GPU Node (Virtual Machine)
GPU Nodes are NodeShift’s GPU Virtual Machines, on-demand resources equipped with diverse GPUs ranging from H100s to A100s. These GPU-powered VMs provide enhanced environmental control, allowing configuration adjustments for GPUs, CPUs, RAM, and Storage based on specific requirements.
Navigate to the menu on the left side. Select the GPU Nodes option, create a GPU Node in the Dashboard, click the Create GPU Node button, and create your first Virtual Machine deploy
Step 3: Select a Model, Region, and Storage
In the “GPU Nodes” tab, select a GPU Model and Storage according to your needs and the geographical region where you want to launch your model.
We will use 1 x H100 SXM GPU for this tutorial to achieve the fastest performance. However, you can choose a more affordable GPU with less VRAM if that better suits your requirements.
Step 4: Select Authentication Method
There are two authentication methods available: Password and SSH Key. SSH keys are a more secure option. To create them, please refer to our official documentation.
Step 5: Choose an Image
Next, you will need to choose an image for your Virtual Machine. We will deploy Microsoft Mineworld on an NVIDIA Cuda Virtual Machine. This proprietary, closed-source parallel computing platform will allow you to install Microsoft Mineworld on your GPU Node.
After choosing the image, click the ‘Create’ button, and your Virtual Machine will be deployed.
Step 6: Virtual Machine Successfully Deployed
You will get visual confirmation that your node is up and running.
Step 7: Connect to GPUs using SSH
NodeShift GPUs can be connected to and controlled through a terminal using the SSH key provided during GPU creation.
Once your GPU Node deployment is successfully created and has reached the ‘RUNNING’ status, you can navigate to the page of your GPU Deployment Instance. Then, click the ‘Connect’ button in the top right corner.
Now open your terminal and paste the proxy SSH IP or direct SSH IP.
Next, if you want to check the GPU details, run the command below:
nvidia-smi
Step 8: Check the Available Python version and Install the new version
Run the following commands to check the available Python version.
If you check the version of the python, system has Python 3.8.1 available by default. To install a higher version of Python, you’ll need to use the deadsnakes PPA.
Run the following commands to add the deadsnakes PPA:
sudo apt update
sudo apt install -y software-properties-common
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt update
Step 9: Install Python 3.11
Now, run the following command to install Python 3.11 or another desired version:
sudo apt install -y python3.11 python3.11-distutils python3.11-venv
Step 10: Update the Default Python3 Version
Now, run the following command to link the new Python version as the default python3:
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 2
sudo update-alternatives --config python3
Then, run the following command to verify that the new Python version is active:
python3 --version
Step 11: Install and Update Pip
Run the following command to install and update the pip:
python3 -m ensurepip --upgrade
python3 -m pip install --upgrade pip
Then, run the following command to check the version of pip:
pip --version
Step 12: Clone the Repository
Run the following command to clone the microsoft mineworld repository:
git clone https://github.com/microsoft/mineworld
cd mineworld
Step 13: Install Dependencies
Run the following command to install the dependencies:
pip3 install -r requirements.txt
Make sure your Python is 3.10, or you’ll face compatibility issues.
Step 14: Install HuggingFace Hub
Run the following command to install the huggingface_hub:
pip install huggingface_hub
Step 15: Login Using Your Hugging Face API Token
Use the huggingface_hub cli to login directly in the terminal.
Run the following command to login in huggingface-cli:
huggingface-cli login
Then, enter the token and press the Enter key. Ensure you press Enter after entering the token so the input will not be visible.
After entering the token, you will see the following output:
Login Successful.
The current active token is (your_token_name).
Check the screenshot below for reference.
Step 16: Download Full Model Repository
Run the following command to download the full model repository:
huggingface-cli download microsoft/mineworld --local-dir ./ --resume-download
Step 17: Run Inference
Execute the following command to run inference:
python3 inference.py \
--data_root input_data \
--model_ckpt checkpoints/700M_16f.ckpt \
--config configs/700M_16f.yaml \
--output_dir outputs \
--frames 16 \
--top_k 5
Step 18: Try Web Demo
Run the following command to try the web demo:
python3 mineworld.py \
--scene gradio_scene/scene.mp4 \
--model_ckpt checkpoints/700M_16f.ckpt \
--config configs/700M_16f.yaml
Step 19: Access the Microsoft Mineworld Game
Access the microsoft minecraft game on:
* Running on local URL: http://127.0.0.1:7860
* Running on public URL: https://391d0e9e0e5a5c9dba.gradio.live
Conclusion
With just a few terminal commands and a GPU-powered VM, you now have Microsoft MineWorld up and running—ready to generate real-time Minecraft scenes with interactive, AI-driven modeling. Whether you’re a researcher experimenting with world models or a developer exploring game-based simulation, MineWorld offers a unique playground inside one of the world’s most iconic games.
Dive in, try the web demo, and start building your own interactive Minecraft experiences today!
