←
Project Malmo383 posts
Project Malmo avatar

Project Malmo

@Project_Malmo

Malmo is an AI experimentation platform built on top of Minecraft, and designed to support fundamental research in AI. aka.ms/malmo

◷ Joined January 2018▧ 21 Photos and videos
AK avatar
⟳ Project Malmo retweeted
AK@_akhaliq·13 Nov 2023

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: huggingface.co/papers/2311.0… Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.

0:22
Julia Kiseleva avatar
⟳ Project Malmo retweeted
Julia Kiseleva@julia_kiseleva·16 Apr 2023

The #greenlands platform is now live! 🤖🧑‍🤝‍🧑 With the ability to test #AI agents with real humans and receive invaluable feedback, this could be a game-changer for evaluation methods! Don't miss out, check it out now! #artificialintelligence #gaming #evaluation #Minecraft

📣BIG NEWS📣 #greenlands has arrived! 🤖🎮🧑‍🤝‍🧑 This cutting-edge platform enables games between #AI agents and human players in #Minecraft while collecting key metrics 📈 Check out the GitHub link below to learn more! github.com/microsoft/greenla… #NeurIPS2023 #NLProc #nlp2023

MsftTechNews avatar
⟳ Project Malmo retweeted
MsftTechNews@MsftTechNews·13 Dec 2022

Students can learn Azure via server creation, dive into #AI using Minecraft & GPT3 OpenAI or learn with Minecraft using Microsoft Research Project Malmo. @wongcyrus shares how to set up a free private Minecraft server in Azure via #EDU yearly credit. https://t.co/6vxpxfU6eI

Post attachment
iglu-contest @ NeurIPS avatar
⟳ Project Malmo retweeted
iglu-contest @ NeurIPS@IgluContest·30 Sep 2022

📢It's never too late! There is still one month left to the IGLU 2022 @NeurIPSConf NLP task deadline. Get your hands dirty by playing with our baseline. "What to ask as Clarifying questions?" 🎁 $9,000 cash prize + Co-authorship #NeurIPS2022 #NLP #AI https://t.co/Nmzzr3AKZX

MineRL Project avatar
⟳ Project Malmo retweeted
MineRL Project@minerl_official·31 Aug 2022

There’s still time to apply! ✍️

Excited to announce that the MineRL BASALT compute grant applications are now open! We're especially interested in funding those who lack access to compute or are underrepresented in machine learning. Applications reviewed on a rolling basis. Apply here: forms.gle/WSLpoEyF2a69YP8R6