Facebook and Matterport Work on Realistic Virtual Training Environments for AI

Matterport is an American spatial data company that allows users to capture, upload and create digital scans of real-world environments and share them online through its 3D camera and interactive viewing platform. Facebook AI Research (FAIR) and Matterport announced the collaboration to make the largest-ever dataset of 3D indoor spaces available exclusively for academic, non-commercial uses. The collaboration and 3D dataset have the potential to change the landscape of embodied AI and computer vision. Researchers can use the dataset to train embodied agents, such as home robots and AI assistants, at scale. FAIR leverages its new Habitat 2.0 training system to simulate the interactivity between robots and 3D objects instead of simply interpreting objects and surfaces. For instance, if a robot’s task is to pick up a fork from table and place it in the sink, previously these movements would just be assumed since we could nott actually simulate it effectively. In the new Habitat system, the fork is physically simulated, as the table, the sink and so on. That makes it more computationally intense, but also way more useful. Matterport has assembled an enormous collection of 3D-scanned buildings. The diversified dataset they share helps researchers to improve their algorithm more efficiently.