Introduction

Contemporary work demands extensive collaboration among individuals with diverse skills, necessitating regular meetings and conferences for sharing insights and discussing projects. Currently, most collaborative efforts occur via screen sharing on computers or through video conferencing, where a presenter displays their work for others to view. Unfortunately, these traditional setups are often inadequate, especially when dealing with complex data visualization and manipulation. Presenters typically have sole control, and their tools are limited to a flat 2D interface, which hinders effective communication.

Participants who are not presenting can only observe, lacking the ability to engage with the material or manipulate the views. This is particularly limiting in fields like natural sciences and engineering, where understanding 3D structures is crucial for meaningful discussions.

However, multi-user virtual and augmented realities can dramatically enhance this experience by allowing all meeting participants to interact with the content simultaneously, moving freely within the space and examining objects at their discretion.

With the advancements in technology that underpin the Metaverse, developers can now create immersive multi-user environments where individuals can navigate the same VR session and manipulate objects as they wish, utilizing WebXR standards and a plethora of web programming tools.

Together with Fabio Cortes, we are building such a platform and have begun testing advanced prototypes at scientific events and educational demonstrations focused on molecular structures, spanning from chemistry to structural biology and physics. The fundamental technologies we are developing can be easily modified for various purposes, such as creating interactive museums for 3D-scanned items, showcasing products for businesses, or simply establishing virtual meeting spaces.

The Power of Manipulating Objects with Bare Hands

In many professional settings that involve 3D graphics and modeling, the limitations of conventional mouse/keyboard/screen interfaces stem from the challenge of using a 2D input device to manipulate 3D objects. Additionally, users can only interact with one object at a time, complicating tasks that require simultaneous comparisons.

Modern VR and AR devices equipped with hand-tracking technology or handheld controllers provide direct input with six degrees of freedom, allowing both hands to be used simultaneously. This creates an intuitive and immersive experience, as demonstrated in the accompanying video.

In another video, you can observe the natural movements involved in manipulating atoms within a molecular simulation, again showcasing the seamless interaction that VR provides.

Multiple Users Acting on Objects in the Same VR Session

By utilizing two or more devices, we can combine their hand-tracking or controller inputs to create a shared VR session, rendering the same 3D view across all devices. Our initial tests employed WebRTC, a web standard for real-time communication, which enabled a passive user to join a VR session hosted on the headset of a master user. The master headset handled all calculations, while the passive device merely updated its VR scene based on the master's coordinates and transmitted finger joint movements and interaction events.

For example, two users can simultaneously interact with a molecular simulation, each controlling different aspects of the model, as illustrated by the green and grey hands in the image.

Further development has seen us implement web sockets for improved connectivity and real-time capabilities, allowing more than two users to collaborate in the same VR environment. The next video demonstrates two individuals working on the same object, despite being over 11,000 kilometers apart.

Multi-user Demonstrations in Web-VR: Applications in Scientific Conferences

With the capability for fast multi-user connectivity, our goal is to enable more than two participants to interact within the same VR scene, moving freely and manipulating objects as they wish. Each user has a simple yet effective avatar rendered based on their VR device, providing a sense of physical presence in the virtual space.

In a recent application, we were invited to showcase our multi-user VR tool during the closing conference of a long-term funding program focused on membrane biology. Scientists were able to discuss their research in a VR setting that mirrored their physical arrangement, enhancing the sense of proximity and facilitating natural conversation.

In the following images, you can see a researcher presenting the structures of proteins designed in their lab, while colleagues observe and interact with the model from various angles. They had the freedom to manipulate the molecule as they wished, enhancing their collaborative experience.

The next tweet and video further illustrate how we maintain a close relationship between the virtual and real worlds during presentations.

Engaging Students with Multiuser Web-VR

Another area of exploration involves using VR to teach students, enabling them to interact with virtual objects in a natural and engaging manner. For instance, students and teachers have experimented with virtual protein structures, leading to excited reactions and recognition of the educational potential of such tools.

The immersive VR experience allows users to visualize complex 3D objects more effectively than traditional 2D representations, leading to a deeper understanding of molecular structures. The interactive nature of these tools fosters self-learning and enhances communication about intricate concepts.

Simulating Molecular Mechanics and Experiment Outcomes

While VR facilitates the manipulation of abstract 3D models like molecules, it also enables realistic simulations of physical interactions within these environments. Users can control various aspects of the VR scene, from textures to forces like gravity, enhancing the learning experience.

We have developed VR prototypes where users can apply forces to virtual objects, allowing for dynamic interactions that were previously limited to science fiction. This feature is particularly valuable in chemistry education, where it can demonstrate concepts like molecular torsions and the relationship between temperature and atomic motion.

Some of our VR tools even allow users to simulate experimental outcomes based on changes they make to molecular structures. For instance, by manipulating protein representations, users can observe how these changes affect the results of Small-Angle X-ray Scattering (SAXS) experiments.

Creating Custom Web-AR and Web-VR Sessions

The PDB2AR tool from our moleculARweb website enables scientists and educators to create personalized AR and VR sessions easily, without requiring coding skills. This free tool has been available for over a year and allows users to upload 3D models created in popular software or parse molecular structures from databases like the Protein Data Bank.

The application generates links for viewing materials in augmented reality on various devices, facilitating access to high-end VR experiences without the need for specialized hardware.

Further Reading on Augmented and Virtual Reality

For those interested in augmented and virtual reality technologies, we invite you to explore further resources, including our free online VR applications and articles on related topics.

In conclusion, my work encompasses a range of interests, from nature and science to technology and programming. Consider becoming a Medium member to access all my stories and subscribe for updates. For inquiries regarding services, please visit my services page.