Predictions on the benefits and challenges to come

The metaverse is now here. Beyond the buzzwords and Mark Zuckerberg’s aspirational video, what is it? And what are the benefits and challenges of a metaverse for medical communications?

The hyperbole ranges from, “it will be a digital twin to the universe” to dismissive charges of the metaverse as “glorified VR platforms”. Some articles read as though an editor used Find & Replace to swap “virtual reality” for the trendier “Metaverse”, so there’s reason to be skeptical of the hype.

In fact, the healthcare industry has already been creating their metaverse. Each interactive and immersive extended reality experience (whether augmented reality or virtual reality) can be seen as a focused and complete metaverse of its own. The hype right now has to do with stitching them all together into larger platforms, which will have its own benefits and challenges for the healthcare industry.

Before we get into those, let’s look at the origins and how the metaverse came together:

SNOW CRASH: the origin of the Metaverse concept

The name “Metaverse” comes from the entertaining cyberpunk novel SNOW CRASH by Neal Stephenson, published in 1992. In the novel, the Metaverse is a shared virtual spherical world that people log into using VR gear. In the Metaverse, they appear as digital avatars, and travel around, watch entertainment, play games, and conduct business.

Once the idea of a Metaverse was described, it has never really gone away. Instead, it has waited for technology and telecommunications to catch up with the concept.

The vision by Meta, née Facebook, looks similar to the Metaverse in the novel. One of the biggest differences in reality however, is that it’s unlikely our experience with a functional metaverse will start with only one metaverse.

Right now, Meta (Facebook), Google, Apple, Nvidia and parent studios to games such as Roblox and Fortnite have all announced that they are either currently developing or have interest in developing their own metaverse. In addition, smaller open source initiatives will likely spring up in attempts to democratize the metaverse market. There is already a rise in start-ups, as reported by the AR/VR Association, which lists over 20 examples in their Q4 2021 report.

Will there be one set of extended reality code, one architecture that emerges so it doesn’t matter whose devices you are using, à la READY PLAYER ONE? Only time will tell, though it’s likely that different platforms by the industry leaders will have some interoperability, similar to iOS and Android for smartphones.

Definitions

• Virtual Reality (VR): Interactive experiences where your whole field of view is in a virtual space, by wearing a headset that blocks out real-world sight & sound.
• Augmented Reality (AR): Interactive experiences where information and objects are overlaid onto the real world, through goggles or through a smartphone. Think Pokémon GO.
• Mixed Reality (MR): Similar to AR, with digital interactive elements anchored to real-world objects and points in space.
• Extended Reality (XR): the umbrella term for all of the above.

Benefits of a Metaverse

Extended Reality (XR), is seen as the foundation for most metaverse concepts. Meta seems to be betting heavily on VR specifically, with their acquisition and development of Oculus VR.

While Apple may be entering into the VR space as well, it’s been reported they are betting on AR as the primary focus. Bloomberg’s Mark Gurman quoted a sources saying, “The augmented-reality headset is Apple’s real priority because it can be worn all day and, naturally, not take anyone out of their real environment.” [link]

Medical Training & Outreach – Immersive training in VR is highly effective. Whether you’re at a large oncology conference with thousands of people visiting booths or at home with chores beckoning, sliding a comfy VR headset on to self-direct you through a training experience means you are free from the distractions of your surroundings.

Templates, animation, learning modules, interactive data sets; all of these can be put together into one experience with user-centered navigation to maintain the user’s engagement as they explore. The latest hardware and software have largely solved problems with nausea for the few people who experienced it in earlier iterations. Extended reality gear varies from expensive desktop computer set-ups, all the way to an app you download on your smartphone which you then pop into some affordable goggles.

Participants using VR to learn concepts in biology showed better recall, increased positive emotions, and higher levels of engagement than those using other applications.

VR training has a 75% knowledge retention rate vs 10% for learning that involves reading or watching videos – and only 5% for classroom learning.

Storytelling – As the technology has become more portable and user-friendly, the focus now is on the quality of the content and storytelling. An effective story told to explain complex science, large data sets, or a patient’s experience will always engage the user far more than words printed on a page.

Scale – We’re already playing in the metaverse as its defined, but not at a shared scale. For example, Web AR is like a small metaverse with contextually relevant information on top of what is visible on screen. It’s a metaverse, but limited to a single view. Building these tools in such a way that they are components with reuse in mind so we can scale up the clients metaverses, and allow for collaboration in addition to singular experiences.

The tech building blocks and creative storytelling are the same; the difference is scale.

Our proprietary system for webAR delivery allows for any image, diagram or concept to be brought to life in beautiful 3D audience environments. This makes this technology perfect for engagement and stickiness from video conference to live interaction.

Hands-on learning – If VR is poised to house the collective experience of the Metaverse, augmented reality (AR) is likely to be how it extends into our everyday lives.

Because AR allows you to see the world around you with interactive visuals displayed on glasses, goggles or even a phone screen, it’s ideal for guided remote training and training within real-world setting. Everything from clinical scans, anatomy at 1:1 scale, or any kind of technical diagram can be brought to life, and multiple pieces within a metaverse can be collected to tell a fuller story. While you’ll never fully replicate the tactile experience of medical training in a cadaver lab, interactive 3D assets and overlays can bring the reality much closer than watching a video; for example, helping nurses find a vein, the goal with the Accuvein startup.

Another example includes placing a heads-up display to the nearest automated external defibrillators (AEDs) in an environment. Below, we share a Red Nucleus project that allows HCPs to experience real-life scenarios as though they have migraine symptoms. Ultimately, AR will likely play a role in metaverse by extending its reach out as an overlay on the world around us.

In this virtual reality experience intended for HCPs created by the Red Nucleus team, users experience common severe migraine symptoms from a first-person perspective, in order to better empathize with patient accounts.

Pharma Industry Challenges posed by the Metaverse

Portals and “guard rails” – During the initial shift to virtual during pandemic, microsites were a popular form of virtual conference engagement, though their success in reaching target audiences is questionable compared to face-to-face. Many of these had a simple Y/N “are you an HCP” popup before allowing people to view their messaging.

If a larger tapestry of extended reality experiences is woven together to form a Metaverse of experiences, whether by a single pharmaceutical company or under the banner of a conference, it’s likely some sort of user verification type or digital “guard rail” will be necessary to limit admittance to the target audience and stop the experience from being overwhelmed by curious visitors.

Patient data – There are ethical considerations if 3D experiences beyond video become the norm. If a surgery is based on a specific patient’s conditions, ethical & legal frameworks will likely need to guide its use in training.

In 1951, the cancer cell line of patient Henrietta Lacks became entrenched in medical use (HeLa cells) without her consent or her family’s consent after her death. If a virtual procedure is effective and draws on real patient simulation using photogrammetry or other technology, guidelines would have to be in place to prevent repeating a similar wrong in digital form. Mis-steps in creating digital assets without consent have already occurred, (“Bataclan survivor shocked as surgeon tries to sell her X-ray as NFT” The Guardian, retrieved Jan 31 2022).

Interoperability – One of the challenges for VR in a shared metaverse comes from interoperability. Would “rooms” in VR function similar to opening apps on your phone? Would there be “portals” from one manufacturer’s metaverse to another? Or the less immersive but functional “exit & return to homescreen”? If a specific metaverse platform becomes industry standard in much the same way iPads have been used in the past for Pharma Sales Reps and Medical Science Liasons, it can help smooth standards and regulatory approvals.

Digital doubles – Part of the concept of the metaverse involves the avatar, people’s “digital doubles”. Who owns them or the likeness? What if an MSL switches companies? If the goal is for a large training, there may only be a few characteristics of a 3D avatar that can change, similar to bitmojis. However, as 3D scanning becomes easier with LiDAR on phones, we may see some Terms of Service address ownership of your likeness, eye-tracking data, and even movement data within the virtual environment.

A training session in VR to may have an introduction screen asking for consent of collection of this data to help optimize the simulation for future updates.

What's next

We’re already building metaverse experiences for our clients. Whether for learning & development programs or scientific communications, we’re able to create the types of storytelling necessary for effective, memorable experiences.

Until a Metaverse exists at scale, (or more likely, a few of them) building smaller, self-contained extended reality experiences will not only engage and educate your current internal teams and customers, but it will certainly help prepare you for the day you create an experience for the metaverse. You may even be able to port or repurpose what you’ve already built.

So get your avatar ready and join us as your trusted guide, on the journey to the metaverse.

By James Hackett, VP of Creative Innovation.
Learn more about Red Nucleus Medical Communications.

Bookmark our News page, and watch our AR/VR Sizzle Reel.