2024

Elon Musk

A team of researchers from Nanyang Technological University in Singapore recently unveiled a new method for tracking human movement for the metaverse.&nbsp;
One of the key features of the metaverse is the ability to represent real world objects and people in the digital world in real time. In virtual reality, for example, users can turn their heads to change their viewpoints or manipulate physical controllers in the real world to affect the digital environment. 
The status quo for capturing human activity in the metaverse uses device-based sensors, cameras or a combination of both. However, as the researchers <a href="https://arxiv.org/pdf/2402.19258.pdf" target="_blank" rel="noopener nofollow">write</a> in their preprint research paper, both of these modalities have immediate limitations. 
Related: <a href="https://cointelegraph.com/news/elon-musk-sues-sam-altman-openai-agreement-breach">Elon Musk sues Sam Altman, OpenAI over agreement breach</a>
A device-based sensing system, such as a hand-held controller with a motion sensor, “only captures the information at one point of the human body and thus cannot model very complex activity,” write the researchers. Meanwhile, camera-based tracking systems struggle with low-light environments and physical obstructions. 
<h2 id="enter-wifi-sensing">Enter WiFi sensing</h2>
Scientists have used WiFi sensors to track human movement for years. Much like radar, the radio signals used to send and receive WiFi data can be used to sense objects in space. <iframe width="100%" height="315" src="https://www.youtube.com/embed/7LTr02cJkiA" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen data-ct-non-breakable="true"></iframe>
WiFi sensors can be fine-tuned to pick up heartbeats, track breathing and sleeping patterns, and even sense people through walls. 
Metaverse researchers have experimented with combining traditional tracking methods with WiFi sensing to varying degrees of success in the past. 
<h2 id="enter-artificial-intelligence">Enter artificial intelligence</h2>
WiFi tracking requires the use of artificial intelligence models. Unfortunately, training these models has proven to have a high degree of difficulty for researchers. 
Per the Singaporean team’s paper:
<blockquote data-ct-non-breakable="true">
 “Existing solutions using Wi-Fi and vision modalities rely on massive, labeled data that are very cumbersome to collect. […] We propose a novel unsupervised multimodal HAR [human activity recognition] solution, MaskFi, that leverages only unlabeled video and Wi-Fi activity data for model training.”
</blockquote>
In order to train the necessary models required to experiment with WiFi sensing for HAR, scientists have to build a library of training data. The data sets used to train AI can contain thousands or even millions of data points depending on the aims of the particular model. 
Often, labeling these data sets can be the most time-consuming part of conducting these experiments. 
<h2 id="enter-maskfi">Enter MaskFi</h2>
The team from Nanyang Technological University built “MaskFi” to overcome this challenge. It uses AI models built using a method called “unsupervised learning.”
In the unsupervised learning paradigm, an AI model is pretrained on a significantly smaller data set and then put through iterations until it’s able to predict output states with a satisfactory level of accuracy. This allows researchers to focus their energy on the models themselves instead of the painstaking effort it takes to build robust training data sets. 
<figure data-ct-non-breakable="true">
 <img src="https://s3.cointelegraph.com/uploads/2024-03/897b53bc-0a97-46f8-a3c8-dec3221c40e6.jpg" alt="" title="">
 <figcaption style="text-align: center;">
 Source: Yang, et. al., 2024
 </figcaption>
</figure>
According to the researchers, the MaskFi system achieved about 97% accuracy across two related benchmarks. This indicates that this system could, through future development, serve as the catalyst for an entirely new metaverse modality: a metaverse that can provide a 1:1 real-world representation in real-time.&nbsp;

Scientists use WiFi signals to track human movement for the metaverse

Table of Contents

Cointelegraph

Other articles published on Mar 1, 2024

Crypto Analyst Outlines 7 Altcoin Narratives To Follow The Recent BTC and ETH Pump

ETH & SOL Coins Grow Strong Pre-Bitcoin Halving While Raffle Coin Gains Momentum

Fed Plans Rate Reductions with a Watchful Eye on Economic Indicators

NFT Community Excited As Blur and Blast Take Impressive Moves

Token.com's Pioneering Platform: A New Dawn for Crypto Interaction

Ethereum Price Gains Momentum

Join our free newsletter for daily crypto updates!