Augmented/virtual reality can help extend critical infrastructure lifespan

University of Waterloo engineers are turning to augmented/virtual reality (AR/VR) to better understand—and maintain—the physical reality of Canada’s critical infrastructure. Their research is published in the journal Automation in Construction.

On-site inspectors are typically limited in what they can observe when inspecting bridges, roads, towers, pipelines, and other structures because they can’t always see or detect every potential problem within large infrastructure

To help fix this issue, Waterloo professor Dr. Chul Min Yeum and his colleagues have developed a cutting-edge system called the Smart Infrastructure Metaverse that uses AR/VR to allow on-site and off-site inspectors to interact with each other as they view both the real structure, and a 3D scanned replica model at the same time.

Not only does their system produce more rapid, comprehensive, and complete results than traditional on-site visual examinations, but it also provides a greater context for problems within the entire structure.

Their innovative work meets an urgent need. Much of Canada’s vital infrastructure was constructed in the mid-20th century and is now nearing or exceeding its lifespan, a situation that poses serious public safety risks. Yet replacing these structures would cost a staggering $264.7 billion, according to Canada’s Core Public Infrastructure Survey. Yeum’s solution combines several advanced technologies so problems can be addressed quickly and the life expectancy of infrastructure extended.

“The Smart Infrastructure Metaverse is all about making it easier for on-site and off-site inspectors to work together on structural inspections,” Yeum said. “We’re using AR/VR headsets to give them a shared view so they can see exactly where they are and what they’re looking at during the inspection, no matter where they’re located.”

The research team included Yeum and Dr. Zaid Abbas Al-Sabbag, both in Waterloo’s Department of Civil and Environmental Engineering, as well as Dr. Sriram Narasimhan, of the Department of Civil and Environmental Engineering, University of California Los Angeles.

In a key experiment focused on a railway bridge in Kitchener, Ontario, Yeum and his colleagues created a three-dimensional model of the structure using 3D scanners and a panorama camera. This model allowed the accurate tracking of the location and head position of both the on-site and off-site inspectors within the 3D map.

Next, an off-site engineer wearing a VR headset explored the 3D model of the bridge the way someone would navigate a VR game. While this was happening, on-site inspectors wearing AR headsets could, through a holographic display, see the actual bridge, the VR user, and additional information within the digital map.

Since everyone in this examination was connected to the digital map, the off-site inspector saw the exact locations of the on-site users along with the areas of the structure they were inspecting. That meant the remote inspector could ensure the data collected was accurate. To back up the human inspectors, the research team used artificial intelligence to analyze the images sent from the on-site inspectors’ AR headsets to further identify any structural damage.