Fujitsu Working Hard to Develop ‘Ocean Digital Twin’ 

AUV-ASV connected system and data acquisition. Credit: Fujitsu

The concept of a digital twin has interested scientists for quite a long time. A virtual representation of an entire system or singular object, a digital twin is updated with real-time data and often relies on AI to ensure the accuracy of the simulation.

While many organizations are endeavoring to create digital twins of various systems – such as NVIDIA’s announcement at GTC24 of the Earth-2 climate digital twin cloud platform – Fujitsu has set its sights on creating a digital twin of the ocean.

By combining LiDAR, AI, autonomous underwater vehicles (AUVs), and a system designed to judge gymnastics, Fujitsu can study highly accurate reproductions of underwater ecosystems. With enough data and hard work, this will enable Fujitsu scientists to predict how an environment might change over time as well as how certain conservation measures might affect an ecosystem.

This is not the first time Fujitsu scientists have endeavored to create a digital twin. The research institution previously teamed up with Carnegie Mellon University to create an AI-powered digital twin of traffic in Pittsburgh, USA. However, this twin of the ocean comes at a time when the planet’s water is heavily threatened by pollution and human development. It could help us understand exactly what we are doing to our oceans.

Gymnastics and Underwater Research

The Fujitsu digital twin uses AUVs and satellites to gather 3D data about oceanic objects like seaweed and coral reefs, as well as man-made structures that can alter the local ecosystem. Once this data is collected, scientists generate a model that measures and simulates changes in the local environment such as the growth of organisms.

Concept of ocean digital twin technology in relation to achieving SX. Credit: Fujitsu

For instance, 3D data enables researchers to compute the quantity of carbon dioxide absorbed by a certain area's plant life and determine its biomass, thereby pre-verifying measurements connected to blue carbon. To do time-series simulations of the development of underwater species, biological data is also included. This enables the simulation of changes in the absorption of carbon dioxide resulting from the implementation of specific conservation measures for marine ecosystems, including the growth status of undersea animals, and the pre-verification of the efficacy of countermeasures against global warming.

However, a major challenge here is the resolution of the 3D data acquired by the satellites and AUVs. Existing technologies like acoustic sonar can’t give a clear enough image, with a resolution limited to about 10cm.

To solve this in real-time underwater, Fujitsu turned to an unlikely technology – a “Judging Support System” designed in collaboration with the International Gymnastics Federation to assist in gymnastics judging. With the help of AI, this system uses 3D sensors and AI to assist judges with capturing and properly assessing the complex movements of gymnasts. While pairing this technology with underwater research may seem strange, it’s actually quite sensible. The AUV must move around in constantly sloshing water, and this adapted gymnastics system enables high-resolution and high-speed scanning of the underwater environment.

The gymnastics judging system was also supported by an underwater Light Detection And Ranging (LiDAR) device which allows users to choose among three laser wavelengths to find the optimal measuring wavelength based on sea conditions. This will allow for the measurement of three dimensions from an autonomous unmanned submarine in motion as well as the development of technologies that will track moving objects.

Additionally, Fujitsu leverages AI for image enhancement to color correct and clarify images. This technique comprises two forms of AI: one that corrects the original color of the topic and enhances the blurred contours of target items to generate enhanced photos before 3D conversion; the other type of AI removes turbidity and restores contours. By doing this, mistakes in 3D processing and subject recognition are avoided, allowing the shape of each object to be measured. This enables accurate identification and measurement of ecosystems down to several centimeters, even when working in murky waters.

Figure 3. 3D measurement results using underwater LiDAR
Credit: Fujitsu

To verify the efficacy of this technology, Fujitsu and the National Maritime Research Institute (NMRI) carried out a field test to gather underwater data automatically and in real time. Fujitsu integrated an underwater sensor into the "AUV-ASV connected system" created by NMRI, combining a camera and LiDAR for real-time 3D measurement. High-resolution 3D real-time data of centimeter-scale objects, such as underwater installation pipes and coral reefs, were successfully obtained from this experiment.

Future Plans

Fujitsu plans to broaden the measuring targets of this technology in the future, and the researchers hope to include seaweed, which absorbs a large amount of blue carbon. By FY 2026, the company hopes to create an ocean digital twin for seaweed beds. This will help businesses and local governments measure the amount of carbon stored in seaweed beds, conserve and grow new seaweed beds, and promote and maintain coral reef biodiversity.

Fujitsu also intends to give a comprehensive report on the field trial’s outcomes at the 2024 Spring Meetings of the Japanese Society of Fisheries Science, which is scheduled from March 27-30, 2024.

Fujitsu will keep working on developing technologies that can consistently gather data in difficult conditions (such as strong currents and uneven seabed topography) and take part in initiatives that should help decarbonize the ocean by encouraging the growth of seaweed that absorbs carbon. Additionally, inspections and accumulating use cases will be added to the list of measurement objectives. Fujitsu intends to create an ocean digital twin tool that performs simulations using knowledge from environmental science, biology, and other subjects based on the observed 3D data. The researchers also want to collaborate with businesses, regional administrations, and groups that are implementing carbon-neutral initiatives.