ROV Piloting Based on Augmented Reality Technology

Advances in piloting and managing scientific operations will be based on several key principles.

A 3D visual model of the underwater environment

Depth measurements generated by echo sounders and optical tools (photogrammetry, laser profilers and LIDAR) will be integrated into a high-resolution 3D model which combines all the environmental data from the dive or series of dives. The 3D model is “dressed up” with visual material generated by photo-video optical imagery (textured). Currently available offline, in two or three years’ time these techniques will be available in real time and completely automated. These models can be viewed using a standard 2D/3D screen or with immersive tools such as a virtual-reality headset. Being georeferenced, the dimensional analysis of the 3D model can be transposed directly and graphically.

Integration of multidimensional data

The environment model will be enriched with themed “layers” derived from data from purpose-specific sensors (such data can be collected during the dive or from earlier dives). Currently produced offline in 2D and 3D GIS tools, these purpose-specific layers will be a precious resource for exploring sites. The differential analysis between existing and new data is a particularly promising avenue for development (characterizing the temporal evolution of the environment).

Positioning the viewpoint relative to the extended environment model

Thanks to the construction of the visual 3D model of the environment, the scientist will be able to navigate through a virtual environment without being limited to the submersible’s field of vision at a given time (a “virtual submersible” would generate graphically the viewpoint relative to the site). The indication of the real field of vision and in particular the field within reach of the manipulators (arms, senses, samplers, etc.) in the overall model will allow for the guiding of operations and provide the basis for a new graphic interface whereby the controls can be situated in the environment model.

Fully automated manipulation

Currently, depending on the objectives defined by the scientists, the operator analyses the feasibility of the requested operations and estimates the most suitable position for the ROV, and then undertakes the remotely operated manipulations. The aim will be to reduce as much as possible the role of the intermediary between the scientists and the working area, by means of automated functions. Eventually manipulations could be performed by a scientist using a virtual machine which would facilitate all operations thanks to fully automated functions, such as defining graphically the manipulation targets (corals, rocks, etc.) in the environment model and automating the management of standard tools (corers, suction devices, probes, etc.).

Analysis assisted by artificial intelligence

Identifying species, geological phenomena and the concordance of signature measurements will indicate in real time elements that are of potential interest to scientists. Sampling or in situ analysis could be triggered by the algorithmic detection of anomalies.