Computer Vision Used to Monitor the Health of Oceans

The Woods Hole Oceanography Institution is bringing the advanced capabilities of computer vision to the oceans.

Peter Fox and Charles Stewart, data scientists at Rensselaer Polytechnic Institute, are beginning a large-scale collaboration with WHOI to study various ocean ecosystems' health, biodiversity and environmental contamination. 

A professor in the Tetherless World Research Constellation at Rensselaer, Fox is bringing his expertise in data science to the study of our immense ocean ecosystems. Stewart, a professor in the Department of Computer Science, will apply his experience in developing computer vision systems, to the analysis of images of the ocean floor and seawater.

“A massive amount of data about the oceans is gathered every single day by sensors, cameras, sonar and other technologies,” Fox said. “Maybe 3 percent of that information is ever even looked at again. Any other valuable knowledge is essentially lost. This makes the return on investment to create this data extremely low, monetarily and scientifically. What we are doing through this collaboration is integrating and enhancing this data so that it can be more easily consumed and used by other scientists and the public.”

To begin this large effort, Fox, Stewart and their students will work with WHOI oceanographers and computer scientists to better analyze and interpret data from some of the sophisticated WHOI underwater imaging technologies.

These technologies include the FlowCytobot, an automated underwater microscope that identifies and counts tiny phytoplankton in the water. The technologies also include SeaBED, an autonomous underwater vehicle that hovers slightly above the seafloor at depths of up to 6,000 feet and takes highly detailed sonar and optical images of the seafloor, and HabCam, the habitat mapping camera system that is moved above the seafloor to create a continuous image ribbon.

All of these technologies provide important information to oceanographers on the health of our oceans. They can also provide unexpected discoveries on everything from underwater geologic activity to the impacts of glacial melting. They also have another important thing in common: They each produce enormous amounts of data that need to be analyzed.

The HabCam alone collects five to six images every second for days at a time, producing more than 2 terabytes of data per day for strips of the ocean floor 100 miles or more in length. These images must be processed to characterize habitat, count economically important species such as scallops, and determine the extent of invasive organisms. 

Stewart will apply sophisticated computer vision techniques in new ways to challenge the limit of what can be accomplished with even the most recent research in computer vision for the underwater environment. He will use algorithms and software to interpret and refine images from the WHOI technologies, helping WHOI researchers and other scientists greatly enhance the research potential of their existing technologies.

Fox will work with WHOI researchers to apply tools that allow the data from these technologies and others like them to be easily accessed and shared among researchers and the public. A large part of this work will be accomplished by incorporating Semantic Web and knowledge provenance or origins technologies to the raw and processed data produced by the WHOI technologies, according to Fox. Semantic technology encodes data with information that computers or other web-enabled devices can use to better share, search and interpret the data. 

“By integrating and enhancing this truly multidisciplinary data, we can enhance the utility of the data by making it usable to a variety of people, of varying expertise, and on various operating systems,” Fox said. “This key investment will help us unlock the full potential of the data being developed with these technologies.”