FathomVerse

Name: FathomVerse
Author: Monterey Bay Aquarium Research Institute

Monterey Bay Aquarium Research Institute

DOCUMENTED

Published in academic literature

For:Researchers & AcademicsGeneral Public & Enthusiasts

Free

Download on the App Store Get it on Google Play

App Summary

FathomVerse is a citizen science game where players help ocean researchers by identifying and labeling marine animals in underwater imagery, contributing to a large-scale database for training artificial intelligence. The associated research highlights that creating these labeled image databases through community science is critical for training AI to analyze marine visual data at a scale that exceeds human capacity. The authors conclude that this approach can accelerate the processing of visual data, enabling new research into species distribution and contributing to the stewardship of a healthy global ocean.

App Screenshots

Detailed Description

Functionality & Mechanism FathomVerse is a citizen science platform that leverages gamified interaction to generate labeled datasets for marine computer vision research. Users engage in sessions involving minigames to view and annotate real underwater imagery, identifying more than 50 distinct groups of marine animals. These annotations are aggregated to train artificial intelligence models, thereby accelerating the analysis of large-scale ocean visual data. The interface integrates quests and educational modules to structure user contributions and deliver pedagogical content.

Evidence & Research Context

The system contributes data to FathomNet, an open-source, standardized image database developed to address the challenge of processing high-volume marine visual data for research.
The app's community science contributions produced the FathomVerse v0 detection dataset, comprising 3,843 images with 8,092 bounding box annotations across 12 morphological groups.
Associated research outlines the dataset's utility for advancing computer vision in fine-grained transfer learning, novel category discovery, species distribution modeling, and carbon cycle analysis.

Intended Use & Scope The platform is designed for the general public, including students and educators, to contribute to marine science research through data annotation. Its primary utility is to generate large, labeled datasets for training machine learning models for ecological monitoring. The app is not a professional species identification tool; data requires aggregation and expert validation for definitive analysis.

Studies & Publications

2 publications

Peer-reviewed research associated with this app.

Development/Design Paper

FathomVerse: A community science dataset for ocean animal discovery

Patterson et al. (2024) · arXiv

Describes the research-driven development of this app

Can computer vision help us explore the ocean? The ultimate challenge for computer vision is to recognize any visual phenomena, more than only the objects and animals humans encounter in their terrestrial lives. Previous datasets have explored everyday objects and fine-grained categories humans see frequently. We present the FathomVerse v0 detection dataset to push the limits of our field by exploring animals that rarely come in contact with people in the deep sea. These animals present a novel vision challenge. The FathomVerse v0 dataset consists of 3843 images with 8092 bounding boxes from 12 distinct morphological groups recorded at two locations on the deep seafloor that are new to computer vision. It features visually perplexing scenarios such as an octopus intertwined with a sea star, and confounding categories like vampire squids and sea spiders. This dataset can push forward research on topics like fine-grained transfer learning, novel category discovery, species distribution modeling, and carbon cycle analysis, all of which are important to the care and husbandry of our planet.

Development/Design Paper

FathomNet: A global image database for enabling artificial intelligence in the ocean

Katija et al. (2022) · Scientific Reports

Describes the research-driven development of this app

AbstractThe ocean is experiencing unprecedented rapid change, and visually monitoring marine biota at the spatiotemporal scales needed for responsible stewardship is a formidable task. As baselines are sought by the research community, the volume and rate of this required data collection rapidly outpaces our abilities to process and analyze them. Recent advances in machine learning enables fast, sophisticated analysis of visual data, but have had limited success in the ocean due to lack of data standardization, insufficient formatting, and demand for large, labeled datasets. To address this need, we built FathomNet, an open-source image database that standardizes and aggregates expertly curated labeled data. FathomNet has been seeded with existing iconic and non-iconic imagery of marine animals, underwater equipment, debris, and other concepts, and allows for future contributions from distributed data sources. We demonstrate how FathomNet data can be used to train and deploy models on other institutional video to reduce annotation effort, and enable automated tracking of underwater concepts when integrated with robotic vehicles. As FathomNet continues to grow and incorporate more labeled data from the community, we can accelerate the processing of visual data to achieve a healthy and sustainable global ocean.

In the Media

FathomVerse replaces doom-scrolling with ocean exploration

The Monterey Bay Aquarium Research Institute developed FathomVerse to replace doom-scrolling with ocean exploration, using real underwater footage to train players to spot and classify marine animals through minigames. "The images you see in FathomVerse are real photos collected by ocean researchers," the creators note, explaining that scientists and communities worldwide share imagery from their ocean monitoring systems. Players can unlock missions and curate galleries of over 50 ocean animal groups while testing their identification skills against AI-generated estimations.

GoodgoodgoodRead article

NCEI Collaborates with FathomVerse to Celebrate Citizen Science Month

NCEI collaborates with Monterey Bay Aquarium Research Institute (MBARI) to feature their FathomVerse mobile game, which launched its latest version on March 26 and uses ocean data stewarded by NCEI/NOAA to create an accessible approach to ocean exploration. The free mobile game invites players to contribute to science by identifying ocean animals in stunning imagery while training AI models, as "AI is used to capture ocean images and analyze them, but relies on people to continuously train and verify the models." Players can learn to identify nearly 50 groups of ocean animals while helping researchers measure and monitor marine life in a changing ocean.

NoaaRead article

Into the FathomVerse: Community science, data, and the sea

MBARI researchers developed FathomVerse to address the challenge of processing vast amounts of marine video data, combining community science, AI, and mobile gaming to accelerate ocean research. "How do we know if anything we're doing is sustainable if we don't even know the animals that we're impacting?" said Dr. Kakani Katija, noting that faster data processing can inform critical projects like offshore wind and wildlife habitat management. The app leverages MBARI's 37 years of footage containing 11 million observations to train machine-learning models that can analyze the institute's 1,000-plus hours of annual video collection.

WesterndigitalRead article

Mobile video game submerges users into ocean depths

Researchers developed FathomVerse to address the challenge of processing hundreds of thousands of hours of underwater video collected by U.S.-based organizations, using a mobile game that combines immersive imagery and gameplay to train AI systems. The NSF-supported project engages ocean enthusiasts worldwide to review and label real underwater images, helping improve AI that assists researchers in studying marine life and assessing ocean health. The game leverages crowdsourcing because "there are very few experts that can look at an image and tell you exactly what that animal is."

NsfRead article

FathomVerse video game expands ocean exploration

Researchers from the Monterey Bay Aquarium Research Institute (MBARI) launched FathomVerse to help scientists identify underwater creatures using real data collected by robotic vehicles, with principal engineer Kakani Katija receiving a National Science Foundation Convergence Accelerator grant for the project. "Game players are looking at real data that's collected by researchers all over the world and sometimes are the first people to look at images that have been collected by these researchers," Katija said. The free app is available on the App Store and Google Play, allowing players to contribute to biodiversity surveys and climate-related decisions.

KcbxRead article

Players of Immersive FathomVerse' Will Help Science Solve Deep Sea Mysteries and Identify New Life Forms

The Monterey Bay Aquarium Research Institute (MBARI) launched FathomVerse, an immersive underwater exploration game designed to help scientists solve ocean mysteries and identify new life forms, using AI training as part of the Ocean Vision AI project. "There are 60,000 images in the FathomVerse game at the time of launch, almost 10 thousand of which are unlabeled data," explained MBARI Principal Engineer Kakani Katija, who led the game's development. The app features deep sea imagery from MBARI, NOAA, Schmidt Ocean Institute, and Ocean Networks Canada, with around a million additional images in the pipeline.

ThedebriefRead article

New app turns gamers into ocean explorers

Ocean Vision AI, a collaboration led by MBARI, developed FathomVerse to address the overwhelming volume of ocean imagery that exceeds researchers' analysis capacity, using gamification to teach AI how to classify marine animals. The beta launch in 2023 drew 1,396 players from 65 countries who interacted with real underwater images collected by researchers. The team is preparing to release the FathomVerse app this spring for smartphones and tablets.

FathomnetRead article

FathomVerse mobile game inspires a new wave of ocean exploration

MBARI Principal Engineer Kakani Katija developed FathomVerse to address the challenge of training AI to recognize ocean animals by engaging global ocean enthusiasts to review and label underwater images through mobile gameplay. "With an enormous challenge comes an enormous opportunity: How do we scale our capacity for analyzing valuable data about the ocean while realizing a new and inclusive vision for ocean exploration and discovery?" said Katija, who collaborated with Netherlands-based &ranj Serious Games and Kenya-based Internet of Elephants. The game launched today on the App Store and Google Play, targeting the more than 3 billion people who play video games globally.

FathomnetRead article