TruthMarker

Name: TruthMarker
Author: University of Iowa

University of Iowa

VALIDATED

Initial evidence from research studies

For:Researchers & AcademicsIndustry Professionals

Free

Download on the App Store

App Summary

TruthMarker is a tablet-based tool for researchers to rapidly annotate images with custom data types, creating datasets to build and validate automated analysis models. In an evaluation where medical experts annotated ophthalmic images for diabetic retinopathy, the associated research found that annotations created with TruthMarker were of equivalent quality to those made using standard desktop-based tools. The authors conclude that the tool streamlines the collection of high-quality annotations, benefiting researchers who are developing and validating diverse image analysis techniques.

App Screenshots

Detailed Description

Functionality & Mechanism TruthMarker is a tablet-based system designed for the rapid annotation of image data in research settings. The platform facilitates the creation of ground truth datasets required for developing and validating automated analysis models. Researchers initiate a project by defining a custom annotation model via an XML file, which configures the interface for the specific task. The system captures rich datatypes, including splines, points, lines, and categorical values, generating annotations in a standardized, parsable format for subsequent analysis.

Evidence & Research Context

The associated research details the system's design for streamlining annotation collection to build and validate automated image analysis models.
A comparative evaluation assessed annotations from medical experts grading diabetic retinopathy severity on ophthalmic images using TruthMarker versus desktop tools.
The study determined that tablet-generated annotations were of equivalent quality to those from desktop systems, based on kappa (κ) statistics and accuracy.
A core design principle is the system's configurability, allowing researchers to define custom annotation models to fit specific research goals.

Intended Use & Scope This system is intended for image analysis researchers, data scientists, and clinical investigators requiring structured image annotations. Its primary utility is the creation of high-quality ground truth datasets for training and validating computational models. The software is a data collection tool and does not perform automated image analysis or provide diagnostic interpretations.

Studies & Publications

1 publication

Peer-reviewed research associated with this app.

Validation Study

Truthmarker: a tablet-based approach for rapid image annotation

Christopher et al. (2011) · University of Iowa

App annotations accurately matched desktop-based tools in quality and accuracy.

The development of automated techniques for the analysis of image data is an important and active area of research. To make progress, this research requires annotations of image data to build and validate models used for analysis. Given this requirement, the development of software tools that streamline the collection of annotations would be of great benefit to image analysis researchers. Such tools should meet the following requirements: rapid generation of annotations for large data sets, annotation and data management that is straightforward for users, flexibility for application to many diverse image datasets, configurability to allow the collection of annotations to be tuned for a specific research goal, and generation of annotation data in a standardized format so that it can be easily parsed and analyzed. Truthmarker was designed as a tablet computer based image annotation tool to meet these requirements. Researchers can configure Truthmarker to fit the needs of a particular study by specifying an annotation model that fine tunes the user interface and resulting data to fit the annotation task. The quality of annotations generated using Truthmarker was evaluated by recruiting medical experts to annotate ophthalmic images for severity of diabetic retinopathy, a leading cause of blindness. These annotations were compared to annotations of the same images assigned using standard desktop computer based tools. The results, as measured by ? statistics and accuracy, indicate that Truthmarker annotations were of equivalent quality compared to those that were created using desktop-based tools.