We propose a novel framework for the problem of minimizing domain expert annotation time for image segmentation using two adversarial and complementary multi-armed agents. Specifically, the framework contains two reinforcement agents where one’s goal is to reinforce the segmentation model’s current knowledge, and the others goal is to exploit the model’s weaknesses.
Abstract
Image segmentation is a powerful computer vision technique with many compelling applications. However, in some applications, such as in medicine and biology, the segmentation task requires expertlevel knowledge and is thus prohibitively expensive. Further, unlike in other domains, these applications require a very precise per pixel segmentation in order to be successful. Herein we propose a multi-armed image segmentation (MAIS) framework to minimize expert annotation time. This framework works by training both an adversarial multi-armed bandit (MAB) agent and a complementary MAB agent to iteratively find the next best set of images to be processed by the domain expert. The MAB decision machines encapsulate different learning algorithms using the parameters of the segmentation algorithm, and both the labeled and unlabeled data. We demonstrate the effectiveness of our framework on an original data set of the aquatic plant Eelgrass which requires the precise segmenting of diseased tissue from healthy tissue.
Framework
Authors
Brendan Rappazzo, Guillaume Perez, Runzhe Yang, Olivia Graham, Drew Harvell and Carla Gomes.