OSC-CO2: coattention and cosegmentation framework for plant state change with multiple features

Rubi Quiñones, Ashok Samal, Sruti Das Choudhury, Francisco Muñoz-Arriola

Research output: Contribution to journalArticlepeer-review

Abstract

Cosegmentation and coattention are extensions of traditional segmentation methods aimed at detecting a common object (or objects) in a group of images. Current cosegmentation and coattention methods are ineffective for objects, such as plants, that change their morphological state while being captured in different modalities and views. The Object State Change using Coattention-Cosegmentation (OSC-CO2) is an end-to-end unsupervised deep-learning framework that enhances traditional segmentation techniques, processing, analyzing, selecting, and combining suitable segmentation results that may contain most of our target object’s pixels, and then displaying a final segmented image. The framework leverages coattention-based convolutional neural networks (CNNs) and cosegmentation-based dense Conditional Random Fields (CRFs) to address segmentation accuracy in high-dimensional plant imagery with evolving plant objects. The efficacy of OSC-CO2 is demonstrated using plant growth sequences imaged with infrared, visible, and fluorescence cameras in multiple views using a remote sensing, high-throughput phenotyping platform, and is evaluated using Jaccard index and precision measures. We also introduce CosegPP+, a dataset that is structured and can provide quantitative information on the efficacy of our framework. Results show that OSC-CO2 out performed state-of-the art segmentation and cosegmentation methods by improving segementation accuracy by 3% to 45%.

Original languageEnglish (US)
Article number1211409
JournalFrontiers in Plant Science
Volume14
DOIs
StatePublished - 2023

Keywords

  • cosegmentation
  • high-throughput plant phenotyping
  • image analysis
  • image sequences
  • multiple dimensions
  • multiple features
  • object state change
  • segmentation

ASJC Scopus subject areas

  • Plant Science

Fingerprint

Dive into the research topics of 'OSC-CO2: coattention and cosegmentation framework for plant state change with multiple features'. Together they form a unique fingerprint.

Cite this