TY - JOUR
T1 - OSC-CO2
T2 - coattention and cosegmentation framework for plant state change with multiple features
AU - Quiñones, Rubi
AU - Samal, Ashok
AU - Das Choudhury, Sruti
AU - Muñoz-Arriola, Francisco
N1 - Publisher Copyright:
Copyright © 2023 Quiñones, Samal, Das Choudhury and Muñoz-Arriola.
PY - 2023
Y1 - 2023
N2 - Cosegmentation and coattention are extensions of traditional segmentation methods aimed at detecting a common object (or objects) in a group of images. Current cosegmentation and coattention methods are ineffective for objects, such as plants, that change their morphological state while being captured in different modalities and views. The Object State Change using Coattention-Cosegmentation (OSC-CO2) is an end-to-end unsupervised deep-learning framework that enhances traditional segmentation techniques, processing, analyzing, selecting, and combining suitable segmentation results that may contain most of our target object’s pixels, and then displaying a final segmented image. The framework leverages coattention-based convolutional neural networks (CNNs) and cosegmentation-based dense Conditional Random Fields (CRFs) to address segmentation accuracy in high-dimensional plant imagery with evolving plant objects. The efficacy of OSC-CO2 is demonstrated using plant growth sequences imaged with infrared, visible, and fluorescence cameras in multiple views using a remote sensing, high-throughput phenotyping platform, and is evaluated using Jaccard index and precision measures. We also introduce CosegPP+, a dataset that is structured and can provide quantitative information on the efficacy of our framework. Results show that OSC-CO2 out performed state-of-the art segmentation and cosegmentation methods by improving segementation accuracy by 3% to 45%.
AB - Cosegmentation and coattention are extensions of traditional segmentation methods aimed at detecting a common object (or objects) in a group of images. Current cosegmentation and coattention methods are ineffective for objects, such as plants, that change their morphological state while being captured in different modalities and views. The Object State Change using Coattention-Cosegmentation (OSC-CO2) is an end-to-end unsupervised deep-learning framework that enhances traditional segmentation techniques, processing, analyzing, selecting, and combining suitable segmentation results that may contain most of our target object’s pixels, and then displaying a final segmented image. The framework leverages coattention-based convolutional neural networks (CNNs) and cosegmentation-based dense Conditional Random Fields (CRFs) to address segmentation accuracy in high-dimensional plant imagery with evolving plant objects. The efficacy of OSC-CO2 is demonstrated using plant growth sequences imaged with infrared, visible, and fluorescence cameras in multiple views using a remote sensing, high-throughput phenotyping platform, and is evaluated using Jaccard index and precision measures. We also introduce CosegPP+, a dataset that is structured and can provide quantitative information on the efficacy of our framework. Results show that OSC-CO2 out performed state-of-the art segmentation and cosegmentation methods by improving segementation accuracy by 3% to 45%.
KW - cosegmentation
KW - high-throughput plant phenotyping
KW - image analysis
KW - image sequences
KW - multiple dimensions
KW - multiple features
KW - object state change
KW - segmentation
UR - http://www.scopus.com/inward/record.url?scp=85176571352&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85176571352&partnerID=8YFLogxK
U2 - 10.3389/fpls.2023.1211409
DO - 10.3389/fpls.2023.1211409
M3 - Article
C2 - 38023863
AN - SCOPUS:85176571352
SN - 1664-462X
VL - 14
JO - Frontiers in Plant Science
JF - Frontiers in Plant Science
M1 - 1211409
ER -