Betrayed by captions: Joint caption grounding and generation for open vocabulary instance segmentation
This paper presents a joint Caption Grounding and Generation (CGG) framework for instance-level open vocabulary segmentation. The main contributions are: (1) using fine-grained object nouns in captions to improve grounding with object queries. (2) using captions as supervision signals to extract rich information from other words helps identify novel categories. To our knowledge, this paper is the first to unify segmentation and caption generation for open vocabulary learning. The proposed framework significantly improves OVIS and OSPS and comparable results on OVOD without pre-training on large-scale datasets.
Oct 2, 2023
Towards efficient scene understanding via squeeze reasoning
Jul 30, 2021
PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation
Feb 13, 2021
Improving Semantic Segmentation via Decoupled Body and Edge Supervision
Jul 3, 2020