Frank R. Schmidt

Interactive Segmentation with Super-Labels

Andrew Delong, Lena Gorelick, Frank R. Schmidt, Olga Veksler, Yuri Boykov
Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), Volume 6819, page 147--162 - Jul 2011
Download the publication: 9 MB 
Share the publication:
In interactive segmentation, the most common way to model object appearance is by GMM or histogram, while MRFs are used to encourage spatial coherence among the object labels. This makes the strong assumption that pixels within each object are i.i.d. when in fact most objects have multiple distinct appearances and exhibit strong spatial correlation among their pixels. At the very least, this calls for an MRF-based appearance model within each object itself and yet, to the best of our knowledge, such a "two-level MRF" has never been proposed.
We propose a novel segmentation energy that can model complex appearance. We represent the appearance of each object by a set of distinct spatially coherent models. This results in a two-level MRF with "super-labels" at the top level that are partitioned into "sub-labels" at the bottom. We introduce the hierarchical Potts (hPotts) prior to govern spatial coherence within each level. Finally, we introduce a novel algorithm with EM-style alternation of proposal, α-expansion and re-estimation steps.
Our experiments demonstrate the conceptual and qualitative improve- ment that a two-level MRF can provide. We show applications in binary segmentation, multi-class segmentation, and interactive co-segmentation. Finally, our energy and algorithm have interesting interpretations in terms of semi-supervised learning.

Images and movies

superlabel0.png (56 KB)
superlabel1.png (39 KB)

BibTex references

  author       = "Delong, Andrew and Gorelick, Lena and Schmidt, Frank R. and Veksler, Olga and Boykov, Yuri",
  title        = "Interactive Segmentation with Super-Labels",
  booktitle    = "Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR)",
  series       = "LNCS",
  volume       = "6819",
  pages        = "147--162",
  month        = "Jul",
  year         = "2011",
  publisher    = "Springer",
  address      = "Saint Petersburg, Russia",
  url          = ""