Image Structure and Syntax for Recognition

What general purpose image representation could serve the diverse needs of image recognition? How to represent the spectral and spatial structure of an image? What makes good image syntax? How do we evaluate it? In this paper, we present partial answers to these and related questions. The syntax we present is defined in terms of image regions, or segments. It captures the recursive embedding of all regions, their geometric and photometric properties, and their spatial layout. We discuss its invariance to changes in imaging conditions (e.g., lighting, scale, orientation), and its ability to isolate and simplify inference of semantics, as would be expected from any syntax. We describe some properties of such syntax, and evaluate its performance on some basic recognition problems.