Contours of Outline Shapes Derived from Everyday Objects
Abstract and Keywords
Computational theories of shape perception predicted that shape information is concentrated along the contour, primarily at locations where curvature takes on special values (maxima, minima, and inflection points). Psychophysical studies inspired by the computational theories explored how humans encode shape and how they identify and segment outlines of everyday objects. These studies found that shape information was distributed along the contour as predicted. But the local neighborhoods and global configurations in which the curvature singularities were embedded also played an important role. This work revealed highly flexible interaction of local and global processes that change as visual processing unfolds over time. The results provide benchmarks for computational models of shape-based object encoding.
Computational theories of shape representation must account for the interaction of local and global aspects of shape used for identification and segmentation of objects.
A few well-placed line segments on a flat canvas are often sufficient to identify objects, but not all points along an object’s boundary are equally informative about its shape. Attneave (1954) advanced the hypothesis that curvature extrema (i.e., points along the contour where curvature reaches a local maximum) are most informative about shape. He used two demonstrations to support this hypothesis. In one demonstration, he asked participants to mark salient points along the contour of a random shape. Frequencies of the marks were highest on the curvature extrema. In the second demonstration, which has become known as Attneave’s sleeping cat, he created a version of a line drawing of his sleeping cat by connecting the curvature extrema by straight lines. The straight-line version was still easy to recognize.
Attneave’s hypothesis inspired theoretical work in at least two directions. First, using classic measures from information theory, Resnikoff (1989) provided a mathematical proof of the information concentration in curvature extrema. More recently, Feldman and Singh (2005) proved that negative curvature extrema (minima) must be more informative than positive curvature extrema (maxima). (Curvature is conventionally defined to have negative values corresponding to “dimples” or “cups” and positive values corresponding to “bumps” or “caps.”) All natural objects are more convex than concave. Therefore, closed contours always turn inward after having turned away from the object center. Second, Koenderink and van Doorn (1976) and Koenderink (1984) showed that, on three-dimensional (3D) surfaces, contour fragments of positive curvature (or convexities) correspond to regions of positive Gaussian curvature (i.e., bump-like regions), whereas contour fragments of negative curvature (or concavities) correspond to regions of negative Gaussian (i.e., saddle-like regions). Accordingly, inflections (i.e., points where curvature changes sign and goes through zero locally) might also be informative because they correspond to parabolic lines on the 3D surface, separating convex and concave surface regions (Koenderink & van Doorn, 1982). Moreover, inflections are based on lower order derivatives, and so they are more stable under small viewpoint changes than are points of extrema. This may be why the visual system makes use of inflections (Van Gool, Moons, Pauwels, & Wagemans, 1994).
To test the role of curvature singularities for shape perception, we investigated perception of a large stimulus set in a variety of tasks (reviewed in De Winter & Wagemans, 2004). To preview, the main conclusions from this line of work are:
1. In both identification and segmentation of object outlines, curvature singularities (extrema and inflections) play important roles, but no single type of curvature singularity dominated perception.
2. As expected, shape information is concentrated in special points along the contour, but local surrounds, two-dimensional (2D) surface patches bounded by contour segments, and global spatial regularities (such as parallelism and symmetry) are also important.
3. Computational theories of shape representation will have to account for the interplay between local and global aspects of shape and also specify how the essential information is extracted rapidly and reliably from natural visual stimuli.
The Role of Curvature Singularities in Shape Perception
We derived stimulus sets from line drawings of everyday objects and asked large groups of observers to identify and segment them. We manipulated information about shape and studied the role of curvature singularities for object identification and segmentation.
Silhouette and Outline Versions
As a point of departure, we used a well-known set of line drawings from Snodgrass and Vanderwart (1980), turned them into black silhouettes, extracted their contours, and spline-fitted them to obtain smoothly curved, closed contours, with known curvature values at all points along the contour (see Figure 1, top row). Not all of these boundary shapes can still be identified. Identifiability ranged from 0% to 100%. We have established norms for the identification of these silhouette and outline versions (i.e., the amount of deformation of the original outline tolerated to yield a certain level of identifiability). We then studied why identifiability of different stimuli was highly variable (Wagemans et al., 2008). We view these data as a benchmark for testing computational theories of 3D object recognition because the results illustrate a wide range of difficulties for perception of shape having to do with diagnostic versus degenerate views, surface structure, texture, diagnostic features, different types of line segments, junctions, and the like.
We then asked subjects to mark salient points along the contours of these shapes. We found that subjects often picked points in the neighborhood of curvature extrema (Figure 2), thus confirming Attneave’s original observation. But we found that more global shape factors also played a significant role (De Winter & Wagemans, 2008a). The saliency of a point along the contour did not only depend on how high was the maximum or how long was the minimum, but it also depended on the local neighborhood (e.g., the contour segment in which it was embedded) and the part structure (e.g., how much a part is seen to protrude or how deep is the perceived indentation between the two parts). Such data evidently constrain computational theories of shape representation, and they are indicative of the spatial scale (i.e., the level of detail vs. smoothness of contour curvature variations) at which object outlines are perceived.
We then created figures consisting of straight-line segments connecting different points along the contour, such as curvature extrema versus inflection points, or salient points versus points midway between them (Figure 1, bottom row, two left images). Again, we have established norms for identification of these straight-line figures allowing us to reveal the reasons for large differences in identifiability across stimuli (De Winter & Wagemans, 2008b). The straight-line figures based on extrema or salient points were generally easier to identify, again confirming Attneave’s intuition illustrated in his famous drawing of a sleeping cat. However, we also noted that the degree to which the straight-line version represented the overall part-structure of the outline shape strongly affected shape identifiability. In some cases, the straight lines preserved the overall structure of the shape, whereas in others they created a different structure. That is, adding line segments between points along the contour can do much more to the resulting shape than just marking these points (a notion often overlooked in discussions of Attneave’s cat).
Next, we created versions of these stimuli consisting of small fragments, centered either on curvature extrema or salient points, on the one hand, or on inflection points or midpoints, on the other. Once again, we have established norms for identification of these fragmented figures and explored the reasons for variable identifiability of stimuli (Panis, De Winter, Vanderkerckhove, & Wagemans, 2008). This time, however, we found that the fragmented figures with contour elements centered on extrema or salient points were generally more difficult to identify than those where contour elements were centered on inflections or midpoints (Figure 1, bottom row, two right images), contradicting the idea that most information is carried by the curvature extrema. Apparently, the fact that the fragments required some perceptual grouping to occur yielded effects opposite to those obtained in figures with closed contours. This observation illustrates the general point that a computational theory of geometric sources of information should also take into account task demands.
Microgenesis of Fragmented Picture Identification
The role of perceptual grouping and the effects of different types of fragments in different types of shape (simple vs. complex, natural vs. artificial) were later studied in more detail using a discrete identification paradigm (which is extending the presentation time until identification) and a discrete time survival analysis (Panis & Wagemans, 2009). These methods allowed us to study the gradual emergence (microgenesis) of fragmented picture identification, including interesting time-course contingencies between component processes (such as perceptual grouping of image fragments) to construct a shape description on the one hand and to match the resulting shape description to existing object representations in visual memory on the other hand. For instance, configural properties (such as symmetry) dominated early grouping processes, compared to local fragment properties (such as fragment curvature) or local relational properties (such as proximity). The complexity of shape also played a role that changed over time. For instance, low-complexity objects showed a decreasing disadvantage compared to medium-complexity objects because possible matches for simpler objects are more numerous. In contrast, high-complex objects showed an increasing advantage due to a low number of activated candidates. Similar results were subsequently obtained in a version of the discrete identification paradigm, in which the fragmented pictures were gradually built up from very sparse, short fragments to longer fragments and almost complete contours (Figure 3; Torfs, Panis, & Wagemans, 2010).
Segmentation into Parts
Using a subset of the same outline stimuli, we tested how human observers segment object outlines into parts (De Winter & Wagemans, 2006). This study provided benchmark data for how humans segment object shapes, using the different segmentation models. In general, we were able to confirm several segmentation rules (Figure 4) previously proposed from studies of simple geometric shapes or limited sets of objects (e.g., the minima rule, the short cut rule). We found several examples of known part types, such as necks and limbs. Many segmentations were located near strong negative minima, although we also found segmentation points around positive maxima and inflections. As predicted by some segmentation theories (e.g., Hoffman & Singh, 1997), semilocal factors (such as proximity and good continuation) also affected segmentation. Comparing objects that were easily identifiable with objects that were hard to identify, but which were equally simple or complex, we found clear examples of top-down effects; namely, segmentations at places without clear curvature changes but a known part structure. For instance, the tail of the rhino in the bottom-right of Figure 4 is segmented, although there is no special point of curvature on the outer edge, which corresponds to the deep minimum on the inner edge.
Overall, segmentation of objects into parts was determined by an intricate interplay of visual and cognitive factors, which we unified into a single framework (Figure 5). This framework, which could be developed into a computational model, clarifies how different types of parts result from combining segmentation cues along the contour. For instance, a limb may result from combining negative minima with collinearity, or a neck may result from combining negative minima with proximity. For other (hitherto unknown) part types, more global shape properties (such as symmetry and elongation axes) come into play. For such factors, we illustrate which geometrical properties determine their strengths, derived from an analysis of our database of segmentations. In some cases, there were no clear grounds for segmentation other than top-down cognitive factors.
Overall, this research program empirically tested previous ideas about the importance of curvature singularities for segmentation and recognition of objects. We found that properties of 2D shape often play a crucial role, in addition to contours (which are one-dimensional [1D] sequential descriptions). All in all, these studies clearly demonstrated a central role of perceptual organization.
Two Important Caveats
Although this mini-review has been focused on the role of contour in perception of shape, two important qualifications must be made to frame these results correctly with respect to low-level and high-level visual processes.
The Distinction Between Contours and Borderlines
First, much of the literature on contours is not about contours that are bounding shapes. For instance, the “snake detection” task in arrays of small oriented elements (e.g., Field, Hayes, & Hess, 1993) does not address contour grouping in this sense. Contours that are borderlines of shapes are characterized by border ownership: one side of the contour owns the figure/shape (e.g., Fulvio & Singh, 2006; Nakayama, Shimojo, & Silverman, 1989; Rubin, 1915). This is a key distinction that arose from our model of subjective contours (such as in the Kanizsa square or triangle), which is better identified as a phenomenon of surface filling-in than of contour filling-in (as we argued in Kogo et al., 2010, based on an extensive review of the literature). Occlusion cues play a central role in this model, and this is consistent with the notion that responses of cells in cortical areas V1 and V2 are differentiated signals that also contain information about border ownership (as shown by von der Heydt and colleagues; e.g., Zhou, Friedman, & von der Heydt, 2000). The differentiated signals are then integrated spatially, yielding both a depth map and a lightness map. The model has been tested using many variations of the Kanizsa figure. In conclusion, we wish to emphasize that perceptual grouping of oriented elements (such as Gabor patches or contour fragments) does not entail border ownership, which is key to the perception of shape and object outlines (see Kogo & Wagemans, 2011; 2013, for further discussion).
The Importance of Contours Versus Other Aspects of Shape
Although most of the just described work implicitly assumed that shapes were encoded mainly by their contours, other work provides clear evidence that there is more to shape than 1D contour descriptions. Spatially extended aspects of 2D shape (such as overall simplicity and symmetry) also play a role, both for human observers (Kayaert & Wagemans, 2009) and for cells in inferotemporal (IT) cortex of the macaque monkey (Kayaert, Wagemans, & Vogels, 2011). In our earlier work (Op de Beeck, Wagemans, & Vogels, 2001), we established correspondences between perceived shape similarity for human and macaque and single-cell tuning in IT based on global shape properties (such as indentations, part curvature, etc.) rather than on small-scale contour detail. Recent work using functional magnetic resonance imaging (fMRI) replicated this finding and extended it to the encoding of 3D shape (Op de Beeck, Torfs, & Wagemans, 2008).
Contours provide important information about object shape, especially at the points where contour curvature has singular values (inflections and extrema). However, no matter how much one reduces the available information (e.g., by presenting only contour fragments), human perception extracts and encodes relational properties, such as relative distance, parallelism, and so forth. Curvature singularities play a role that depends on both the local neighborhood and the global configuration in which they are embedded. Shape perception, shape-based object identification, and segmentation all require perceptual organization. Developing computational models that do justice to the flexible and efficient interplay of local and global processes remains a significant challenge.
This research is supported by long-term structural funding from the Flemish Government (METH/08/02). The chapter was written with support from the Research Foundation–Flanders (K8.009.12N). I would also like to acknowledge the hospitality of the Department of Psychology at the University of California, Berkeley; the Institut d’études avancées (IEA), Paris; and the Department of Experimental Psychology at the University of Oxford.
Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review, 61, 183–193.Find this resource:
De Winter, J., & Wagemans, J. (2004). Contour-based object identification and segmentation: Stimuli, norms and data, and software tools. Behavior Research Methods, Instruments, & Computers, 36, 604–624.Find this resource:
De Winter, J., & Wagemans, J. (2006). Segmentation of object outlines into parts: A large-scale, integrative study. Cognition, 99, 275–325.Find this resource:
De Winter, J., & Wagemans, J. (2008a). Perceptual saliency of points along the contour of everyday objects: A large-scale study. Perception & Psychophysics, 70, 50–64.Find this resource:
De Winter, J., & Wagemans, J. (2008b). The awakening of Attneave’s sleeping cat: Identification of everyday objects on the basis of straight-line versions of outlines. Perception, 37, 245–270.Find this resource:
Feldman, J., & Singh, M. (2005). Information along contours and object boundaries. Psychological Review, 112, 243–252.Find this resource:
Field, D. J., Hayes, A., & Hess, R. F. (1993). Contour integration by the human visual system: Evidence for a local “association field.” Vision Research, 33, 173–193.Find this resource:
Fulvio, J. M., & Singh, M. (2006). Surface geometry influences the shape of illusory contours. Acta Psychologica, 123(1–2), 20–40.Find this resource:
Hoffman, D., & Singh, M. (1997). Salience of visual parts. Cognition, 63, 29–78.Find this resource:
Kayaert G. & Wagemans J. (2009). Delayed shape matching benefits from simplicity and symmetry. Vision Research, 49, 708–717.Find this resource:
Kayaert, G., Wagemans, J., & Vogels, R. (2011). Encoding of complexity, shape and curvature by macaque infero-temporal neurons. Frontiers in Systems Neuroscience, 5(51), 1–16. doi: 10.3389/fnsys.2011.00051Find this resource:
Koenderink, J. J. (1984). What does the occluding contour tell us about solid shape? Perception, 13, 321–330.Find this resource:
Koenderink, J. J., & van Doorn, A. J. (1976). The singularities of the visual mapping. Biological Cybernetics, 24, 51–59.Find this resource:
Koenderink, J. J., & van Doorn, A. J. (1982). The shape of smooth objects and the way contours end. Perception, 11, 129–137.Find this resource:
Kogo, N., Strecha, C., Van Gool, L., & Wagemans, J. (2010). Surface construction by a 2-D differentiation-integration process: A neurocomputational model for perceived border ownership, depth, and lightness in Kanizsa figures. Psychological Review, 117, 406–439.Find this resource:
Kogo, N., & Wagemans, J. (2011). What is required for a signal to be qualified as a “grouping” tag? British Journal of Psychology, 102, 676–681.Find this resource:
Kogo, N., & Wagemans, J. (2013). The emergent property of border-ownership and the perception of illusory surfaces in a dynamic hierarchical system. Cognitive Neuroscience, 4, 31–61. doi: 10.1080/17588928.2012.727387Find this resource:
Nakayama, K., Shimojo, S., & Silverman, G. H. (1989). Stereoscopic depth: Its relation to image segmentation, grouping, and the recognition of occluded objects. Perception, 18, 55–68.Find this resource:
Op de Beeck, H. P., Torfs, K., & Wagemans, J. (2008). Perceived shape similarity among unfamiliar objects and the organization of the human object vision pathway. Journal of Neuroscience, 28, 10111–10123.Find this resource:
Op de Beeck, H., Wagemans, J., & Vogels, R. (2001). Macaque inferotemporal neurons represent low-dimensional configurations of parameterized shapes. Nature Neuroscience, 4, 1244–1252.Find this resource:
Panis, S., & Wagemans, J. (2009). Time-course contingencies in perceptual organization and identification of fragmented object outlines. Journal of Experimental Psychology: Human Perception and Performance, 35, 661–687.Find this resource:
Panis, S., De Winter, J., Vandekerckhove, J., & Wagemans, J. (2008). Identification of everyday objects on the basis of fragmented versions of outlines. Perception, 37, 271–289.Find this resource:
Resnikoff, H. L. (1989). The illusion of reality. New York: Springer-Verlag.Find this resource:
Rossion, B., & Pourtois, G. (2004). Revisiting Snodgrass and Vanderwart’s object pictorial set: The role of surface detail in basic-level object recognition. Perception, 33, 217–236.Find this resource:
Rubin, E. (1915). Synsoplevede Figurer. Studier i psykologisk Analyse /Visuell wahrgenommene Figuren. Studien in psychologischer Analyse [Visually perceived figures. Studies in psychological analysis]. Copenhagen, Denmark/Berlin, Germany: Gyldendalske Boghandel.Find this resource:
Snodgrass, J. G., & Vanderwart, M. (1980). A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6, 174–215.Find this resource:
Torfs, K., Panis, S., & Wagemans, J. (2010). Identification of fragmented object outlines: A dynamic interplay between different component processes. Visual Cognition, 18, 1133–1164.Find this resource:
Van Gool, L., Moons, T., Pauwels, E., & Wagemans, J. (1994). Invariance from the Euclidean geometer’s perspective. Perception, 23, 547–561.Find this resource:
Wagemans, J., De Winter, J., Op de Beeck, H. P., Ploeger, A., Beckers, T., & Vanroose, P. (2008). Identification of everyday objects on the basis of silhouette and outline versions. Perception, 37, 207–244.Find this resource:
Zhou, H., Friedman, H. S., & von der Heydt, R. (2000). Coding of border ownership in monkey visual cortex. Journal of Neuroscience, 20(17), 6594–6611.Find this resource: