Complex channels consist of small receptive fields at first stage with large fields at the second stage separated by a nonlinearity of the rectification type. Responds to low-spatial-frequency patterns of high-spatial-frequency information. "Complex channels" are often called by other names, e.g. "second-order units", or "non-Fourier mechanisms."
Physiological substrate seems likely to be cortical area V1 or V2 in our texture-segregation task, but we are not sure, and similar processes probably occur in other cortical areas
What's on this page? To go to a section below on this page you can click here in this table of contents or scroll. - Choosing a suprathreshold task: texture segregation
- Ruling out a simple (Fourier, linear) channels model and introducing a family of decision rules appropriate for texture segregation tasks.
- Introducing Complex Channels
---Function of Complex Channels?
- Some further properties of Complex Channels
---Question 1: Is the first stage of complex channels selective for orientation or not? More generally, what is the orientation- and spatial-frequency bandwidth of the first stage of complex channels?
---Question 2: Is the intermediate nonlinearity in complex channels piecewise-linear? or expansive? or compressive?
---Question 3: Are the dynamics of simple and complex channels the same?
---Question 4: Is there orientation-opponent coding in the second-order channels of pattern vision?
---Question 5: What is the relationship between the preferred orientations (spatial frequencies) of the two stages of filtering?
---Question 6: (i) Are the outputs of the complex channels acting in texture segregation probabilistically independent? (ii) Are these outputs labeled as they go upstream?
---Some other questions about complex channels
The near-threshold tasks, briefly discussed on the Multiple-Analyzer page, were well suited to studying the multiple analyzers but we wanted to extend this model into suprathreshold tasks, thereby hoping to show some other processes that had all been subsumed into the decision rule of the multiple-analyzers model.
On many suprathreshold tasks the existing literature showed a great deal of trouble replicating results across laboratories (due perhaps to individual differences and undocumented learning effects), so we were interested in finding a task that, while suprathreshold, might not have these difficulties. We tried texture segregation partly for this reason as it was thought to be "immediate and effortless" process. It was also appealing because spatial frequency and orientation content of textures were already known to be important and thus we could build on the existing knowledge of spatial-frequency and orientation analyzers. Further, texture segregation experiments seemed likely to be a laboratory version of an important process in everyday perception -- a intermediate stage at which the visual system divides the visual scene into regions in order to reduce the computational load on higher stages of processing.
The observer's task in this kind of experiment is to indicate whether two regions containing different textures are seen immediately and effortless as being separate regions. Sometimes the observer is simply asked to rate the degree of perceived segregation (e.g. in the figure below, how well -- "immediately and effortlessly" -- do the three regions segregate perceptually?
In other experiments, the observer must say whether an embedded region of one texture is elongated horizontally or vertically (sometimes under cued conditions to force a response in a given time period, the "speed-accuracy-tradeoff paradigm", e.g. Sutter and Graham, 1995).
Lately (e.g. Wolfson and Graham, 2004 in press) we have also started using tasks involving only one region of texture (striped arrangement) and requiring the observer to IDENTIFY THE ORIENTATION of the stripes of elements or to DETECT THE PRESENCE OF STRIPES (that is, to discriminate the striped arrangement of two types of elements from the case where both types of elements are identical and thus there are no visible stripes).
The figure below shows a model -- based on the previous multiple-analyzer research -- that contains only linear spatial-frequency- and orientation-selective channels. Each of the simple channels in the figure below may be thought of as a set of neurons all having receptive fields identical except in position. Further, each channel is a linear systems (e.g. its response to a sum of 2 stimuli is the sum of its responses to the 2 by itself). So far this model is quite like the multiple analyzers model used for near-threshold experiments. However, the simple decision rule shown there is not applicable to suprathreshold experiments like the texture segregation experiment studied in our lab. Instead we have explored a family of decision rules, all of which have the form shown in the diagram below and can be expressed as relatively simple pooling equations. These decision rules all say that: an observer perceives two regions as perceptually segregated if and only there is sufficient difference between the spatial-frequencies and orientations contained in the two regions. See further information in Sutter, Beck and Graham, 1989; Graham, 1991; Graham, Beck, and Sutter,1992.
We tested this simple-channels model extensively on several kinds of texture and it was dramatically deficient, not only in ways that have to be described quantitatively but also in ways that are often considered qualitative (Sutter, Beck, and Graham, 1989; Graham, 1991; Graham, Beck, and Sutter, 1992).
One particularly clear demonstration occurs in constant-difference-series experiments which are discussed further on another page.
Some of the difficulties of the simple model in describing texture segregation phenomena could be corrected by postulating complex channels (Sutter, Beck, and Graham 1989; Graham, 1991; Graham, Sutter, and Beck 1992). We were particularly inspired by John Robson's suggestion (Robson, 1980) that complex cortical cells might perform tasks like texture segregation, but one should treat this possible physiological analog cautiously. Very similar structures have been suggested by others for both static and moving patterns (e.g. Chubb and Sperling, 1988). See introduction to Graham and Sutter (1998) for references to the work of many others who have made similar suggestions.
Robson, J. G. (1980) Neural images: The physiological basis of spatial vision. (C.S. Harris, Ed.) Visual coding and adaptability, Hillsdale, N.J.: Erlbaum, 177-214.
Chubb, C. and Sperling, G. (1988) Drift-balanced random stimuli: A general basis for studying non-Fourier motion perception. In Journal of the Optical Society of America A, 5, 1986-2007.
The complex channels used by us contain two stages of linear filtering separated by a pointwise nonlinearity, as illustrated here for one potential complex channel.
Click here to see a demonstration of the response of complex channels to a particular texture.
Note that in this complex channel, information is "pooled" from receptive fields at different spatial positions, all at the same spatial frequency and orientation. Complex channels of this type can be distinguished from the pooling across different spatial frequencies and orientations at the same spatial position as postulated in some other second-order mechanisms. (This distinction and a very-mini-review of a number of people's second-order mechanisms can be found in introduction to Graham and Sutter 1998.)
Complex channels can be described as responding to " low-frequency patterns of high-frequency information" as in the previously-mentioned demonstration of the response of complex channels.
Why should there be a need for complex or non-Fourier mechanisms like those here? (Where in nature do important "low-frequency patterns of high-frequency information" exist? ) A number of people have attempted to answer this question. One line of answer invokes occlusion by partially-transparent surfaces.
Quite different kinds of function have been suggested for other kinds of second-order structures--e.g. the piecing together of bits of stimulus that will eventually be seen as one contour.
Answer 1: Complex channels are orientation and spatial-frequency selective but not as narrow tuned as simple channels (Graham, Sutter, Venkatesan, and Humaran, 1992; Graham, Sutter, Venkatesan, 1993). This answer based on the degree of interference that occurs when two element types in an element-arrangement texture differ only in spatial frequency or orientation (and contrast).
Of course, the intrusion of other channels ("off-frequency" and "off-orientation" channels) and the properties of the intensive nonlinearity must be considered in arriving at this answer.
Example of texture used to investigate 1st-stage bandwidth
We have also investigated the first-stage-spatial-frequency bandwidth of complex channels involved in another task, a competing-paths paradigm (p. 325 in discussion after Graham, 1994). The bandwidth was very similar to that in our texture-segregation task.
Example of competing-paths stimulus to investigate 1st-stage bandwidth
The competing-paths task is analogous to that used by Werkhoven, Sperling, and Chubb,1993 to study non-Fourier mechanisms in motion perception. They found broader bandwidths for motion than we do for static patterns in either the texture-segregation or competing-paths experiments.
Answer 2: The intermediate nonlinearity is expansive with an exponent k probably greater than 2, perhaps 3 or 4 (Graham and Sutter, 1998).
Area-contrast-tradeoff experiments. This answer is based on the kind of tradeoff between contrast and area that occurs when the two element types in an element arrangement texture differ only in area (and contrast).
Examples of these patterns on another page.
That tradeoff is approximately linear for square-element gratings (i.e. the contrast in the small patch must be bigger than that in the large patch by about the ratio of their areas). The tradeoff is expansive for grating-element patterns (i.e. the contrast in the small patch need only be a little greater than that in the large patch - by much less than the area ratio -- for the two to balance out)
Again, the intrusion of other channels and the properties of the intensive nonlinearity must be considered. Substantial individual differences exist in some aspects of performance on this task; correlations among these differences suggest their cause is individual differences in the extent of intrusion by other complex channels.
Answer 3: No, complex channels are substantially slower as demonstrated in a speed-accuracy-tradeoff paradigm (Sutter and Graham, 1995; Sutter and Hwang, 1999). The stimuli were presented for 50 ms, and the response time was computed from stimulus onset. The observers were forced to respond within 200 msec after an auditory cue.
Sutter. A. and Hwang, D. (1999) A comparison of the dynamics of simple (Fourier) and complex (non-Fourier) mechanisms in texture segregation. Vision Research, 39, 1943-1962.
åIs there opponency between orientation-selective
processes in pattern perception, analogous to opponency between
color mechanisms? A number of investigators have suggested the
possibility for a number of different reasons. Thus, Graham and
Wolfson (2004) considered several possible second-order structures
as shown in the figure below:
(i) SIGN-opponent-only channels in which there is no opponency
between orientations (which is the structure we have generally
assumed for complex channels and what have also been called filter-rectify-filter
mechanisms);
(ii) three structures we group under the name ORIENTATION-opponent
;
(iii) BOTH-opponent channels which combine features of both SIGN-opponent-only
and ORIENTATION-opponent channels but lead to predictions that
are distinct from either of theirs.
Answer 4. There is no orientation opponency. The experimental results are consistent with SIGN-opponent-only channels.
We measured observers' ability to segregate textures composed of checkerboard and striped arrangements of vertical and horizontal Gabor grating patches. The observers' performance was compared to model predictions from the alternative opponent structures.The results ruled out the ORIENTATION-opponent and BOTH-opponent structures.
Further, when the models were expanded to include a contrast gain-control (inhibition among channels in a normalization network) the SIGN-opponent-only model was also able to explain a contrast-dependent effect we found, thus providing another piece of evidence that such normalization is an important process in human texture perception.
Question 5. What is the relationship between the preferred orientations (spatial frequencies) of the two stages of filtering?
For example, are horizontal receptive fields at the first stage only coupled more with horizontal receptive fields at the second stage? Or are they equally likely to be coupled with all orientations at the second stage? Or what?
Similarly, do relatively-low preferred spatial frequencies at the first stage of filtering imply that the spatial frequency of the second stage will also be relatively low?
Answer 5. There is probably a slight positive correlation between the preferred values at the two stages.
About orientation: There is a bias toward similar orientations at the two stages. We explored these mappings using a texture-segregation task with textures composedof arrangements of grating-patch elements. We considered two possible mappings: that where the orientations at the two stages are identical (called ''consistent'' here) and that where the orientations at the two stages are per-pendicular (''inconsistent'').The results imply that, for our tasks, complex channels with a consistent orientation mapping must be either somewhat more prevalent or more effective than those with an inconsistent mapping (Graham and Wolfson, 2001).
About spatial frequency: Perhaps the fairest conclusion at the point is that, if there is a correlation between the preferred spatial frequencies at the two stages of second-order channels, it is very small and is a bias toward the preferred spatial frequency of the first filter being several octaves than that of the second. (See discussion, p. 2280, of Graham and Wolfson, 2001)
Question 6. (i) Are the outputs of the complex channels acting in texture segregation probabilistically independent? (ii) Are these outputs labeled as they go upstream?
Answer 6. The outputs of complex channels seem to be probabilistically independent and labelled.
The results of some detection and identification experiments under conditions of certainty or uncertainty (Wolfson and Graham, in press 2004) showed uncertainty effects in the detection of second-order patterns. Also, the (second-order) orientation of these patterns can be identified as soon as it is detected. These results in turn suggest that the complex channels are noisy and probabilistically independent, with labeled outputs so that processes upstream can selectively attend to subsets and can identify them. Further there is no apparent capacity limitation to this attention, at least when only two subsets of channels are involved. Thus, the complex channels are like the simple (first-order) spatial-frequency- and orientation-selective channels in this regard, and these detection and identification experiments can be interpreted by multidimensional signal-detection models analogous to those used for the detection of near-threshold patterns (to relevant section for these models).
Other known (and unknown) properties of complex
(second-order, non-Fourier) channels are briefly discussed in
our papers mentioned above. However, those discussions are already
out of date, as new work is regularly appearing on the properties
of second-order mechanisms for spatial tasks (as in texture segregation)
and for (probably different) second-order mechanisms in motion
perception. Some of these other questions are mentioned here (although
we make no attempt to reference everyone's work here unfortunately):