Signal Detection Theory and Signal Effectiveness

The major problem with using receiver responses as an index of signal effectiveness is that responses confound the effects of the amount of information provided by a signal, the receiver’s estimates of prior probabilities, and the relative payoffs of alternative actions. A female faced with choosing between two displaying males may fail to discriminate between their displays a) because the differences are too small for her to detect, or b) it does not pay for her to expend the effort to compare them.

Signal detection theory provides tools for separating the roles of the amount of information in signals from the value of that information. It allows one to compute an index, called **receiver sensitivity** and denoted by *d'*, that can be used as another measure of the effectiveness of a signal set. The following discussion assumes that the reader is familiar with the general approach of signal detection theory as summarized in Web Topic 8.4.

Consider a hypothetical example in which females seek to identify a healthy male instead of a sick male as a mate. All males sing songs and song speed varies continuously among males. However, the distribution of song speeds for healthy males has a higher mean value than that for sick males. Let male song speed be denoted by *w*. The task for each female is to define a “red line” at some critical value *w _{c}* such that any male whose song speed exceeds w

If the distributions of song speed for sick and healthy males are at all overlapping, a female invoking her particular *w _{c}* will make some correct choices and some errors. Let

Now consider three different populations of males that vary in the degree to which the distributions of song rate for sick and healthy males overlap. In each example, a graph on the left will plot song speed (*w*) on the horizontal axis and the probability that a given type of male (sick or healthy) will sing that song speed on the vertical axis. In all cases, we shall assume the distributions are roughly bell-shaped with the same variances. The mean song speed for each distribution is the one under the peak value of the bell-curve’s vertical axis. Consider first a population (A) in which there is little or no difference in the mean values of sick and healthy male song speeds: the two distributions are completely overlapping. This is shown on the left graph below:

Suppose we select pairs of healthy and sick males at random from this population and record their songs. We then play the two songs back to a test female from that population and see which speaker she approaches. We do this multiple times with different pairs of randomly sampled males to get an estimate of how often she correctly selects the healthy males (*P _{hit}*) and how often she incorrectly selects sick ones (

If the distributions of song speed for sick and healthy males are completely overlapping, it will be impossible for females to make accurate discriminations between them using song speed: there is no correlation between song speed and health, and it should be obvious that attending to song speed provides no information to females. In this case, the ROC graph is a straight line as shown in this example: *P _{hit}* and

Next, consider a population (B) in which song speed is somewhat correlated with male health. This implies that the two distributions are not entirely overlapping, and there is thus a non-zero difference between their means. Let us call that difference *d'.*

If we now undertake playbacks of sick and healthy male songs to a series of females, we will get the plot of *P _{hit}* versus

In population (C), the correlation between male song speed and male health is even stronger than in population (B). The difference between distribution means, *d'*, is a much larger number and the curvature of the ROC plot towards the upper left corner of the graph is even stronger:

These examples suggest that one should be able to estimate the difference between the distribution means, *d',* by estimating the degree to which the curvature in the ROC plots deviates from the straight line expected when there is no correlation between signal and condition. And surprisingly, this measure of the amount of information can be obtained using receiver responses. Even more surprising is the observation that if both distributions are bell-shaped and have similar variances, any pair of *P _{hit}* and

The major point of measures of signal effectiveness is to be able to compare one signal set to another, or perhaps obtain an average value for how effective most threat signals or most alarm signals are. Clearly, one cannot compare *d' *values if the units for one signal set are in songs/second and another is in brightness of red plumage coloration. As long as the relevant distributions are bell-shaped (Gaussian) or can be made so with appropriate transformations, one can convert the *w* values in any distribution plot into **z scores**. This is a scaling widely used in statistics and computed as follows. If the mean of a normal distribution is *μ*, and its **standard deviation** is *σ* , (where *σ* = √ variance
), then the *z *score for *w* is

We can thus replot any original probability distribution of *w* values as a probability distribution of *z*(*w*) values. This distribution will have its maximum when *z*(*w*) = 0 (e.g. when *w* = *μ*), and all *z*(*w*) values to the left of this peak will be negative (e.g. *w* < *μ*), and all *z*(*w*) values to the right of the peak will be positive (*w* > *μ* ). The difference between the means of two *z*-scaled distributions, *d'*, will then be given as a multiple of their common standard deviation (if it is the same for both), or as a multiple of their average standard deviation (if they are different). Because *d'* is measured in standard deviation units, decreasing the average standard deviation of the distributions is equivalent to increasing the distances between their means: either reduces overlap between the distributions, and thus reduces errors.

Let the means for the two probability distributions be *μ*_{1} for healthy males and *μ*_{2} for sick males. We wish to convert the *w* axis for each distribution into *z*(*w*) values. For the first distribution,

and for the second distribution and the same *w*,

We note that

$${z}_{2}\left(w\right)-{z}_{1}\left(w\right)=\frac{{\mu}_{1}-{\mu}_{1}}{\sigma}=\mathrm{d\text{'}}$$which is the measure we seek. We can thus estimate *d' *if we can estimate *z*_{1}(*w*) and* z*_{2}(*w*) from observations of a female’s decisions.

Suppose we perform our playback experiments on a female using songs of sick and healthy males from the same population. We now have values for *P _{hit}* and

An additional parameter of signal detection theory that can be extracted from *P _{hit}* and

It is also possible to estimate the **likelihood ratio** parameter *β*, which is equal to the ratio of the likelihoods that a male is healthy to the likelihood that he is sick (see Web Topic 8.4 for derivation). It can be computed using ln (*β*) = *c d'* if the female is making optimal decisions. Using hit rates and false alarm rates, we can rewrite this as ln(*β*) = – 0.5 [*z*_{hit} – *z*_{false alarm}]^{2}.

If we know that the distributions of *w* for healthy and sick males are normally distributed with equal variances, we saw that we do not have to compute an entire ROC curve to obtain estimates of *d'*, *c*, and *β*: instead, one pair of hit and false alarm rates will do. However, distributions may not be normal or have equal variances. The only way to detect this is to plot the ROC curve by obtaining data from multiple females or by manipulating one female's prior probabilities or payoff values. We can still compute a single *d'*, *c*, and *β* from such a situation; however the analysis is more complicated than that given here. See MacMillan and Creelman (1991) for details.

Macmillan, N. A. and C.D. Creelman. 2004. *Detection Theory: A User’s Guide.* 2^{nd} Edition. New York: Cambridge University Press.

Wiley, R.H. 1994. Errors, exaggeration, and deception in animal communication. In* Behavioral Mechanisms in Evolutionary Biology,* (L.A. Real, ed.). Chicago: Chicago University Press. pp. 157–189.

© 2011 Sinauer Associates, Inc. | Privacy Policy | Cookie Policy