Your friend, the question mark
Why, and how, to honestly record uncertainty in your wild counts.
written May 21, 2018 (last updated Oct 10, 2021) • by Jon Sullivan • Category: Wild Counting
Guess what? No, don’t! When you’re counting the wild, you should never, ever, EVER guess. Guesses + data = data you can’t trust. Data you can’t trust are pretty much worthless.
It’s very important, when counting the wild, to keep your observations separate from your interpretation. Honestly recorded pure observations are data that can be accurately compared across observers. They can also be compared within one observer’s observations across many years. Recording your interpretations as your counts pollutes your data and riddles these comparisons with doubt.
I don’t trust wild counts that don’t contain uncertainty. It means that uncertainty in what was seen or heard has been replaced with the observer’s interpretations of what they thought they likely saw or heard. That applies just as much to counts you made ten years ago as counts made by other people. There’s no easy way to later untangle reality from mistakes made in interpretations. They need to be kept separate right from the start.
In wild counts that are free of interpretation, two equally diligent observers with differing identification skills should record completely consistent data, if they independently count the same moment in the wild. The more skilled observer’s observations might be more precise than the others’, but they should be equally accurate. That doesn’t happen when observations are polluted with observers’ interpretations of what they probably saw.
If you’re counting species, and you’re not sure what something is, the only correct thing to do is write down that you’re not sure. Don’t skip it and don’t guess. If it was probably a blackbird, you’ve got to write that it was probably a blackbird. You don’t write that it was a blackbird.
This is all particularly important for long term monitoring. Your expertise as an observer will likely improve over time, and your ability may decline as your eye sight and hearing gradually gets worse. Yet, you’ll need every detected wild individual to be recorded accurately, so you can reveal changes in abundance over time.
There’s one easy solution to this: when you’re not sure what something is, you’ve got to honestly record that you’re not sure.
How to do that? Use the mighty question mark!
Unsure? Question mark!
Let’s start with an example. If I’m not 100% sure something is a blackbird, then it gets entered with a question mark.
Even if I’m 99% sure it was a blackbird, it still gets a question mark, always. The WildCounts principle is “Never guess and never interpret”, not “Sometimes guess and sometimes interpret”.
If I’m 99% sure that it’s a blackbird, and I state with certainty that it’s a blackbird, then that’s my interpretation of what I observed. It’s not what I observed. Interpretation happens later in the data analysis.
My job when making wild counts is to honestly record exactly what I observe, nothing less and nothing more.
By adding a question mark in this way, when we’re analysing blackbird trends, researchers can look at the trends both including and not including my questionable birds. If we get the same result both times, we can be confident that a trend is robust.
But, wait, you might say, why are you even bothering to count birds if you can’t identify a blackbird? True, they are distinctive birds, but that’s not always the case. If you’re counting every individual of a species, no matter how easy it is to identify, sometimes all you’ll get is a brief glimpse.
For example, that happens when I’m moving along a transect and I glimpse the brown back of what looks like a female blackbird, foraging in the shade under some vegetation. In Christchurch, New Zealand, that’s going to be a blackbird, or perhaps a song thrush. Thrushes are lighter brown on the back (and very speckled on the breast) but that’s not always clear from a view of the back in the shade. If I had time and binoculars, I could work it out. If I don’t have time, or if it flies off, I can write the following and keep moving.
Blkb 1? (or thrush)
(I just say
thrush, and not
song thrush, here because in New Zealand that’s the only species of thrush we have.)
Usually, I can see enough to enter a count as the species I think it is, just with a question mark. Other times, though, I may be less sure, such as when a small brownish bird flies quickly overhead. It can be too quick to see clearly, catch in binoculars, or photograph. At least it was definitely a bird. Like other birders, I use a generic code for these small birds. Mine is
sbb. One small mostly brown passerine bird, flying over my path at my mid-distance, 20—80 m away, would be recorded as the following.
The important thing for monitoring is that I didn’t ignore what I saw but instead recorded it honestly. During analyses, researchers can then ensure that the trends they uncover are robust to substituting my
sbb with various plausible small birds previously seen in this area.
Ensure that the Future You can trust the Now You 100% when you’re counting the wild. Embrace the question mark!
You’re really unsure!? Double question mark??
There’s also a more drastic scenario that you’ll eventually encounter when you’re counting every individual of your predetermined set of species. That’s when you’re not sure whether what you saw was a living thing at all. This is most likely to happen when making wild counts from a fast moving vehicle. Sometimes I glimpse things whooshing by that I’m really unsure about. That happens especially when I’m counting roadkill, which can be misshapen and broken and often difficult to identify when on the move.
Do I record something only when I’m sure it is, or was, alive? No, I can’t do that. That’s interpretation. What if I get better over time at recognising and identifying roadkill? That would create a false trend of increasing roadkill rates in my data.
Others could also be better, or worse, at recognising roadkill than I am. If we only recorded the roadkill that we confidently recognised, we wouldn’t be able to reliably compare our datasets.
The rule remains: our interpretations must be kept separate from our observations.
It’s in these cases that I use the WildCounts double question mark (
??). This, again, encourages me to honestly record what I see without requiring me to record my interpretation of what I saw.
For example, I’m in a car and see something black and flat whooshing by me on the left road verge. It might have been a squashed roadkill blackbird, but could also plausibly have been a blackbird sized piece of cloth or other rubbish. I can record this as follows.
blkb?? lv unsure sq
It’s on the left verge (
lv), I’m unsure whether the roadkill is less than 24 hours old (
unsure) and it was squashed (
sq). Note the double question mark here in
blkb??. That’s quite different from
blkb?, which would have meant a blackbird or similar looking bird, but definitely a roadkill. In other words,
? means that there was a roadkill but I’m unsure about the species. In contrast,
?? means that there might have been a roadkill.
Embrace the uncertainty
This might all sound a bit silly and pedantic. If you’re experienced enough to know that it was probably a blackbird, why not just write down “blackbird”? Does it really matter whether or not you record when you’re uncertain about your identifications?
In some cases, it might not matter much. Real population trends, if strong and consistent, will emerge from your data regardless. Any extra noise caused by varying, inaccurate interpretations will only mean that counts need to be collected for a little longer to reveal reliable population trends.
However, in other cases, it definitely will matter. Fake trends can emerge caused by gradual changes in observers’ skills. Real trends can be obscured by the noise of variability in the accuracy of observers’ interpretations.
Importantly, you won’t be able to tell whether or not your trends are real or fake. That is, unless your observations and interpretations are always kept separate.
It’s also just easier to honestly record exactly what you observe. You don’t need to get distracted considering what your best estimate is of what the species was that you just saw, or heard. Just write down that you’re unsure, with
??, and keep on counting the wild.