by Ralf Neumann, Labtimes 02/2017
Last evening started with a shock. Still drowsy from a long and restful day’s sleep, I had just started gliding around in my forest to search for some breakfast prey, when suddenly I spotted three of your fellow humans. Of course, it wasn’t a problem that they were there – they would never catch a glimpse of me, not even of a single tail feather, if I so wished. No, no, it was rather the shock that struck me, when I realised what they were doing...
To cut it short: the threesome had caught several dozen of my smaller bird neighbours. My neck feathers quivered, when I actually recognised some of my close friends! Aghast, I felt compelled to watch what they were doing in the light of their lamps...
Quite quickly, I noticed that those three guys were some kind of researchers. I had already heard from some of my fellows about this kind of human “bird research activity” but never seen it with my own eyes – until yesternight! And what I was told was obviously true – for every single birdie they measured just everything conceivable: weight, length, width and height of whole bodies, heads, beaks, claws, wings, single feathers, yada-yada-yada...
But what did they hope to elucidate through all this torturous manhandling? Okay, I admit they can do some statistical analyses with the data, e.g., concerning nutritional states of populations or means and variations of certain morphological traits. Provided they collect enough data and do it properly! (Please forgive me but this, in particular, does not exactly seem to be a given for your human sciences these days. Even I have heard about the recently highlighted, major problems with the statistical soundness of many of your studies. And that, therefore, your whole science world has suddenly found itself in a deep “reproducibility crisis”.)
Finally, however, I left the disturbing “bird research” scene, in the sincere hope that my dear fellows had, at least, suffered for the sake of something really meaningful.
I have to admit, though, I was not very optimistic in this respect. Somehow, I doubted that the three were actually clear about the boundaries of what they can infer from their data – and what not. For example, I heard one of them declare, “Look at those long tail feathers. For sure, they are the most elegant fliers in the whole forest.” Argh! An evaluation of feather lengths alone will never reliably identify the most elegant, enduring or whatsoever fliers. Just as a mere evaluation of beak and claw sizes will never reveal, which is the most successful bird of prey. You just cannot infer complex qualities from some simple numbers alone.
However – and now, pay attention, here comes a rather heretical question – how should this become clear to scientists, who themselves are regularly evaluated by solely applying some simplistic “indicator numbers”?… Okay, okay – I admit, in bird science we also once experienced such a terrible outbreak of “Evaluitus infection”, mostly caused by that damned superbug strain Evaluitus impactfactoriensis. But in contrast to you humans, we birds managed to successfully and quite quickly eradicate this nasty curse from our science system.
Nevertheless, I still remember one very revealing anecdote from this period with blatant amusement. Back then, I was already a long and well-respected senior researcher – and comfortably thought that I no longer actually had to prove anything to anyone. One day, however, I was informed out of the blue that a consortium of avian science funders had set up a permanent “evaluation office” – and that one of their “experts” would be round to visit me very soon to evaluate my research. Hmm...?
Only a couple of weeks later, I found a young and dynamic Snipe sat next to me on my branch – the “evaluation expert”.
“My dear Professor Owl,” he came straight to the point, “I’ve looked up the number of times your work has been cited during the last five years. And I regret to inform you that your numbers are below average, when compared to the number of citations that biochemical papers collect in the world-wide means.”
“You’ve wasted your time, my young fellow,” I winked at him, cheekily. “I could have told you that. Indeed, I’m not quoted very often – simply because only a few birds work in my field.”
“In plain language, that means your work has little impact on any other research,” replied the young Snipe.
My feathers puffed up in the face of such a flat phrase (Snipe by name and snipe by nature, I indignantly thought) and started my counter-attack, “Are you sometimes plagued by feather mites?”
“Yes, why?” came the astonished answer.
“And what do you do against them?”
“Usually, I rub my feathers with that new medical lotion from Waxo Smooth Klone – as almost everybirdy does.” The evaluator was visibly somewhat unsettled.
“Good, and do you know what’s in it?” I continued the “interrogation”.
“I’ll tell you: a mixture of miticidal compounds from a moth named Euxoa auxiliaris.”
“I’m sorry, Professor Owl, but that’s definitely ...”
“... Not relevant? One moment! Not so long ago, I identified the miticidal effect of the moth’s body fat. Scientifically, no big deal. But then Waxo got the point, ‘optimised’ the moth fat and finally started the production of their new blockbuster lotion. The only stupid thing is that the industry does not publish any results – and therefore, of course, has never cited my work. And nobody else has neither.”
Young Snipe just stared at me.
“But nevertheless, my work has had quite some impact in the last few years, right?” I smiled smugly from ear to ear.
So much for what evaluations by pure “indicator numbers” can tell – or cannot tell.
Last Changed: 26.04.2017