On my daily ride through Arlington National Cemetery the other day there was a big truck parked in the middle of a row of gravestones, with a crane on which were workers taking down a tree. I looked around at all the trees in the cemetery -- big, beautiful trees, many of them, and as I coasted by Robert E. Lee's old house, I wondered how many of them were standing when he lived there.
The tree the workers were taking down had clearly been given a death sentence by some arborist. I pictured the Arlington Cemetery staff arborist and how he must keep track of all those trees. That made me wonder how they know when a trees should be taken down. And, how clear is it for an arborist when a tree needs to be taken down? If you got 100 arborists and had them look at all the trees in Arlington Cemetery, how consistent would their recommendations be?
Which raises the question, what about other professional fields? Of 100 economists, how many would prescribe the same course of action when faced with a situation? How many podiatrists would diagnose the same problem and prescribe the same solution.
It would be nice if there was a way to create a standardized measurement for this across fields -- an "intra-professional consistency" metric, if you will. Of 100 specialists, how many
reach the same conclusion? And what is the standard deviation of their
responses? Which fields are the most unified, which the most fragmented?
Clearly, there would be a scale. How many poets would describe love the same way? Probably none. On the other hand, how many physicists would calculate the mass of an object the same way? Probably all of them.
In addition, intra-professional consistency varies not by profession, but by question. 100 physicists will agree on the mass of an object -- but not on the nature of a quark. Here we spin off into the land of Thomas Kuhn and the complex questions of the philosophy of science.
That means that it will never really be possible to do this in any kind of genuinely standardized
way across professions -- a standardized way of testing could be created *within each
field* but not one that permits genuine rigorous comparison between,
for example, astronomers and radiologists.
Really, there are two core questions. First, what are the questions on which a profession agrees. In physics, the basic questions of mass and motion and newtonian gravity are basically figured out in that the entire field agrees upon certain answers. But there is much disagreement over the nature of energy. Each field has its areas of agreement and its unknowns where the experts disagree.
The second question is whether the answers on which a profession does agree are actually correct. It may be that 100 out of 100
podiatrists would all diagnose a runner with the same problem and
prescribe the same arch supports -- and yet all of them could be under
the sway of a false historical-cultural understanding of feet and be
misguided in their prescription. Similar errors can be made by any profession including physics. At the same time, 100 poets
describing love are certain to overlap very little in their "solutions"
to the problem -- and yet a high number of them are likely to express
truth of various kinds. So, professional consistency does not necessarily correlate with
the truth or usefulness of a profession's beliefs. As I written about, there have always been groups of experts who are completely wrong.
That
said, it is a useful analytic concept -- a concept that may not be found in nature but which is nonetheless useful for
thinking about things. I *would* be curious about the degree of consistency of recommendation of various experts on various matters. It's no guarantee of correctness, but a lack of consistency sure tells you something (what philosophers might call a lack of intersubjectivity).
Economics
of course is very low on the consistency meter -- nothing wrong with
that, that's to be expected in the humanities; history is the same way. The only problem is that unlike with history and poetry and the like, too many people fail to perceive
that economics is, in fact, one of the humanities (a particular, warped form of history, I would argue; others argue it's more like literature). Instead our culture treats economics as if it were a science. And that illusion of objectivity has very real damaging political and social effects.
When there are only four cities, it’s easy to solve. But once you start to expand the number of cities, it quickly gets very difficult – with only 25 cities, there are so many possible routes that a computer measuring one million per second would still take 9.8 billion years to search through them all. (I get this from Peter Coveney and Roger Highfield,