Please don’t run away! This stuff is not rocket science– heck, it’s not even neuroscience. It’s just learning how to look critically at graphs.

What we have in Fig. 1 is a heavily edited graph– I erased a bunch of labels and lines (using Gimp) so that all our eyes see are the data points and their spread across the graphed space.

The X-axis is an Index which describes how our unit of analysis scored, from 0 to 100, on some measure. Think test scores. Let’s make the assumption that the Index is on an integral scale. That means that the quantitative distance between, say, 20 and 21 is the same as between 65 and 66. The distance (of whatever the Index is measuring) between 80 and 100 is 20 times the distance between 20 and 21. Think inches, minutes, years, ounces, positive and negative integers.

The Y-axis represents some measurable change over time, presented as a percent, for all Xs.

Presented in this way, we ask, does the Change appear to be related to the Index? Looking at the entire data set, the only correct answer is “yes & no.”

Look for example at the Index score of ~50. There are two points. One is (50, +15) with +15 being the greatest percent change among all 13 points. The other is (49, -11) (approx.) with -11 about the fourth lowest percent change. So no.

But my eyes are hard pressed to tell me there isn’t some sort of left-to-right downward trend. Just a back of the envelope ignorant don’t know what I’m talking about visual examination confirms this.

Look at the horizontal line that is 0 on the Y-axis. If all of the 13 points more or less clustered uniformly around zero, no matter what the Index score, we’d say Index was not related to change. Again, no.

There are five points that are clustering around 0. “Uniformly clustered” in this sense means that even if the clustering isn’t tight (as in the five points), we’d expect just about as many above as below the 0 line.

13 – 5 = 8, so four above and four below. But here there are two above, and six below. What my eyes are telling me is correct, there is some sort of relationship between Index and Change. Yes.

[Note. This is not the formula for fitting a line. It is a way to ask if the line makes sense.]

Let’s consider the term “relationship” with respect to Xs and Ys. Prior to the invention of the modern computer, God had revealed to mankind, with the assistance of pencil and paper, a few thousand relationships. Since that time, mankind with his computers has on its own, and seldom with any help from God, invented orders of magnitude more. (Many of these we call “models.”) Fortunately, we need be concerned only with two of the oldest and truest relationships, the positive and negative linear relationship.

Let’s also re-consider the Index. Recall, Index describes how our unit of analysis scored, from 0 to 100, on some measure. And we’re still assuming that Index is on an integral scale (through pretending is a better word). “Index” very often indicates that what was measured was not a single thing. If we’re only concerned with test scores, we’d label the X-axis “Test Score.” Index scores are usually a compilation of multiple things– multiple measurements. And more often than not, those individual measurements are themselves taken to be relational.

Body Mass Index is body mass (k) divided by the square of body height (m). The Democracy Index is based on weighted answers to 60 questions. The Academic Index is is a compilation of class standing, SAT score and SAT subject scores all converted to a scale ranging from 20-80 and then added together. Again, very seldom is an index simply the adding together of measurements of multiple factors, but a manipulation of measurements. The reason to manipulate– give more or less weight to the various factors– is often straightforward. In the case of BMI, it’s to get a simple statistic that folks can understand and isn’t too unwieldy. The range of BMIs is expansive if you just divide mass by height. So height is squared to make a bigger divisor, thus generating numbers across a smaller, more comparative, range.

Do the math 5.5′ / 110 pounds. 6′ / 200 pounds. Versus squaring pounds converted to k, divided by the square of height in m. Makes sense.

Sometimes, as is probably the case in the Democracy Index, some factors are deemed to be more important than others. Free elections is one such. Yes. It is good to be skeptical of who is deciding what is more important.

With that all in mind, grab a straight edge and take a look. Remember, a linear relationship doesn’t mean all of the points have to be on a line, rather they cluster around it.

What do you see?

My eyes see two, possibly three, distinct lines. (There’s a caveat coming.)

One line begins around (18, 0) and goes through (60, -16) and could also include (83, -29). Another, with a steeper slope, begins at (30, +15) and goes through (75, -12). It too could include (83, -29). The potential third– three points!– has the same slope as the first and is that small cluster.

Keep in mind that what we are doing is exploring. We are exploring because we want to be better citizens. We want to be better critics.

So for the sake of argument, let’s say we think there are three separate groups of data here, and the relationship between Change and Index differs among them. Why might that be?

First, something may have been left out. A factor(s) that matters to how Y relates to X may have been omitted in the Index. That factor may be known to exist, but either intensionally or accidentally ignored. It could be unknown, in which case looking at the data one would ask, “What am I missing?”

It could also be the case that the way the individual factors are weighed in the Index varies. All of the factors are taken into consideration, but it’s a mistaken assumption to think their relevance to the overall Index is uniform across all Xs.

Food deserts– which were based on an index of food availability– were a classic example of this. A lot of what into the food desert index assumed distance walking to a grocery mattered to country mice as much as it did to city mice. It does not.

A third issue may be that a complex factor is conflated into a simple one. BMI is a good example. A short, fit, very muscular woman is “obese” because muscle weighs more than fat. Body mass is complex.

Ta. Da.

I have no doubt that the trend line is indeed what the stats package the folks at the Blavatnik School of Government at Oxford University are using is a line of best fit. None whatsoever. I am, however, a poor little country mouse who thinks sometimes we ought to see with our eyes before we rush into find a line.*

I am ashamed to say that sometime after May 22 I closed the tab from which I screen shotted this, and subsequently cleared my history. So while the graphic does come from BSG, I cannot find the article in which this appeared– the article I was taking issue with initially. My bad.

But every point I tried to make above stands. Why would Ireland and Germany, with virtually identical Stringency Indexes have so vastly different economic responses?

Which, by the way, totally contradicts the subtitle of the graph.

Is there a factor(s) that comprises the Index that systematically gets more stringent as we look at the policies of Sweden, Portugal, Germany, France, and possibly Italy? And the same for Ireland, Greece, Hungary, Spain?

Why are the policies of those countries hovering around zero change in economic production, which include the least, and almost most stringent, having no affect? Note, too, how the data points of Netherlands, Finland, Norway, and Denmark absolutely sort of cluster at a sweet point. How do they differ from less stringent Sweden?

My take away is this. There is not more than meets the eye going on in this graphic. There is more than meets the computer model generating a line of best fit.

Use your eyes.

*John W. Tukey