The Correlation Coefficient R and the reduction of range.

 Those who followed my series on climate change might remember that I would take some area I could define as a rectangle in longitude and latitude and track the average temperature by year, comparing equivalent seasons. (To be precise, if a longitude/latitude area include either the North or South Pole, it would be more like a slice of pie than a rectangle.) I then split the time span from 1955 to 2010 into four eras based on the El Niño/La Niña cycles, these in particular each starting and ending in a strong La Niña year.

This particular region and season, Siberia in Spring, has a clearly increasing trend of the median temperature, the dotted red line moving up in four separate steps. The lowest temperature registered only moves upward twice and the maximum average temperature takes a step down in the era of 1999-2010, as the highest temperature was registered back in the 1990s.

Another method would be to add a line of regression, also known as a trendline or predictor line or the line of least squares. Excel has an option which gives the equation of the line and the variance R². R is called the correlation coefficient and it varies between -1 and 1. R² must be between 0 and 1 and is sometimes thought of as a proportion. In this case, the .3655 would be the proportion we would assign to the general increase we see in the temperatures, while the fluctuations are about .6345 of the influence.

This doesn't sound very convincing, but if R² = .3655, R in this case would be +.6046, a value that by nearly every standard of correlation is considered high, though it doesn't meet the Rule of Thumb criteria for very high, which would be over .8.

This is one of the many reasons I don't love using the predictor line and the correlation coefficient. The statements of confidence seem arbitrary - I know of three different systems and they disagree radically on whether an R score is strong or not - but also there is a way to cherry pick data in both directions, either to show more correlation or less.
 

Generally though not always, taking a subset of a sample by restricting the range will result in R and R² being reduced. For example, if we look at the first Consistent Oceanic Niña Interval from 1955 to 1975, we see somewhat less overall increase (here we check the number multiplying x, which went from .0417 to .0346) and a drastic drop in R² from .3655 to .05097. Here I can say without fear of contradiction that the correlation is not impressive.

In our second interval, R² is stronger at .19237, but still well below the larger set's value of .3655. It could be considered moderately strong by some measures, but notice that here the trend shows the region cooling. (It really is coincidence that the first year of this era shows a large jump in temperature over the previous.)

Here again we see a downward sloping trendline and an extremely weak R² value of .01592, which is to say nearly no correlation.

Yet again, a small downward slope and a low R² value.

In my view, the problem is cherry picking in both directions. People who wish to downplay or deny warming temperatures can take smaller samples, but when they do, the R² value will often give little confidence in the trend they try to show. On the other hand, people wanting to show strong evidence of warming have a natural advantage of generally higher R² scores in larger data sets. To be fair, in this particular set it is impossible to create a subset longer than thirty years that doesn't show a warming trend, though it can be minimized and so can the correlation coefficient.

If anyone is coming to the blog for the first time, you should know that I am not a denier of the general warming trend in temperatures around the globe in my lifetime, which started in the Strong La Niña year of 1955. What I hope for is a discussion where both sides can agree on terms and methods and avoid cherry picking at all costs. I realize this hope may very well be in vain, and yet I hold on to it.

New data method: Consistent weather station map

I had a new idea for how to look at the big data set made available by Berkeley Earth Surface Temperature. While I now understand the effects of La Niña and El Niño are not worldwide, they do make a difference over a huge amount of the earth's surface so I will continue to consider eras that start and end with a strong La Niña years (or conversely with a strong El Niño years) to be time periods that are worth comparing.

I took the data set and wrote a C program that is interested only in the weather stations worldwide that reported a temperature for every season from 1955 to 2010, both of which are strong La Niña years.  The earliest La Niña to La Niña era is 1955 to 1975, while the most recent spans from 1999 to 2010.  Complete data for 2011 and 2012 is dribbling in, but neither is measured as strong La Niña or strong El Niño, so these are the time periods I use to avoid cherry picking the data, which will often mean the data is not completely up to date.

Tomorrow, I will start showing the results for these consistent weather stations, looking at the difference of the averages of the early and late eras. Splitting the data into the two hemispheres, the northern hemisphere is much better covered than the south, completely unsurprising given the differences in both land mass and population. There are many ways to look at a data set this big and as the week progresses we will go from the simplest ideas to the more complex.

A new (to me) climate data idea.

As I have stated earlier, the effects of the warming and cooling of vast regions of the Indian and Pacific Oceans near the equator are a very big part of climate in a large area of the world.  Any northern Californian who pays attention to weather has heard about La Niña and El Niño as the cause of dry or wet years and we are near the middle of the Norther Temperate zone.

The messy chart presented here, which can be clicked on to show a larger version, shows the effect of the ocean temperatures from 1955 to 2010 through all four seasons each year, as seasons are counted in the northern hemisphere. The thick blue line with empty squares as the markers show the Winter variance from normal, just as green dotted line (circles) shows Spring, the line red line with Xs shows Summer and the thin orange line with diamonds shows Fall.

The idea is to look at regions with this data factored out to see if the pattern gets less variable. We will test "less variability" both visually and with the R² variable from statistics, which shows how much a data set wanders from a regression line.  My first test cases will be tropical India, which we would expect will become much less variable being so close to the weather anomaly, and Greenland, which is far from the equator and not bordering the Indian or Pacific Oceans at all.

Four weeks of climate data: Arctic Circle Region #5

The final two regions of the Arctic Circle have a lot more land mass than any of the first four regions, though not quite as much population as Region #1, which encompasses polar Scandinavia and northwestern Russia. Region #5 is dominated by the Queen Elizabeth Islands, and we should expect a lot of weather stations relatively close to the North Pole.

We expected them and here they are. Each square represents a grid that get included in our regional average. The ones with thicker outlines get data from all or nearly all seasons in the time span we are considering, 1955 to 2010, years chosen because they were both years with strong La Niña currents. We split this era into four intervals based on Consistent Oceanic Niña Intervals.

First interval: 1955-1975
Second interval: 1975-1988
Third interval: 1988-1999
Fourth interval: 1999-2010

The black line at the top follows the record warm temperatures in the four intervals. It looked like a static trend or slight cooling in the last half of the 20th Century, but a big jump up in the 1999-2010, which would say warming and an increasing warming rate of warming.

The red line is the median. Again, the first three eras have a static pattern and the last time intervals shows warming and an increasing rate.

The bottom black line follows the coldest winters in each time interval. There was a warming jump between the 1975-1988 and the 1988-1999 eras, but a slight decline in the most recent. The general trend here would say warming but the rate is not increasing.
 

Every indicator for Spring says a warming trend a step up every time.  The high and low average temperature trends say an increasing rate, but the median does not.

Summer's trends are not as unanimous as Spring's in terms of warming, with little up-down-up stutter steps breaking up the median readings and the highs, but there is no compelling argument this is showing static data, it's clearly warming.  As for an increasing rate for warming, the median says yes and the other two indicators say now.

Fall data also convincingly argues for warming and the median and low trends argue strongly for an increasing rate, while the record high trend is not as clear.

Warming trend: 11 indicators say yes, 1 is uncertain. The ayes have it.

Increasing rate warming of warming: The split is much closer 7 yes to 5 no.  While there is a majority, it is not convincing, so over all seasons we cannot say the rate of warming is increasing.

Warming regions: 5-0
Increasing trend regions: 2-3

Later today, a look at last slice of the Arctic Circle, which encompasses almost of Greenland and nothing else.

Four weeks of climate data: Arctic Circle Region #4

We are now looking at the Western Hemisphere regions of the Arctic Circle for the first time. Lots of open water, very little land, all of it as far from the North Pole as you can get and stay in the Arctic Circle.

We think of Alaska as a huge ice box. It's Fort Lauderdale at Spring Break compared to most other Arctic regions.

Not surprisingly, the only grid points that have strong coverage are in the southern part of the Arctic Circle.

In the Winter months, there was a big jump in the 1970s, a static period until the turn of the century and then a small warming trend recently. I would say this is warming but definitely not getting warmer faster.

In the Spring months, the big jump is in the 1980s and 1980s, the era that had cool Winters. Go figure. I'd still argue a warming trend of sorts, definitely not an increasing warming trend.

Summer has that upward step pattern in all three measurements, record highs, record lows and median for the Consistent Oceanic Niña Intervals. Definite vote for warming, definite vote against it getting warmer at a faster rate.

The Fall data also argues for a warming region and makes an argument for the warming trend increasing. It helps win the first argument, it's definitely a warming region, but it loses the second one, the rate of increase isn't getting faster here over most the seasons.

So this region has to be counted as warming but not increasingly so.

Warming regions: 4-0
Increasing trend regions: 2-2

Tomorrow, we finish the Arctic data with the region dominated by the Queen Elizabeth Islands and the final region, encompassing the lion's share of Greenland.

There are lions in Greenland? I swear, nobody tells me anything.
 

Climate Change regional reports #1: Greenland 1955-2010

And so it begins. I am still new to collecting climate data, so my methods may change over time. Right now, the program I've written takes trends by seasons over an area defined by high and low latitudes and high and low longitudes. On a Mercator projection, this means a rectangle unless the North or South Pole is included. Taken from a pole, a Mercator region will look like a circle or a slice of a pie.

I chose not to take all of Greenland because I wanted to exclude Iceland and parts of eastern Canada. Iceland will be measure on it own in a future post.
 

 While the map looks like a rectangle, the slice will actually be a curved shape. The stations have a clump in the mid lower right, which corresponds to the middle south of the map.  Right now, I do not show that not all regions report all the years in the range. Future versions of this graphic will show how strong a station is by the number of reports it gives.

For example, the clump has a lot of stations that were only temporary. The areas that got the most reporting were in the southwest and the southeast, marked on the map in red.  The next strongest regions are the orange marks in the northeast.  If a region is well covered, it will have 100 diamonds on it. Parts of the middle of Greenland are so remote they have no nearby stations, so those grid points do not even appear.

And now to the reporting of the seasons, starting with winter. The jagged line is the year by year reports. The more blocky lines are the high temperature, median temperature and low temperature for an Oceanic Niña Interval, or ONI for short.  We will be using the 1955 to 2010 range for our work until there is another Strong La Niña or Strong El Niño. The intervals are

1955-1975
1975-1988
1988-1999
1999-2010

 And now the answers to The Two Questions when looking at Greenland Winters.

The trend is Greenland Winters are getting warmer, both the warmest and the coolest.

The most recent interval shows that warming trend in Greenland Winters is not slowing down.

And now the data for Spring.


The trend is Greenland Springs are getting warmer, both the warmest and the coolest, though the rise in the coolest is not as dramatic.

The most recent interval shows that warming trend in Greenland Springs is not slowing down.

And now the data for Summer.

The trend is Greenland Summers are getting warmer, both the warmest and the coolest, though the rise in the warmest is not as dramatic as the Winter increase and the median temperature for 1999-2010 is lower than the median for 1975-1988, the only counterexample in all our measurements for this recent ONI to count as the warmest in more than a half century.

The most recent interval shows that warming trend in Greenland Summers may be alternating. The next Strong La Niña or El Niño year may give us a better idea.

And now the data for Fall.

The trend is Greenland Falls was inconclusive in the first three ONI in our time span, but the most recent showed a dramatic increase in highest temperature, median temperature and lowest temperature.

The most recent interval shows that warming trend in Greenland Falls is not slowing down.


And this brings me to my first important points about terminology. "Global warming" is a bad phrase because not all regions show data as convincing as Greenland. "Climate change" isn't as bad, but it fails to say what the change is.

I propose three categories about regions: Warming regions, static regions and cooling regions. Greenland counts as a warming region.

Tomorrow: The Arctic Circle.

The Math behind Climate Change: Part 8 Trendlines based on Oceanic Niño Intervals (ONI)

We start again with the data for average Winter temperatures in Greenland from 1955 to 2010. The start and end years were not chosen at random. They coincide with strong La Niña years as measured by climate scientists on a system called Oceanic Niño Intervals or ONI for short. The list of strong La Niña years in this range are as follows.

1955, 1973, 1975, 1988, 1999, 2010

The two years in the 1970s are too close together to make a meaningful trend and they have no El Niño between them, since 1974 was a weaker La Niña. I'll remove 1973 from the list and we get 1955, 1975, 1988, 1999 and 2010.

We use the years to create intervals. In each interval, we mark the highest temperature in red, the average in black and the lowest in blue. These trends are easy to read and take no difficult to explain math. (Note: I am not against difficult to explain math. The math of best fitting curves comes from Gauss and is completely legitimate. My complaint against it is how many different curves can be chosen and the possibilities of cherry picking to make a point.

The three trends tell slightly different stories. All agree that the 1988 to 1999 interval saw much cooler winters than any other span and that the recent span from 1999 to 2010 is by far the warmest interval. While the average in the final interval is only slightly higher than the second warmest interval from 1975 to 1988, the high and the low both increase significantly over the second warmest in each of those categories.

Here are the statements I read most often in the papers of climate skeptics and denialists.

1. The climate is not warming.
2. The warming trend is decreasing.

For all the data I will produce, I will end with a statement addressing these questions for all four seasons in a form

Greenland 1955-2010
Winter: getting warmer, trend increasing
Spring:
Summer:
Fall:


Tomorrow, we will see the data for all the seasons from Greenland.

The Math behind Climate Change: Part 4Better data about El Niño and La Niña years

On Friday, I proposed a system for making fair intervals on which to measure climate data. Looking around the Internet, I found a webpage about the Oceanic Niño Index, also known as ONI, that looks to be more thorough and informative. Based on that info, I make a new list of fair intervals.

Here is the list of strong La Niña and El Niño years, written in blue and red respectively.

1955 1957 1965 1972 1973 1975 1982 1988 1991 1997 1999 2010

The list of fair years to start and end has been reduced significantly, so I am going to modify the rules as to what constitutes the fair years to be at the beginning and end of an interval.

1. The beginning and end of a fair interval have to be the same type of years, either both Strong El Niño or both Strong La Niña.

2. There has to be one year between the start and end of a fair interval that is of the opposite type from the type used for the beginning and end year.

Example #1: 1973 and 1975 are both Strong La Niña years, but there was no Strong El Niño between them, so 1973-1975 would not count as a fair interval. The earliest year that would make a fair interval starting in 1973 would be 1988.

Example #2: 1957, 1965 and 1972 are three Strong El Niño years without an intervening Strong La Niña year to break them up. That means 1957-1965, 1957-1972 and 1965-1972 would not count as fair intervals.

Here is a list of what I call the Consistent ONI.

First, the usable La Niña intervals.

Starting in 1955:
1955-1973 1955-1975 1955-1988 1955-1999 1955-2010

Starting in 1973:
1973-1988 1973-1999 1973-2010

Starting in 1975:
1975-1988 1975-1999 1975-2010

Starting in 1988:
1988-1999 1988-2010

And the fair El Niño Intervals.

Starting in 1957
1957-1982 1957-1991 1957-1997

Starting in 1965
1965-1982 1965-1991 1965-1997

Starting in 1972
1972-1982 1972-1991 1972-1997


Starting in 1982
1982-1991 1982-1997

As you can see, if we want to talk about the 21st Century, the shortest Consistent ONI is 1988 to 2010. When the next strong El Niño year is confirmed, it will make 1999 to 20xx a Consistent ONI.

As the proposer of the system, I will admit this is a weakness. Human nature wants to know what is happening now. An option would be to make Moderately Consistent ONI and Weakly Consistent ONI. For example, 2011 was a weak La Niña year and the previous weak La Niña years were 2005 and 2000. Because the weak years are more plentiful, it would make sense not to go back just one previous weak year but two, so 2000-2011 could be called a Weakly Consistent ONI.


I'm not a climate scientist, just a mathematician. I have no standing in the community to make the Consistent ONI the industry standard. El Niño and La Niña patterns have an effect over a vast region in the Indian and Pacific Oceans and the land masses that border them. I will do some research to see if the Atlantic has a similar known warming-cooling trend and if they exist, I will propose fair intervals based on them, which would be of use for regions bordering the Atlantic, including the eastern parts of North and South America, the west coast of Africa and all of Europe.

Tomorrow, we will look at a typical region and the distribution of stations.