Some thoughts about election prediction.

It's been nearly two weeks since there has been any new data on the New York City mayoral race. The last few polls agree the three at the front of the pack are Christine Quinn, Bill De Blasio and Bill Thompson. Unless one of them gets over 40% of the vote, and the recent polls make that look very unlikely, the top two will go into a runoff election, and the odds make it look like an uphill climb for Thompson.

The New York Times endorsed Quinn. Her bio is certainly compelling from a progressive standpoint. She would be the first female mayor of America's largest city and she would be the first openly gay mayor. Her record on the issues is not as compelling from a progressive standpoint. She is seen by many as a person who would continue the Bloomberg status quo. The Nation, a truly progressive periodical unlike the Times, has endorsed De Blasio.

Nate Silver has published numbers saying that history favors the long term front runner of the year, which would be Quinn in this case. She had competition from Anthony Weiner after he announced and before his spectacular crash, but for months before that she lead in every poll.

This is one of the many places in predictions where I part company with Silver. I assume that every election stands on its own and historical data should not outweigh current polling.

It's impossible to predict the big events and impossible to gauge the weight they will have. It looked like based on name recognition Anthony Weiner was a serious candidate, but the Carlos Danger situation cut his support in half, his position plummeting from first to a distant fourth. A judge deciding "stop and frisk" was unconstitutional gave De Blasio a big boost, but it's hard to say if it's a surge or a bounce until we get more data. The Times endorsement should be much more important that The Nation's, but are either of them really events that make a difference in the polls?

I have no idea.

When I say that the last polls look like an overwhelming likelihood of a Quinn/De Blasio runoff, I am completely willing to say something else if the next polls change. It's very hard to say what events will change the polling numbers, known in journalistic circles as "moving the needle".

In 2012, Romney's 47% comments moved the needle against him, but he was already in a horrible position when he said that. Obama's awful first debate definitely moved the needle in Romney's favor, but by the time of the Biden/Ryan and the second Obama/Romney debate, the movement towards Romney had stopped and the tide reversed. He never was favored to win.

Conservatives hoped Benghazi would move the needle. It didn't.

Silver went to Princeton and I went to Cal State Hayward. Silver was hired by the Times and I haven't been able to get anyone to pay me for my work. In my favor, I did better than he did in 2008 and 2012. (He missed Indiana in the electoral vote in 2008 and two Senate races in 2012 that I got right. One of them he missed by giving weight to historical data, something I would never do.)

Here's the big difference between us as far as I'm concerned. My education is in pure mathematics and his is in applied.

Election prediction is applied math. I deeply distrust it and I wait until I think the data really means something. I have arbitrarily chosen about a week before the polls close. By then, I figure enough mail-in ballots have been cast that the most important proviso of any opinion poll is actually true, that "the election is being held when the poll was taken".

Is seven days the right number? Maybe ten would be better, maybe four. If my predictions become less reliable in the future, changing this time period is one of my first places to look to improve my system.

This brings me to something else Silver has done that I disagree with vehemently. He is predicting the make-up of the Senate in 2014 and giving a margin of error. These numbers are being used as scare tactics by the Democrats begging for money.

As far as I'm concerned, these numbers are worse than nonsense. He does actual work, so I am not happy comparing him to lazy, ignorant anti-intellectuals like Dick Morris, Karl Rove, Jim Cramer or George F. Will, but predictions about a multi-part election more than fifteen months away from polls closing are mere balloon juice. Even if someone "crunches numbers", given how many unknowable events there will be in the next year and a quarter means his work has no more meaning than doing exact bio-rhythms or a complete astrological chart.

I do not know if Nate Silver will ever read what I write. If he does, I want to say to him, older nerd to younger nerd, stop doing the stuff that is most likely to hurt your reputation. A fifteen month prediction about elections is no better than your work in fantasy football or Oscar prediction, which you have been gracious enough to admit has been less than optimal.

Wait until just before the election. Then the numbers of people doing actual work, work like yours and mine and Sam Wang's, have meaning. Right now, your work in 2013 is like Gene Simmons of KISS going out on a limb in 2011 predicting President Rick Perry and giving us as the reason to believe him, the quote "I'm never wrong."

He was wrong. Even with your margin of error, saying there is 95% confidence in your numbers is plain silly.

Stop being silly. You are much better than that.

Here endeth the lesson.

The Witch of (Maria) Agnesi

 Consider a circle that fits perfectly between two parallel lines. One of the easiest pairs of parallel lines to picture is a top and a bottom. In this picture, the point O is the point tangent to the bottom and the point M is the point tangent to the top.

Take any point on the circle and call it A. The line drawn from O to A is called a secant and it will cross the line at the top at a point we call N. We can create a right triangle by making a line parallel to both top and bottom. The point P is the corner of the right triangle that has the 90° angle. For every new A, there is a new N and a new P. We are interested in the curve created by all the points P.
  

This may seem like a very strange and roundabout way to make a curve in the plane. You wouldn't be wrong to think so. In the early days of the xy-plane, mathematicians worked on these kinds of very strange curves. The first work on this was done by Pierre De Fermat, an amateur mathematician working with Rene Decartes at the beginning of the discovery of the Cartesian plane, named for Descartes ("of the cards" in French.) The formula for the curve is given by the formula in x, y and a, where a is the radius of the circle.

The simplest version of the equation is when the radius is ½, so the diameter is 1.

The curves are sometimes called bell-shaped curves, but this is not the formula for the most famous of the bell-shaped curves, the most studied version of that being called the normal curve. This curve is called the Witch of Maria Agnesi.

The name comes about by a mistranslation of Italian to English, a mistranslation that might have been intentional but has most certainly lasted until today.

Fermat's first work on the curve is done is 1630, but in 1703, the Italian Guido Grandi also studies the shape and names it in Latin, the versoria, which is also the name for one of the ropes that holds a sail in place, usually called a sheet in English.

Move forward again to 1748 and a very odd occurrence at the time, an important mathematical paper written by a woman, the Italian mathematician Maria Agnesi. She writes her paper in Italian and calls the curve "versiera", the Italian translation for Grandi's Latin word "versoria". Here comes the confusion, possibly done on purpose.

The Italian for adversary is "aversiera" is sometimes shortened to "versiera". The adversary of God is the Devil, but because this is the feminine form, it can be translated to "witch". The translation by Cambridge professor John Colson, a contemporary of Maria Agnesi, turns the name for a rope on a boat to witch, a play on words that likely was done at the expense of the very rare female mathematician.

So here is to Maria Agnesi, not an originator of an idea but, like so many of us in mathematics, a worker in the field, doing her best to preserve knowledge for future generations. Her name lasts to this day in a somewhat mocking form, but at least we remember her, one of the few women in all of Europe to publish a mathematical work that has lasted from an era when women were not allowed to get a degree from any university on the continent.

One more rule about Pythagorean triples

Yet again, let's consider Pythagorean triples, sets of three positive integers a, b and c such that a² + b² = c².  Here are several examples where all three numbers are less than 100.

3² + 4² = 5²
12² + 5² = 13²
15² + 8² = 17²
7² + 24² = 25²
9² + 40² = 41²
20² + 21² = 29²
48² + 55² = 73²

We know that we can generate Pythagorean triples using any two positive integers j and k with the formulas, where j > k with

a = j² - k²    b = 2jk and   c = j² + k²

This means there must be infinitely many different Pythagorean triples. If you aren't sure, consider k=1 and we get

a = j² - 1   b = 2j and  c = j² + 1

For any j > 1, we will get a unique Pythagorean triple.

Here's a pattern I noticed. Every triple on our list above has one number that is divisible by 5. Let us start with a speculation.

Speculation: Every Pythagorean triple has at least one number that is divisible by 5.

We could create a triple where all three numbers are divisible by 5 just by taking any Pythagorean triple and multiplying all the numbers by 5.

Example: Because  3² + 4² = 5², 15² + 20² = 25², The original numbers are 9 +16 = 25, and if we multiply all the numbers by 5, all the squares are multiplied by 25 and we get 225 + 400 = 625.

That explains why the modifier "at least" is used in the speculation.

Let's find facts that will help us prove the speculation.

Fact: The last digit of a square is determined by the last digit of the original number.

Think about multiplying two numbers that have two digits together, like 72 × 72. 

__72
_×72
_144
504 
5184

The ones place is just 2 × 2 and the 70 isn't involved. This would be true no matter how many digits in a number.

Fact: The last digit of a perfect square can only be 0, 1, 4, 9 or 6.

We can prove this by looking at the list of perfect squares of the numbers 0 through 9 (0, 1, 4, 16, 25, 36, 49, 64, 81) and using the first fact from above that the last digit of a number determines the last digit of the square.

Fact: The remainder of a perfect square when divided by 5 can only be 0, 1 or 4.

Because 10 = 2 × 5, the remainder upon division by 5 will look a lot like the last digit, except 6 becomes 1, and 9 becomes 4.

Fact: If you want to make a sum of two numbers from the set {0, 1, 4} to add up to 0, 1 or 4, there are only a finite number of patterns and not all of them work.

0 + 0 = 0 works
0 + 1 = 1 works
0 + 4 = 4 works
1 + 0 = 1 works
1 + 1 = 2 doesn't work
1 + 4 = 5 doesn't work, except when we divide 5 by 5 we get remainder 0, so it does work for us.
4 + 0 = 4 works
4 + 1 = 5  doesn't work, except when we divide 5 by 5 we get remainder 0, so it does work for us.
4 + 4 = 8 doesn't

Fact: Every pattern that works has a 0 or 5 in it.

Fact: A remainder of 0 when dividing by five means the number is divisible by 5.

Fact: Because 5 is prime, getting a perfect square that is divisible by 5 means the original number is divisible by 5.

This means our speculation about the Pythagorean triples is in fact a pattern that is always true. We could call it a "theorem", but I'm going to save that word for

Fact: Every Pythagorean triple has at least one number that is divisible by 5.

Here are two other facts that can be proved using similar methods, which the reader can try to prove if interested.

Fact: Every Pythagorean triple has at least one number that is divisible by 3.

Fact: Every Pythagorean triple has at least one number that is divisible by 2 and one number  divisible by 4.

We don't have any other primes for which this is true. That is proved simply by the triple 3, 4, 5. All the primes greater than 5 (7, 11, 13, 19, ...) do not divide any of these numbers evenly, so we have a single counter-example that works for an infinite number of cases, a situation a lazy mathematician is always happy to find.

Another New York City mayoral poll.

A new poll for the New York City races was released Thursday night, this one from Marist, yet another small East Coast college that does polling.(The earlier polls were from Qunnipiac and Siena, also small East Coast colleges.)

The latest poll, completed on August 14 sampling 679 Democrats, puts City Council Speaker Christine Quinn and public advocate Bill DiBlasio in a tie at 24% and former comptroller William Thompson in third with 16%.

The rules of the race are that if any single candidate polls over 40%, he or she is declared the winner and goes onto the general election. If not, the top two vote getters are in a run-off.

If the election were held when the poll was taken, one candidate getting over the 40% mark is highly unlikely, so it's really a race for the top two spots. A week ago, that looked like Quinn and Thompson, but the two polls this week say Quinn and DiBlasio. The event that is considered the turning point is the decision by a judge that "stop and frisk" policies are unconstitutional and DiBlasio's long held objection to the policy in New York City is his key advantage.

Marist also polled the Republican race, where Joe Lhota has a large lead but not polling over 40% yet. He could win without need for a run-off depending on how the undecided vote breaks, but he is not given a very good chance against whoever survives the Democratic race.

There will certainly be more polls between now and the election and I will give updates when available.

Another rule about Pythagorean triples.

Last January, I wrote several posts about Pythagorean Triples, a set of three whole numbers a, b and c that follows the rule of the Pythagorean Theorem a² + b² = c². Here are some examples.

3² +4² = 5²
8² + 15² = 17²
24² + 7² = 25²

If we think about these numbers as sides of a triangle, the area will be ½ab. In the cases from above, the areas as 6, 60 and 84, respectively.

There is a way to generate all such whole number triples, using two positive integers j and k, with j>k.

a = j² - k²
b = 2jk
c = j² + k²

Our first rule is that we should stipulate that j does not equal k, because in the that case a = 0. Re-writing a and b in the area formula into multiples of j and k, we get this.

Area = ½ab = ½(j² - k²)2jk = (j² - k²)jk

Since j and k are whole numbers, the area must be a whole number. But more than that the number must be divisible by 6. The proof is split into two parts, first that the area is divisible by 2 and secondly that the area is divisible by 3.

Proof of the area is divisible by 2. If j or k is even, then (j² - k²)jk is even, since if you have a product of whole numbers and one is even, the product is even. The only other option is that both j and k is odd, and if that is the case then j² - k² = odd - odd = even.

Proof of the area is divisible by 3. If j or k is a multiple of 3, then (j² - k²)jk is a multiple of 3. The only other option is that neither j and k is a mulitple of 3. For any number n that isn't a multiple of 3,  n² will be of the form 3p + 1, which is to say one more than some multiple of 3. If that is the case then j² - k² = (3p + 1) - (3q + 1) = 3(p- q), which is a multiple of 3 as well.

More election news:Results from New Jersey, new polls for New York City mayor

The results from the primary for the New Jersey special election came in and there were no surprises. Cory Booker won the Democratic nomination handily with 59% while his closest rival had 20%. Republican Steve Lonegan beat his single opponent Alieta Eck 79% to 21%.

The Booker vs. Lonegan election will be held on Wednesday, Oct. 16. Two months is a long time in politics, but currently Booker is leading comfortably in all the polls that have asked about this particular match-up.
  

The New York City mayoral race is much more up in the air. The Democratic primary polls taken this month by Siena and Quinnipiac don't agree on much except the fall of Anthony Weiner from the top tier of candidates.

The rules of the primary state that if a single candidate gets more than 40% of the vote, he or she is declared the winner. Otherwise the top two candidates are in a run-off. Here are the results for the top three candidates in the most recent polls.

Siena (8/2 to 8/7)
Quinn 25%
DiBlasio 14%
Thompson 16%

Quinnipiac (8/7 to 8/12)
DiBlasio 30%
Quinn 24%
Thompson 22%

My Confidence of Victory method does have ways of turning each of these sets of numbers into probabilities of victory for all three of these candidates, and as you might expect the Siena numbers from earlier in the month would make Quinn the prohibitive favorite while the Quinnipiac number would hand the big favorite role to DiBlasio.

Quinn has been the favorite for several months, never polling less than second. Similar to the 2012 Republican presidential primary, there is a remarkable amount of what I call "churn" in the numbers. If I stretched the analogy a little farther (perhaps too far, I can't be sure), this would put Christine Quinn, the openly gay speaker of the New York City Council in the role of Romney, while Weiner and DiBlasio, the  New York City public advocate and considered the most liberal candidate, in the roles of the many Not Romneys that the Republican field had to offer.

The last five elections for mayor have been won by a Republican (Giuliani) and a former Republican turned Independent (Bloomberg), but the current assumption is that the star power is gone from the Republican ranks currently and the Democratic primary winner will move to Gracie Mansion.

I've read analyses by Harry Enten and Nate Silver, both of whom are very keen on historical track records. Maybe it's because I'm older than either of them or I've just seen so many predictions done on flimsy data turn out so badly, but I'm not convinced the historical record is the right thing to look at. An issue like "stop and frisk" has a strong chance to be a game changer, and that would work to DiBlasio's advantage.

Enten believes it's all about ethnic politics, which gives Thompson,who is black, an edge over DiBlasio. Silver thinks Quinn's position as front runner for most of the year gives her the inside track. I claim no expertise about New York politics, but I believe more strongly in event driven elections being the correct modern model and that historical data is often so much balloon juice.

I'll post again when there is new data.

I will keep track of these races through the primary on September 10, the likely run-off on October 1 and the general election on November 5.

The not much election news to report.

My Internet crashed on the day of the reporting of the Massachusetts special Senate election. My prediction said Ed Markey looked like a prohibitive favorite and he won. The median poll said he was leading by seven to ten points, he won by ten.

The other two elections in the news are the governor's race in Virginia and the special Senate election in New Jersey. There hasn't been any news since the middle of July in Virginia, where McAuliffe is still leading in the median polls.

There is news from New Jersey, but it is of zero interest to those who want an exciting race. Newark mayor Cory Booker has a huge lead in the Democratic primary and former Bogota mayor Steve Lonegan looks to be a shoo-in in the Republican primary. In the general election, Booker has a massive lead as well, though there is a long time between now and the October 16 special election, strangely scheduled for a Wednesday.

inifinite sums of power series

Yesterday, it was shown that 1/2 + 1/4 + 1/8 + ... = 1. Today we are going to look at the power series for any number x between -1 and 1 (not including 0) for x^0 + x^1 + x^2 + ...

We are going to use a trick called telescoping.

(1 - x)(1 + x + x²) can be split the positive terms and the negative terms.

Positive: 1 + x + x²
Negative: - x - x² - x³

The two of the positive and negative terms cancel out and we are left with just  1 - x³.

If we have a number x between -1 and 1, as the powers increase towards infinity the number gets closer to zero. Dividing both sides by (1 - x) tells us the sum will be equal to 1/(1 - x).

For example, if x = ½, the sum 1 + 1/2 + 1/4 + 1/8 + ... = 1/(1 - ½) = 1/½ = 2.

if x = -½, the sum 1 - ½ + 1/4 - 1/8 + ... = 1/(1 - (-½)) = 1/(3/2) = 2/3.

There are many other methods used in infinite sums, but this is one of the most basic.