Thursday, November 19, 2020

Hot takes from the election

We didn't learn anything new about America : 
        And we never had a chance to learn anything new.  People who thought it would be a landslide for Biden were always going to be wrong. There was never going to be a repudiation or a blue (or a red) wave. There is no silent majority, nor are Trump supporters just a vocal minority.
Caveat: You could argue that the extent (post-election) to which Trump's base and some republicans sank to has been surprising, and I'll give you that. 

Hard to say whether Trump hurt or helped the republican party at the polls :
      Looking at - for instance - Pennsylvania numbers and the fact that Republican congressmen got a lot more votes than Trump, it is easy to lay a hot take that there were many more negative Republican votes for Trump. However, it is not that simple. We don't know how many congressmen actually got votes because of Trump. Overall, the numbers show that Trump underperformed with relation to the GOP, but that doesn't tell a complete story.
Caveat: Trump clearly hurt the party in suburbia, but made gains among non-white voters, but it is hard to disambiguate, especially because of the incumbency effect.

The polls were "accurate" (again) : 
         Whatever your hot take is, I think that the polls (I mean, high quality aggregates like NYT and 538) were pretty good (especially, but not only because of all the pandemic/turnout/mail-in uncertainties). Every single state and every single senate seat that was supposed to go red or blue went red or blue. See the attached pictures from NYT

There were also exactly ZERO big upsets (and 7 mild upsets) out of 535 house seats.   Perhaps if you just follow American politics, you're used to 98% certainty, but if you step out of the red, white and blue sphere, it is pretty remarkable how predictable American elections are (goes for 2016 and 2018 also.)
Caveats: You could argue there was a 3-4% red shift in the tossups, and I'd give you that (though I was sure that it would happen). But really? a 3-4% shift in such a complex system is a big deal?

There was a tiny blue shift (overall) in the presidential election
            In 38 states, Biden's margin was +2% or higher than Clinton's. Only in -2- states did Trump have a +1% or higher margin compared to 2016. This was evenly distributed over battleground and safe states. Geographically, the only major surprise was a big red swing in south Texas. See pic below from Nate Cohn.


Compare that to the 2012-2016 shift, and that's all we need to know.

If that's not very clear, take a look at this (from Cook political report).. 





Trump never had a plan :
           The defining characteristics of Trump : make shit up, survive news cycle to news cycle and bully/fire people to get his way. All those who credited him as an evil genius were always wrong. He is certainly as much a moron as he is evil, and he is certainly not a genius. He has lost 34 out of 36 post-election lawsuits so far, and the two that he won are insignificant. We all know this is just a ploy to create more doubt and stall  certification, but even then it has been an absolute shit-show. 

American institutions are neither strong nor weak :
          But they are getting dangerously weaker by the day. Well.. this one is obvious and doesn't need qualifiers

I was wrong about one thing :
        I thought Americans hated "Losers" more and thought people would jump ship sooner. It could still turn out that Trump will be cast into the rubbish dump of history soon, but as of November 19, I am certainly wrong.
          

Thursday, November 5, 2020

Homestretch in PA, GA & AZ

OK.. There seems to be a lot of confusion about how many votes are left and such. One thing we know for real, however, are the margins. So what I will do below is to plot the real margin vs assumed number of votes left. 

The narrowing of the lead has been remarkably linear in PA (I went in to the reasons why in the previous post), and piecewise linear (it transitioned to a slightly smaller slope on Thursday and back on track on Friday) in GA.

Pennsylvania

Assuming 340375 votes were left when the margin was 97900 

Thursday
10:50 pm Biden is down by 26132 votes and needs to win 57.0 percent of the remaining 185551 votes.
Friday
8:22 am Biden is down by 18049 votes and needs to win 55.3 percent of the remaining 169238 votes
8:57 am Biden leads by   5587 votes and needs to win 48.0 percent of the remaining 138082 votes
6:00 pm Biden leads by 14541 votes and needs to win 43.2 percent of the remaining 106154 votes
9:50 pm Biden leads by 27130 votes and needs to win 33.3 percent of the remaining 81331 votes






Georgia

Assuming 14097 votes were left when the margin was 1775 votes (confirmed by SOS) . Note: In addition to the below, there appear to be about 8000 military/overseas ballots. While I don't think Trump will win those by a lot, Biden coming out with a 2k-3k vote lead out of this lot will guarantee that he gets GA (and he is well on track)

Thursday
11:12 pm Biden is down by 1805 votes and needs to win 57.1 percent of the remaining 12781 votes
Friday
08:22 am Biden leads by 1097 votes and needs to win 43.9 percent of the remaining 8941 votes
03:30 pm Biden leads by 1590 votes and needs to win 39.8 percent of the remaining 7816 votes
Saturday
07:00 pm Biden leads by 9160 votes (** MY FINAL UPDATE **)



Arizona

Arizona has been up and down a bit, but Trump has been consistently underperforming by 5 percent and seems set to run out of votes. I used a few different models (including Gaussian process-based ones with uncertainty, but I will keep it simple and real here).


Friday
10:00 pm Biden leads by 29861 votes and Trump needs to win 58.6 percent of the remaining 172944 votes






State of the races in PA, GA and AZ

8am Thursday:  Biden looks in a terrific position (I would say > 95%) to win Pennsylvania, and in a reasonable position to win Georgia (I would say about 60%). Again, these are for insurance if AZ or NV flips to Trump (and I would give a 30% chance of that happening). OK.. I made up all the numbers above, but you can make your inferences based on the below. 


Pennsylvania:   

89% has been counted and Trump's lead has been steadily declining as seen in the plot below. The decline has been remarkably linear. The reason is that almost all of the votes are coming from mailed-in ballots in very heavily democratic counties (Mainly in Philadelphia where Biden has been winning 80-20). Also, in PA, Biden is crushing Trump 80-20 on the early vote. According to the plot, Biden will cross Trump when approximately 95% of the vote is counted and will win by about 200k.  If the first or second derivatives show a change, I will spot it an update the prediction. 

The main uncertainty is with regard to how many votes are left to count. If Biden maintains his current rate, he'd win with 350k votes left. Looks like there may be at least 650k votes left to count. So even if the rate of winning flattens out, he should be in a good position.



1:20pm:  Biden down 114011 with 550k votes left. This means he has to win 60% of the remaining votes. More importantly, the % counted number in the graph above is correct. 

5:05pm: Biden down 90542 with 493k votes left. Of the 56k votes counted since 1:20pm, Biden has won 70%. 








Georgia:

Georgia officials say that 61000 absentee ballots are left, and they are most highly concentrated in Fulton (Atlanta) and Chatham (Savannah). Biden is behind by 18500, meaning he'd need to win 68% of the remaining votes. He has been doing better than that over the past many hundred thousand votes. 

Here is the clearest picture on how many are left, and where

https://www.ajc.com/politics/about-200000-absentee-ballots-still-need-to-be-counted-in-georgia/BYGJAXV5FRBGBAMA6JPIC7ZL34/?D

25000 left in Fulton, where Biden has been winning by 47% = 11700

7000 left in Clayton, where Biden has been winning by 70% = 4900

So Biden needs to gain roughly 3000 votes out of the remaining 29000. If these were general votes, that'd be more uncertain, but since this is absentee, he has a very good shot.


1:20pm Update. With 50401 votes left, Biden is behind by 13540. As reliable as it gets (from GA SOS). So he needs to win 63% of the remaining votes. Doable. 

Arizona:

About 470k ballots are left and Trump is trailing by about 68k. If Trump wins by the same rate as he won last night (18%), he could come out front by about 10k. But there is no reason to assume linearity as in Pennsylvania for the following reasons:

- PA early votes have been 80-20 for Biden over the past two days CONSISTENTLY and what is left is from that pool. In AZ, there are different types of ballots left to be counted (drop-offs, mail-in, etc). No reason to assume they will all be the same... in fact, the remaining type seems to be less favorable for Trump compared to last night's batch.

- We just don't have enough data points (PA is based on the last 10 updates on nearly 2 M votes, and AZ is based on 2 updates on 50k votes)

- In AZ, ballots are mainly left in blue areas in which Biden has been winning 




Tuesday, November 3, 2020

Election night updates

11 pm

AZ and NV are in. So the math is for Biden to win two out of MI/PA/GA/WI. It is certainly not over because Atlanta, Philadelphia and Detroit have been undercounted (heavily), and so is the early vote (heavily)

12 am 

I started tracking and projecting MI and PA. The simple math is that Biden is winning a higher fraction of the early vote than Trump is winning election day votes... and there are more Early votes than election day votes. When X% of the total vote has been counted, I compare it to the total early vote (from electproject.github.io) and the fraction of the early vote counted thus far, and I ran the numbers assuming the % early and % election day numbers hold up for each candidate. This is assuming that reporting is accurate (and it ignores 3rd party candidates, so there will be +/- 2% error, but that's the margin anyway)

2 am

MI: 64% done ; Projection: Biden +311k

PA: 69% done : Projection; Biden +178k


2:25 am

MI: 66 % done ; Projection: Biden + 355k

PA: 72 % done ; Lead: Trump: + 696k ; Projection: Biden + 250k


3:07 am

MI: 71% done ; Lead: Trump + 295k; Projection: Biden + 400k

PA: 74% done ; Lead: Trump +673k ; Projection: Biden +300k


7:07 am

MI: 84% done ; Lead: Trump + 57k; Projection: Biden +690k

PA: 75% done ; Lead: Trump + 620k ; Projection: Biden +378k


9:12 am

MI: 89% done: Lead: Biden + 7k; Projection: Biden +738k


10:57 am

PA: 78% done: Lead: Trump +542k; Projection: Biden + 400k


11:20 am

MI: 92% done: Lead: Biden + 33k; Projection: Biden + 720k


4:30 pm

MI: 97% done: Lead: Biden + 61k; Projection: Biden + 760k

PA: 84% done: Lead: Trump + 320k; Projection: Biden + 720k


Note: As we get closer to the end, my numbers will be inaccurate because I am rounding out noise, ignoring third party candidates, etc.. and assuming whatever the number of votes NYT says is left is actually left. Also, neither PA nor MI seem to be reporting absentee ballots accurately. This will be fixed when we get actual numbers from the SOS

What is most important, however, is to see how Trump's lead is shrinking. When PA was 72% done, he had a 696k lead. When PA is 84% done, he had a 320k lead. Linear extrapolation tells us that Biden will come out on top. While there is no strict reason for this to hold, we know for sure that first and second derivatives don't lie, and that's why continuous monitoring is important. Besides, most of the remaining votes are left in predictable places.

============================

Predictions going into Election Day :



Nate Silver has the blue wall (MN/MI/WI) @ 96%/95%/94% for Biden respectively. That gets Biden to 253.

He needs 17 from the following

----small lead----

NV 6   (87% by Nate Silver,  +6 ahead in polls)

PA 20 (87%, +5)

-----borderline------

AZ 11  (70%,  + 3)

------polling margin-----

FL 29 (68%)

NC 15 (65%)

In my opinion, Biden only has as much of a shot at winning GA as he has of losing WI, so let's not worry about the likes of GA, IA, etc..

I have no confidence for Biden in FL, and i think Trump will win NC. PA will not finish counting until Friday. So the path to victory is NV+AZ = 17.  Corrected for my gut, I'll give Biden a 70% chance of winning tomorrow.



3 Comments
Like
Comment
Share

Sunday, August 23, 2020

Favorite Documentaries

 Documentary (Series):

- Up (Follows the lives of a few British children .. filmed every 7 years since 1956)

- The Last dance (Michael Jordan's final season & career)

- Tiger king (a league of its own)

- Wild wild country (About the Rajneeshes & Osho's secretary)

- Drive to survive, Season 1 (Inside Formula 1)

Documentary (1-2 hours)

- King of kong (About a bunch of competitive arcade gamers)

- Exit through the gift shop (An amazing documentary about street art)

- Apollo 11 (Amazing footage of the first moon landing. No nonsense.)

- Inside job (Insightful take on the Great Recession)

- Free solo

Not the Gold standard, but still really good

- March of the penguins

- Bowling for Columbine

- Winged migration

- Who killed the electric car?

- Senna

- Triumph of the nerds (Despite the name, a very captivating/informative series about the computer revolution)

- The two Escobars (ESPN documentary about Colombian soccer star & the Colombian drug lord) 



Thursday, June 29, 2017

Economic diversity in US universities

NYTimes has a really neat comparison of economic diversity (the lack thereof, actually) in US universities. Worth checking it outhttps://www.nytimes.com/interactive/2017/01/18/upshot/some-colleges-have-more-students-from-the-top-1-percent-than-the-bottom-60.html. If it wasn't worse than you would've imagined, I'd be surprised.

- Our elite universities have more students from the Top 1%  (households that make $630,000+ / year) than the bottom 60%. For instance, 10% of the students at Michigan come from the Top 1% of the income ladder.
- Half the students at Stanford or Michigan come from the top 10% of wealthy households
- The household income of the median student at Stanford or Michigan is in the 80th percentile 
- There isn't much of a difference between elite public and private universities

- These are 2013 numbers for the incoming freshman class. I can guarantee you that (at least at UM) it has stratified further since then). 
- That said, at most public universities, a good number of students do transfer in from smaller colleges, so these number should look a bit better.





and of course.... going to college matters







Wednesday, April 12, 2017

On College Teaching

I've been wanting to do this for a long while, but now is perhaps the time. A former student (now on the academic market) has been asking for advice and a colleague (probably sarcastic, probably not) wanted to know the "secret" behind why I am a popular teacher. Besides, I am in transit in Atlanta and I still have an hour left, even after a long messaging session with a friend.  

Disclaimer 1: I am very aware that great teaching ratings do not necessarily imply great teaching, nor does it necessarily correlate with teaching outcomes. 

Disclaimer 2: It goes without saying that there is no secret formula to good teaching and quite a bit depends on the circumstances, the teacher, the students, the course (not to mention other things such as sexism, etc).  Nevertheless (by some odd coincidences), I've taught full courses at four different universities (east coast, midwest, west coast and the UK), and at different stages of my career (as a student/lecturer/non-tenure-track/tenure track, etc) and observations hold, more-or-less. Perhaps even more importantly, I really enjoy teaching (especially so in Ann Arbor), and so I thought I'd share my impressions of it.

Another tidbit: Before I started at the Univ of Michigan, I spoke to two of my colleagues-to-be. One said something to the effect of "do an adequate job; never say anything controversial; stick to the topic; play it safe". The other said "I've taught at XXXX, but I am convinced that this is the best university in the world to teach in. A lot depends on your personality. Just be yourself and have fun". Both are excellent teachers, and so you get the point about the generality of advice (feel free to stop reading right now, btw.)

Nevertheless, here goes. 

1. Be less than perfect ; improvise
     Don't script everything. It pays more to screw up (infrequently, of course), make the necessary corrections and apologize, than motoring through a plan. This is unless you are impeccably perfect. Trust me on this: Students are really forgiving when it comes to honest mistakes. Almost all of my lectures have unplanned sequences (well.. sometimes the whole lecture is unplanned, but I won't say how often it happens) and I let my explanations and thinking processes evolve during the lecture. Based on what I've gathered, my lecture preparation time is anywhere between 0% and 50% compared to colleagues.  Sure, every semester, I have the odd crappy lecture or two (or more), but evolving/thinking through a lecture lets students know that teaching is an active process and not a mechanical one.

2. Realize your capital , spend it wisely and have a feedback mechanism
     During my orientation at Michigan, there was an interactive student panel. One particularly eloquent kid said something to the effect of "We know what you had to go through to get to where you are. The moment you walk in to the first lecture, you have great respect. Most often, you have to try hard to lose it". Now, of course, that's not always true, but the logic is sound, even if you walk in with a not-so-stellar reputation. I also think that I do a decent job of gauging how the students are responding to the course in general and make periodic adjustments.

3. Try relating ideas to content in other courses
    In many of my lectures, I try to establish some type of link to something the students have learnt in a math / physics course or a departmental course. I might even explicitly point out to a specific part of the course (or a book) if I knew of the link. Students really appreciate this, and in general it is a good idea to try to harmonize the entire curriculum.

4. Being relatable helps
       I think this is very important (at least for my teaching style). Making the classroom a fun activity pays dividends. I don't shy away from anything relevant (in short bursts). Football, politics, game of thrones, etc. Here is a (but silly) anecdote: A couple of years ago, I was teaching a grad class in the fall. I mentioned a football reference, but didn't get much traction from the class of 50. I asked how many actually followed football. 7 students raised their hands. I said "I quit this job", walked out and came back in after a few seconds... everyone laughed, some of them clapped, but one kid (a back bencher, of course) actually had a football with him and pointed it at me. I asked him to throw it, ran half a route and caught it. This was week 2. I had the class for the entire semester. Bottomline is, being relatable goes a long way.

5. Be lenient, but set boundaries
      Students will be grateful if you do simple things such as moving a submission deadline because they have a congested week, etc. But I also make sure, from the outset,  that there are some aspects with zero compromise (quality of submissions, messing up units, etc).

6. Give a heavy, but well-justified workload
     I can't come up with a reason as to why students respond well to this, but almost every course I've taught has had long, hard homeworks (at least from the perspective of most of the class), but it doesn't seem to piss them off and I end up getting good reviews. If they see that you care enough, they'll be willing to walk the mile.

7. Iterate, provide context
     With the evolution the internet (and other things), students have a shorter attention span (duh!), less patience (duh!) and are less receptive to abstract concepts than ever before. However, it becomes easier to explain abstract concepts (or long derivations) by quickly providing context and iterating on it.  I cut through the first pass very quickly and will purposefully be vague about the details. I'll get to the end, provide context and then re-do the derivation a bit more carefully. Sometimes I conveniently move difficult things towards the end of a lecture, so I can re-do the entire thing again in the next lecture.

8. Provide the big picture
    Telling students what the big picture / overall goals are at the beginning / every so often / at the end of the semester goes a long way. Always emphasize the essential components of the course and try to make sure everyone at least gets the big picture and not bogged down by the finer details.

9. Care
     Students are real people too.. and some of them have problems that real people have. Asking an (appropriate) personal question during office hours can go a long way in establishing a relationship that is mutually beneficial and enhances learning outcomes.

10. Show them that you care
     As important as #9.

11. Don't fake it, let loose (once in a while)
     If you try to be someone your personality doesn't easily lend itself to, be prepared for an uphill climb. If you swear in real life, feel free to swear (tread lightly of course). I drop a few "I can prove this claim, but it is going to be unnecessary bullshit" . If you hate Ohio, do throw in a few "This sucks worse than the state of Ohio."  Point is... let loose.

12. Confuse them (once in a while)
     Sometimes, I pose a question and deliberately give a wrong answer. I would soon emphatically correct it, but seeding a bit of self-doubt helps them work through the process and makes the concept more likely to remember.

13. Use technology, but don't abuse it
     This one is obvious. Show videos, throw in slides once in a while, use fancy active learning tools. There is no such thing as a terrible tool unless you use it unwisely.

14. Read up on education research
     And tell me what it says ;). I haven't done this, but seriously, I am sure one can gain from it (at least anecdotally).