ATAGI and the law of large numbers


ATAGI provided an estimate of the risk associated with TTS events (bad post vaccination events). This was at the time based on the evidence that was available. The problem with this is that we run into de Moivre’s `Law of Large Numbers’. Which in essence says we can expect small samples to have much greater variability than larger sample sizes. As a result when tracking vaccine side effects you can expect the estimated rate of occurrence to be all over the shop initially because one incident against a very low number of vaccinations (our sample) can skew the incident rate a lot.

To give you a feel for this effect our friend de Moivre found that the size of a typical discrepancy (variance) goes up proportional to the square root of the number of samples. As we divide that number by the total number of samples to get our proportional rate the proportional discrepancy is going to increase as our sample gets smaller. It’s a bit like throwing a small rock into a small pool (big splash proportionally) and then retreiving it and tossing it into a big lake (small small splash proportionally).

So what ATAGI did wrong with their initial estimate of TTS was not considering that for a very rare event like TTS the population needed to dial back the variance associated with a proportionally small size sample was going to big, and the needed to account for it. But they didn’t. Instead the just threw up the raw frequencies in the risk assessment. Of course the world has moved on and it turns out that the initial TTS estimates totally over estimated the rate of TTS. Well I guess a 17th century mathematician still has a few things to teach AGATI.