With a Bachelor’s in Mechanical Engineering and a Master’s in Systems Engineering, Matthew Squair is a principal consultant with Jacobs Australia. His professional practice is the assurance of safety, software and cyber-security, and he writes, teaches and consults on these subjects. He can be contacted at mattsquair@gmail.com
This sentiment has been stewing for a while. Even with a consensus “hindsight has 20/20 vision” vis-a-vis, if we had more big data and unlimited processing power, AI would work better, the irony is what to do about it. The trenches lie where they lay and yet using AI is, for the most part, considered the best solution to the problems confronting us, in the real world. Here’s another article, same disposition with more implementation examples and a few links to analysis. https://qz.com/1316050/tech-companies-just-woke-up-to-a-big-problem-with-their-ai/
As a side note, I prefer the ML (Machine Learning) label for the technology, because you only really notice it is in use when it doesn’t perform as expected, at which point Artificial Stupidity would seem the natural differentiation, whereas with under performing ML, it’s just that.
Yes, it’s shaping up to be the tech companies version of commercial fusion, always ‘just five years away’.
And a research paper on ML failure modes…
https://arxiv.org/abs/1803.04585
I had a quick skim of the paper and I noted that David uses a Gaussian noise model for his regressional Goodhart failure mode. Which in itself is a critical assumption. If noise has say a Cauchy distribution and therefore a heavy tail then what would be the effect? Would this modality tend to dominate the ensemble?
Any distribution will show the same effect. I’ve actually done the simulation for cauchy distributions specifically. In fact, the normal distribution for noise was used as an extreme example, since the fatter the tails are the worse the effect is. This is because the divergence in the tails is what drives the phenomenon of optimization error.
If you’d like to simulate it yourself, try the following in R:
x <- rcauchy(10000,0,1)/10
Goal <- rnorm(10000)
Metric <- rnorm(10000,x+Goal)
trunc_val = 3
plot(Goal[which(Metric<=trunc_val)],Metric[which(Metrictrunc_val)],Metric[which(Metric>trunc_val)],cex=0.5,col=”blue”)
The truncation value shows the effect of selecting for high values of the metric instead of the (unobservable) goal.
Matthew
I’m sorry I’d didn’t think of contacting you before leaving Australia last week. I was speaking at the PGSC conference in Canberra. We’ll be there again for next years conference and a second conference in Sydney both focused on ADF and Large Infrastructure. Kym Henderson, Peter Colquhoun, and Pat Weaver are the conference organizers. If this domain area is of interest, give some though to speaking
Glen B. Alleman, MSSM Integrated Program Performance Management DOD, DHS, DOE, NASA, DOJ, GSA 303-241-9633 glen.alleman@niwotridge.com
>
Next time your in country 🙂