The brief overlap of Anti-fragility and Machine Learning models
The mental model of Anti-fragility developed by Nassim Nicholas Taleb in his book Anti-fragile is a one-size-fits-all model for timeless successful systems. It’s crazy how many systems, from financial markets to human biology, show characteristics of benefitting from stressors. The more you think about it, the more universal the idea becomes.
Anti-fragility is the property of systems to thrive in adversity and volatility
As I was reading the section about Interventionism and Iatrogenics, I realised the overlap Machine Learning has with Anti-fragility in order to model the real world.
Before going forward, I’d inform that this is just a macro non-mathematical, rather philosophical outtake of machine learning methods and in no way would I be going into any details of it. If you have a general idea of how ML models work, you’ll be able to follow along.
Applying anti-fragility to ML models
It can be seen that every successful attempt at modelling the real world has a touch of anti-fragile behaviour. Machine Learning or computational ways of understanding a phenomena does too. Drawing parallels from the definition, in case of ML models, adversity corresponds to data that the model hasn’t seen before i.e. test data. To be a truly anti-fragile model, it has to not only correctly classify and interpret it, it also has to learn from it to better classify unseen data in the future. As Taleb stresses in almost every chapter, the aim of an anti-fragile system should be to gain from positive volatility and have minimum damage from negative volatility, instead of trying to predict future events.
Iatrogenics and Overfitting
Iatrogenics is when a treatment causes more harm than benefit. As iatros means healer in Greek, the word means “caused by the healer” or “brought by the healer.”
“If you want to accelerate someone’s death, give him a personal doctor. I don’t mean provide him with a bad doctor: just pay for him to choose his own.” — Nassim Nicholas Taleb
As Taleb began explaining the negative effects of unnecessary interventionism through Iatrogenics, I couldn’t help but draw a parallel to the overfitting problem in ML models. Overfitting is when a model exactly represents the test data and ends up having low accuracy for the real world data. Much like Iatrogenics, overfitting ends up solving a non-existent problem. And both arise from our love of sophisticated methods, even when not necessary. The solution of course is to minimize interventionism. In case of doctors causing illness, trying to heal non-existent problems, they come in the way of natural anti-fragility of human biology which heals itself better than any doctor. Similarly, a good ML model must prevent interventionism and have a general approach.
Procrastination and Dropout
“Few understand that procrastination is our natural defense, letting things take care of themselves and exercise their antifragility; it results from some ecological or naturalistic wisdom, and is not always bad — at an existential level, it is my body rebelling against its entrapment. It is my soul fighting the Procrustean bed of modernity.” — Nassim Nicholas Taleb
Taleb, much like a college student, defends procrastination as a way to enable execution of work only when its necessary, resisting interventionism. The concept of Dropout in Neural Networks works on similar philosophy. Just like procrastination is the cure to interventionism, dropout works like a cure for overfitting.
Simply put, dropout method is probabilistically dropping out nodes in a neural network. It is the process of forgetting learnings from the data, on purpose, in order to achieve generalization. It is generally used with large neural networks trained on relatively small datasets. It ends up approximating the metrics involved, and the model procrastinates until a general trend is observed.
Skin in the game and Test data validity
“If your private life conflicts with your intellectual opinion, it cancels your intellectual ideas, not your private life.” — Nassim Nicholas Taleb
Another interesting interpretation of skin in the game that I found online was, — look at a person’s portfolio instead of asking him/her for financial advice. Only actions are undeniable proofs, unlike ideas which are fragile. It simply means, that the test of any theory is its validity in the real world and proof of conviction is the ability to take risk for your opinions. My claim that a particular ML model works well, is risky if and only if I test it for previously unseen data i.e. test data. Only this act of validating a model proves its accuracy. If a model only works with the data it was trained on, it’s playing safe and has no skin in the game.
Not a complete overlap
Other features of Antifragile systems (like optionality, barbell strategy etc) can be explored further for their existence in ML models as well. But I believe it won’t be a complete overlap. Through different lenses, even this overlap might seem to be a far off connection. The scope of anti-fragility is beyond just computational methods of modelling the world. In fact, conversely, probabilities and statistics have done an awful job at predicting or explaining successful phenomenas. Be it human evolution or pervasive technology, numbers have never been able to keep up. Only the risk taking appetite and the benefit from occasional losses has made these systems successful.
I welcome any supporting argument, or criticism to this article. Its just a connection my mind made while reading the book. And ideas, until tested, are fragile :)
