Learnings

https://arxiv.org/abs/1908.07442 With the growth of the neural networks, DNN is showing a good performance in text, image, audio data. However, for the everyday tabular data, we've yet to see the success compare to XGB LGBM. Tree based models have advantages in: decision manifolds, like a super hyperplane (infinite expandable, cuts the table well) interpretable fast to train DNN able to encode data (representation learning) reduce feature engineering online learning discuss: reasons why DNN not performant on tabular data non-linear, not restriction on convergence, easy overfit compare to tree ensemble adding more layers can cause overparameterization, this may be why it isn't performant in tabular data. How good would it be if we can have a framework that's both end-to-end, representation and can do online update for tabular data build decision boundary using DNN tree:  DNN: we can treat the mask as a decision boundary the mask+FC+ReLU is like a vanilla decision tree this i...