Forget Me Not: Fighting Local Overfitting With Knowledge Fusion and Distillation.

Overfitting in deep neural networks occurs less frequently than expected. This is a puzzling observation, as theory predicts that greater model capacity should eventually lead to overfitting - yet this is rarely seen in practice. But what if overfitt…

Preview only. Read the full abstract at the source