Anti-Distillation: Knowledge Transfer from a Simple Model to the Complex One
[In Russian]
Антидистилляция: перенос знаний от простой модели к более сложной
Petrushina, Kseniia,
Bakhteev, Oleg,
Grabovoy, Andrey abd Strijov, Vadim
2022
Speaker: Kseniia Petrushina
Moscow
The paper considers the problem of adapting the model to new data with a large amount of information. We propose to build a more complex model using the parameters of a simple one. We take into account not only the accuracy of the prediction on the original samples but also the adaptability to new data and the robustness of the obtained solution. The work is devoted to developing the method that allows adapting the pre-trained model to a more heterogeneous dataset. In the computational experiment, we analyse the quality of predictions and model robustness on Fashion-MNIST dataset.
@conference{petrushina2022isp,
presenter = {Петрушина К. Е.},
presenter_en = {Kseniia Petrushina},
abbr = {ИСП},
abbr_en = {ISP},
title = {Антидистилляция: перенос знаний от простой модели к более сложной},
title_en = {Anti-Distillation: Knowledge Transfer from a Simple Model to the Complex One},
author_en = {Petrushina, Kseniia and Bakhteev, Oleg and Grabovoy, Andrey abd Strijov, Vadim},
author = {Петрушина, К. Е. and Бахтеев, О. Ю. and Грабовой, А. В. and Стрижов, В. В.},
booktitle = {Ivannikov ISPRAS Open Conference},
supp = {https://youtu.be/RtttuJMnyzo?t=14545},
note = {Москва},
note_en = {Moscow},
year = {2022}
}