Presenting papers at academic conferences such as MLSys and promoting collaboration with the Silicon Valley company MosaicML

 

A research team led by Professor Seo Ji-won of Hanyang University’s School of Computer Science recently developed an algorithm to learn deep learning models efficiently. This technology, called “Gyro Dropout” improves the dropout algorithm used in deep learning model learning and inference accuracy by understanding how to expand the model’s generalization.

 

Existing dropout algorithms randomly learn the subnetworks of deep learning models to increase the accuracy of the entire network consisting of their aggregation. In the process, each subnetwork is learned only once. Professor Seo’s research team confirmed that learning efficiency and generalization performance improved when the number of learning single subnetworks is increased while the number of learning subnetworks is reduced, and designed a Gyro Dropout algorithm for optimization.

 

Professor Seo’s research team experimentally found that computer vision models, such as AlexNet and ResNet, and natural language processing models, such as BERT, improve performance by applying Gyro Dropout algorithms. In addition, they designed a variant algorithm of Gyro Dropout that can efficiently omit the calculation of neurons dropped out of the graphics processing unit (GPU) upon application, demonstrating that it can improve the learning speed of deep learning models by up to 30%.

 

The team presented the Gyro Dropout algorithm and related research results at MLSys, an excellent academic conference in the machine learning system field. Immediately after the announcement, Silicon Valley–based startup MosaicML showed interest in the research results and proposed collaboration. MosaicML provides an open-source software named Composer and a cloud platform that can easily and conveniently perform artificial intelligence (AI) learning.

 

Professor Seo, with Lee Joon-yeol and Park Ki-hyun, Master’s and PhD students of the research team, worked jointly with MosaicML to release the Gyro Dropout algorithm in Composer version 0.12, an open-source system of MosaicML.

 

[Figure 1] Timeline comparing the existing Dropout algorithm (a) and the Gyro Dropout algorithm (b, c). For Gyro Dropout, the same dropout mask is applied for consecutive iterations.

 

Professor Seo Ji-won