Enhancing Deep Learning Performance Using Displaced Rectifier Linear UnitКНИГИ » ПРОГРАММИНГ
Название: Enhancing Deep Learning Performance Using Displaced Rectifier Linear Unit Автор: David Macedo Издательство: Editora Dialetica Год: 2022 Страниц: 121 Язык: английский Формат: epub Размер: 10.2 MB
Recently, Deep Learning has caused a significant impact on computer vision, speech recognition, and natural language understanding. In spite of the remarkable advances, Deep Learning (DL) recent performance gains have been modest and usually rely on increasing the depth of the models, which often requires more computational resources such as processing time and memory usage. To tackle this problem, we turned our attention to the interworking between the activation functions and the batch normalization, which is virtually mandatory currently. In this work, we propose the activation function Displaced Rectifier Linear Unit (DReLU) by conjecturing that extending the identity function of ReLU to the third quadrant enhances compatibility with batch normalization. Moreover, we used statistical tests to compare the impact of using distinct activation functions (ReLU, LReLU, PReLU, ELU, and DReLU) on the learning speed and test accuracy performance of VGG and Residual Networks state-of-the-art models. These convolutional neural networks were trained on CIFAR-10 and CIFAR-100, the most commonly used Deep Learning computer vision datasets. The results showed DReLU speeded up learning in all models and datasets. Besides, statistical significant performance assessments (p<0.05) showed DReLU enhanced the test accuracy obtained by ReLU in all scenarios. Furthermore, DReLU showed better test accuracy than any other tested activation function in all experiments with one exception.
Deep Learning is based on the simple principle of hierarchical composition of trivial uniform procedures, the artificial neuron computation. This simplicity produces a considerable advantage over classical Machine Learning approaches. In fact, this characteristic is responsible by the extreme scalability that deep learning system are known to present. Hence, if enough training time and data are available, the massive scalability of Deep Learning approaches appears to achieve lower error rate performance than traditional Machine Learning alternatives.
Скачать Enhancing Deep Learning Performance Using Displaced Rectifier Linear Unit
Deep Learning and Linguistic Representation Название: Deep Learning and Linguistic Representation Автор: Shalom Lappin Издательство: Chapman and Hall/CRC Год: 2021 Страниц: 162 Язык:...
Deep Learning with Applications Using Python Название: Deep Learning with Applications Using Python: Chatbots and Face, Object, and Speech Recognition With TensorFlow and Keras Автор: Navin...
Deep Learning with Python Название: Deep Learning with Python Автор: Francois Chollet Издательство: Manning Publications Год: 2017 Страниц: 384 Формат: True PDF, EPUB Размер:...
Информация
Посетители, находящиеся в группе Гости, не могут оставлять комментарии к данной публикации.
Навигация
Вход на сайт
Реклама
Enhancing Deep Learning Performance Using Displaced Rectifier Linear Unit 
Название: Enhancing Deep Learning Performance Using Displaced Rectifier Linear Unit
