应304am永利集团官网李朋副教授邀请, 中山大学数学学院(珠 海)张海樟教授, 将于2022年11月11号(星期五)下午15:00-16:00在线举办学术报告.

报告题目：Uniform Convergence of Deep Neural Networks with Contractive Activation Functions and Poolings

腾讯会议ID: 897-121-032

报告摘要: Deep neural networks, as a powerful system to represent high dimensional complex functions, play a key role in deep learning. Convergence of deep neural networks is a fundamental issue in building the mathematical foundation for deep learning. Existing researches on this subject only studied neural networks with the Rectified Linear Unit (ReLU) activation function. The important pooling strategy was not considered either. The methods in these studies made use of the piecewise linearity of ReLU. On the other hand, there are over forty activation functions that are commonly used in artificial neural networks, many of which are nonlinear. In this paper, we study the convergence of deep neural networks as the depth tends to infinity for general activation functions which cover most of commonly-used activation functions in artificial neural networks. Pooling will also be studied. Specifically, for contractive activation functions such as the logistic sigmoid function, we establish a uniform and exponential convergence of the associated deep neural networks. Such a convergence will not be affectedwhen the max pooling or average pooling is added to the structure of the network. We also adapt the matrix-vector multiplication approach developed in our recent papers to prove that the major condition there is still sufficient for the neural networks defined by nonexpansive activation functions to converge.

报告人简介

张海樟,中山大学数学学院(珠海)教授. 研究兴趣包括学习理论、应用调和分析和函数逼近. 代表性成果有再生核的Weierstrass逼近定理, 以及在国际上首创的再生核巴拿赫空间理论. 以再生核巴拿赫空间为基础的心理学分类方法入选剑桥大学出版社的《数学心理学新手册》.在Journal of Machine Learning Research、Applied and Computational Harmonic Analysis、Neural Networks, Neural Computation、Neurocomputing、Journal of Approximation Theory等发表多篇原创性工作, 单篇最高他引超过200次. 主持包括优秀青年基金在内的四项国家基金. 详情见其主页https://mathzh.sysu.edu.cn/zh-hans/teacher/106

甘肃应用数学中心

甘肃省高校应用数学与复杂系统省级重点实验室

萃英学院

304am永利集团官网

2022年11月3日