Meta-Heuristic Optimization Methods for Quaternion-Valued Neural Networks

Mathematics

Bruce Cox¹  Jeremiah Bill¹  Lance Champagne¹  Trevor Bihl² 

[1] Air Force Institute of Technology, Department of Operational Sciences, WPAFB, OH 45433, USA;Air Force Research Laboratory, Sensors Directorate, WPAFB, OH 45433, USA;

关键词: multilayer perceptrons;    quaternion neural networks;    metaheuristic optimization;    genetic algorithms;   

DOI  :  10.3390/math9090938

来源: DOAJ

【 摘 要 】

In recent years, real-valued neural networks have demonstrated promising, and often striking, results across a broad range of domains. This has driven a surge of applications utilizing high-dimensional datasets. While many techniques exist to alleviate issues of high-dimensionality, they all induce a cost in terms of network size or computational runtime. This work examines the use of quaternions, a form of hypercomplex numbers, in neural networks. The constructed networks demonstrate the ability of quaternions to encode high-dimensional data in an efficient neural network structure, showing that hypercomplex neural networks reduce the number of total trainable parameters compared to their real-valued equivalents. Finally, this work introduces a novel training algorithm using a meta-heuristic approach that bypasses the need for analytic quaternion loss or activation functions. This algorithm allows for a broader range of activation functions over current quaternion networks and presents a proof-of-concept for future work.

【 授权许可】

Unknown