Leapfrog Polymorphic Neural Ordinary Differential Equation

Xiao Zhang, Wei Wei, Zhen Zhang, Lei Zhang

科研成果: 期刊稿件文章同行评审

摘要

Neural Ordinary Differential Equations (NODEs) revolutionize the way we view residual networks as solvers for initial value problems (IVPs), with layer depth serving as the time step. In this study, we propose a more efficient extension of NODEs called Leap-Frog Polymorphic Neural ODEs (LF-NODEs). LF-NODEs introduce the leap-frog updating scheme, breaking free from the specific structure of time-evolving mixtures of multiple dynamical systems. In each time step (corresponding to each residual network layer), LF-NODEs integrate the features of multiple dynamical systems (residual network layers) by employing time-variant weights from different dynamical systems. Our LF-NODEs not only overcome the limitations in the representation power of individual residual networks but also provide a more flexible and effective structure for residual networks based on multiple dynamical systems. The efficiency of the leap-frog updating scheme is theoretically derived and demonstrated. Furthermore, we expand the model by incorporating a multi-scale approach (LF-MSNODEs) to achieve even more accurate model representations. Empirical results showcase the improvements provided by our proposed methods in 2D concentric spheres task, irregularly sampled time series prediction, and classification tasks. Additionally, the results provide empirical evidence that the learned feature space enhances system efficiency with fewer parameters and function evaluations compared to the baseline methods.

源语言英语
文章编号0b00006493f45646
期刊IEEE Signal Processing Letters
DOI
出版状态已接受/待刊 - 2025

指纹

探究 'Leapfrog Polymorphic Neural Ordinary Differential Equation' 的科研主题。它们共同构成独一无二的指纹。

引用此