Physics-Informed Neural Networks

📌 Project Overview

Faculty Advisor: Dr. Yang Peng - Department of Physics and Astronomy
Institution: California State Univeristy, Northridge
Dates of Research: June 2024 - August 2024
Funding: SECURE For Student Success \((\textrm{SfS}^2)\)

This project investigates the use of Physics-Informed Neural Networks (PINNs) to model the chaotic behavior of a driven damped pendulum. By comparing Kolmogorov-Arnold Networks (KANs), Multilayer Perceptrons (MLPs), and traditional numerical solvers, we explore the strengths and limitations of neural architectures in capturing complex dynamical systems. This work contributes to a deeper understanding of how emerging deep learning models can offer efficient and accurate alternatives to classical methods in solving nonlinear physical problems.

Intro. & Background

The Kolmogorov–Arnold representation theorem states that any continuous multivariate function can be represented exactly by a finite composition of continuous single-variable functions. Mathematically, for a continuous multivariate function ( f(\mathbf{x}) ):

\[ f(\mathbf{x}) = f(x_1,\dots,x_n) = \sum_{q=0}^{2n}\Phi_q\left(\sum_{p=1}^{n}\phi_{q,p}(x_p)\right) \]

where \(\phi_{q,p}: [0,1] \rightarrow \mathbb{R}\) are continuous univariate functions, and \(\Phi_q: \mathbb{R}\rightarrow \mathbb{R}\).

Kolmogorov-Arnold Networks (KANs) utilize this theorem within neural network architectures by learning these individual univariate functions. Unlike Multilayer Perceptrons (MLPs), which employ fixed activation functions at each neuron as shown below:

\[ MLP(x) = \sigma(W_3 \sigma(W_2 \sigma(W_1 x + b_1) + b_2) + b_3) \]

KANs offer potentially improved interpretability and accuracy because they employ learnable, flexible activation functions directly on connections between neurons:

\[ KAN(x) = \sum_{q=1}^{2n+1} \Phi_q\left(\sum_{p=1}^{n}\phi_{q,p}(x_p)\right) \]

This structural difference may significantly affect model scalability, interpretability, and performance, particularly when approximating complex functions encountered in chaotic dynamical systems.

🎯 Objectives

• Primary goal: Evaluate the performance of Physics-Informed Neural Networks (PINNs) using Kolmogorov-Arnold Networks (KANs) and Multilayer Perceptrons (MLPs) for solving the chaotic driven damped pendulum problem.
• Secondary goal: Assess accuracy and interpretability differences between neural network-based solutions (KAN and MLP) and traditional numerical solvers.
• Technical objective: Explore neural scaling laws and computational efficiency of PINNs.

🧠 Big Research Questions

• How well can KANs and MLPs approximate solutions to chaotic nonlinear differential equations compared to traditional numerical solvers?
• Are KANs inherently more accurate or interpretable than MLPs for modeling chaotic systems?

🛠️ Methods & Tools

• Data Source(s): Numerical solutions generated via SciPy’s Radau ODE solver (benchmarking).
• Algorithms / Models: Kolmogorov-Arnold Networks (KANs), Multilayer Perceptrons (MLPs).
• Software/Environments: Python, Pykan, PyTorch, SciPy, Jupyter notebooks.
• Model Evaluation/Analysis: Loss functions incorporating physics-based boundary conditions and differential equation constraints; comparison of solutions against numerical solver benchmarks.

🖼️ Poster

📦 Deliverables

• Presented research poster at CSUN SfS2 Undergraduate Research Symposium.
• Oral presentation slides developed and delivered.
• Implemented PINN architectures (KAN and MLP) in PyTorch for modeling the damped driven pendulum.

📈 Outcomes

• Both KAN and MLP showed difficulty in modeling chaotic behaviors accurately, especially at higher gamma values (>1.01).
• Numerical solvers, particularly the Radau solver, consistently provided more reliable solutions with significantly faster computational performance compared to neural methods.
• KAN architectures did not outperform MLPs significantly in accuracy or interpretability for the studied chaotic system.

🔁 Ongoing / Future Work

• Further exploration of advanced architectures such as MultKAN (KAN 2.0) to enhance PINN performance.
• Systematic hyperparameter tuning and adaptive spline adjustments for KAN to improve accuracy.
• [Any future papers/presentations tied to this?]

🔗 References / Related Work

1. Liu, Z., et al. (2024). “KAN: Kolmogorov-Arnold Networks.” arXiv:2404.19756
2. Liu, Z., et al. (2024). “KAN 2.0: Kolmogorov-Arnold Networks Meet Science.” arXiv:2408.10205

🧠 What I Learned

This research deepened my understanding of applying PINNs to complex physical systems, highlighting practical limitations in current neural architectures for chaotic dynamics. It provided insight into methodological rigor, benchmarking practices, and computational constraints. The challenges encountered fostered a more critical perspective on neural network applicability and a stronger appreciation for traditional numerical methods.

🙏 Acknowledgments

Thanks to Dr. Yang Peng and the \((\textrm{SfS}^2)\) Program, funded by the United States Department of Education FY 2023 Title V, Part A.