FourierKAN
FourierKAN is a Pytorch layer that serves as an alternative to traditional Linear + non-linear activations, utilizing 1D Fourier coefficients inspired by Kolmogorov-Arnold Networks. It optimizes computational efficiency and offers periodic function benefits. The layer is usable on both CPU and GPU, with a naive implementation that manages memory proportional to gridsize and plans for advanced fused operations. Training is enhanced with Brownian noise initialization and frequency regularization for function smoothness. Current offerings are MIT licensed, while future versions may include proprietary fused kernels.
