On Architecture Selection for Linear Inverse Problems with Untrained Neural Networks

Entropy

Jonathan Scarlett¹  Hangdong Zhao¹  Yang Sun¹ 

[1] Department of Computer Science, National University of Singapore, 15 Computing Dr., Singapore 117418, Singapore;

关键词: linear inverse problems;    untrained neural networks;    compressive sensing;    deep decoder;    architecture design;    hyperparameters;   

DOI  :  10.3390/e23111481

来源: DOAJ

【 摘 要 】

In recent years, neural network based image priors have been shown to be highly effective for linear inverse problems, often significantly outperforming conventional methods that are based on sparsity and related notions. While pre-trained generative models are perhaps the most common, it has additionally been shown that even untrained neural networks can serve as excellent priors in various imaging applications. In this paper, we seek to broaden the applicability and understanding of untrained neural network priors by investigating the interaction between architecture selection, measurement models (e.g., inpainting vs. denoising vs. compressive sensing), and signal types (e.g., smooth vs. erratic). We motivate the problem via statistical learning theory, and provide two practical algorithms for tuning architectural hyperparameters. Using experimental evaluations, we demonstrate that the optimal hyperparameters may vary significantly between tasks and can exhibit large performance gaps when tuned for the wrong task. In addition, we investigate which hyperparameters tend to be more important, and which are robust to deviations from the optimum.

【 授权许可】

Unknown