Why do most neural networks first increase dimensionality and then reduce it?

Why do most neural networks first increase dimensions and then reduce them?

I came across a very hardcore theoretical analysis on Zhihu, and to fully understand it, one needs at least a grasp of:

1. The VC dimension from Watermelon Book in machine learning (the book discusses it, but only briefly)
2. Functional analysis
3. Stochastic processes

Still, constrained by the saying "when the book is needed, one regrets having read too little," my knowledge and ability fall short, and I suffer from the disadvantage of insufficient learning, so I cannot fully comprehend this analysis article.

Here is a brief description of the philosophical idea:

1. Complex, inseparable tasks in low-dimensional space are likely to be separable and easy to solve in high-dimensional space. This means that a difficult task is merely a projection of a simple task in high-dimensional space (this description also appears in Zhang Yiming's quotes).
2. Most of what neural networks handle is inseparable, so projecting into high-dimensional space makes it linearly separable, allowing for simple solutions.
3. This idea has long existed in traditional machine learning, namely in SVM, except that the kernel function had to be manually designed (earlier mathematicians designed many kernels with excellent properties, such as the Gaussian kernel). The dimension increase in neural networks is also a kernel function, but one that is dynamic, automatically learnable, and implicit.