Purdue Proposal

# Experiment Proposal

### hypothesis 1: neural net learns low level features first

#### How to set up:

train neural network to fit polynomial

using random sampling and expectation to calculate each coefficient.

calculate the MSE of coefficient

• if the MSE of low level features are lower, hypothesis is true.

### hypothesis 2: the more data used, the higher accuracy of a coefficient will be achieved.

#### How to set up:

using different data scale to train a network

calculate the MSE of each coefficient

draw the MSE-coefficient curve for different data scale.

draw the MSE of an coefficient v.s. different data scale.

• whether a critical behavior appears
• See whether the MSE of a coefficient is incrasing by data scale

### hypothesis 4: With the incrase of network parameters, there will be a critical behavior.

• How to set up:

fixed data scale

draw the MSE-coefficient curve for different parameter scale.

draw the MSE of an coefficient v.s. different parameter scale.

• whether a critical behavior appears
• See whether the MSE of a coefficient is incrasing by parameter scale

### hypothesis 3: If only degree one appears, i.e. $f(x)=a_1x_1+a_2x_2$, the coefficient with large scale will be learnt first.

#### How to set up:

using fixed data scale and parameters.

Set $a_1$ as larger value.

calculate the MSE of each coefficient

• If the MSE of $a_1$ is larger, the hypothesis is proved