Weekly Report [3]

# Weekly Report [3]

Jinning, 08/02/2018

## Propensity weighted BCE loss

Run experiment with both propensity weighted or not. All the two result are based on same hyperparameters:

• Non-Propensity (weight: 1) Epoch 10: IPS: 54.5463082902, IPS_std: 2.943

• propensity (weight: 1/propensity) Epoch 10: IPS: 55.1079999611, IPS_std: 6.328

#### Intuition:

Introducing propensity can improve the performance of LR model slightly. Overfitting will probably happen at epoch 10. It’s possible that propensity can reduce the bias and relieve overfitting.

## Clip Experiment

Clip propensity weighted BCE loss. This means the weight applied to BCE loss is not simply $\frac{1}{p}$, but $\min{\frac{1}{p}, c}$, where $c$ is a constant. In my experiment, $c$ is selected as 1, 5, 10, 20, 50, 100, 200, 300, 500.

When $c=1$, this is equivalent to unweighted BCE. When $c=500$, since $\frac{1}{p}$ is less than $500$, this is equivalent to $\frac{1}{p}\times loss$.

Result:

Note that this experiment is trained on full training set and test on small test set.
Not so reliable.


The highest IPS appears around $c=200$. So I guess the best $c$ is the average or median of all the propensity values.