Only the weights from wich there is a path with all non zero-enforced (= dropped out) connections to the output layer will be trained, based on the (backpropagatable) loss.
Also, in case there is no path from the input to the output, a constant function will be learned (output does not vary with input). This training then happens solely on the output data.
Dropout method for disconnected networks
Hello,
I was wondering how would the subnetwork be trained in the case it is disconnected (figure 5 of neural nets lecture 09b)
Thank you for your help,
Good question!
Only the weights from wich there is a path with all non zero-enforced (= dropped out) connections to the output layer will be trained, based on the (backpropagatable) loss.
Also, in case there is no path from the input to the output, a constant function will be learned (output does not vary with input). This training then happens solely on the output data.
1
Add comment