Monday, January 26, 2015

Difference between multinomial logistic regression and artificial neural network (multilayer perceptron) without hidden layer

Initially, they seems to be the same: they both have sigmoids within and NxK parameters where N is number of inputs, K is number of outputs. However, regression uses so-called softmax function, so all outputs sum to 1. The difference comes when one needs to run optimization routine. Logistic regression optimizes cross-entropy and ANN optimizes mean squared error. One takes a partial derivatives of these equations which result in slightly different formulas for regression and back-propagation. See more at http://ufldl.stanford.edu/wiki/index.php/Backpropagation_Algorithm and http://ufldl.stanford.edu/wiki/index.php/Softmax_Regression.

Friday, January 23, 2015

Git rebase

Rebase is needed to put your history on top of history of the given branch, when you don't want to see instances of merge in the given branch history. This is important for keeping the main branch history clear.
Rebase process:
git rebase [upstream/master]
Hopefully, everything is ok. If there are conflicts, resolve them by hand or use theirs/ours:
git checkout --theirs (or --ours) filename
Then you need to add the edited file:
git add filename
And continue rebase if hand edited/used theirs or skip it otherwise:
git rebase --continue (or --skip)