Skip to content

Commit

Permalink
rearrange
Browse files Browse the repository at this point in the history
  • Loading branch information
sigmoidandsoftmax authored Oct 23, 2024
1 parent 9c77365 commit 69bd32f
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion _posts/2024-05-07-sigmoid-and-softmax.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,12 +143,13 @@ $$
\end{align}
$$

## Discussion and Conclusion

![Sigmoid with entropy](https://github.com/sigmoidandsoftmax/sigmoidandsoftmax.github.io/blob/main/assets/img/sigmoid_and_softmax/sigmoid_neg_entropy.png?raw=true)


![Softmax with entropy](https://github.com/sigmoidandsoftmax/sigmoidandsoftmax.github.io/blob/main/assets/img/sigmoid_and_softmax/softmax_neg_entropy.png?raw=true)
Figure 1. Effect of the entropy term in values of the objective function of sigmoid and softmax.

## Discussion and Conclusion

Sigmoid is the function when applied to a scalar gives the probability that has maximum entropy and multiplication value with the input scalar. Softmax is the function that when applied on a vector gives a probability vector that has minimum negative entropy and dot product value with the input vector. Figure 1 shows the values of the objective function for different weights to the entropy term.

0 comments on commit 69bd32f

Please sign in to comment.