Where can I find all the activation function identity?

[NeroBlackstone]

Like sigmoid in Dense(hidden_dims => out_dims, sigmoid)

[avik-pal]

• Built-in ones: https://fluxml.ai/NNlib.jl/dev/reference/#Activation-Functions. Lux implements special fusion routines for these, so using them with Dense, Conv, etc. is extremely efficient.
• In general, any julia function that implements func(::Number) is supported (as long as the AD backend you are using can differentiate that function) (identity falls in this category).
  • If you want the full performance (with operation fusion) you would want to define the rrule manually via https://juliadiff.org/ChainRulesCore.jl/dev/rule_author/rule_definition_tools.html#[@scalar_rule](@ref) but that is not mandatory.