You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add supports_desc_act self declarative property for BaseModel so models can expose which desc_act order they support. Hymba is a little special since the sequence of module/layer forward is very different from traditional per-layer transformers where input/output is shared between every 2 sequential layers. @ZX-ModelCloud
The text was updated successfully, but these errors were encountered:
Add
supports_desc_act
self declarative property forBaseModel
so models can expose which desc_act order they support. Hymba is a little special since the sequence of module/layer forward is very different from traditional per-layer transformers where input/output is shared between every 2 sequential layers. @ZX-ModelCloudThe text was updated successfully, but these errors were encountered: