Skip to content
This repository has been archived by the owner on Oct 16, 2023. It is now read-only.

Support bias in Attention #158

Merged
merged 1 commit into from
Oct 5, 2023
Merged

Support bias in Attention #158

merged 1 commit into from
Oct 5, 2023

Conversation

kakao-steve-ai
Copy link
Contributor

@kakao-steve-ai kakao-steve-ai commented Oct 5, 2023 •
edited
Loading

🙏 Describe the pull request

✅ Checklist

  • Code follows the project's coding conventions and style.
  • Tests have been added or updated to cover the changes.
  • Documentation has been updated, if necessary.

@kakao-steve-ai kakao-steve-ai self-assigned this Oct 5, 2023
@kakao-steve-ai kakao-steve-ai added the enhancement New feature or request label Oct 5, 2023
daemyung
daemyung suggested changes Oct 5, 2023
View reviewed changes
tests/test_attention.py Outdated Show resolved Hide resolved
tests/test_attention.py Outdated Show resolved Hide resolved
trident/function/function.py Show resolved Hide resolved
trident/kernel/attention.py Outdated Show resolved Hide resolved
trident/kernel/attention.py Outdated Show resolved Hide resolved
@kakao-steve-ai kakao-steve-ai force-pushed the attention branch 3 times, most recently from 9b37dac to a8511a3 Compare October 5, 2023 07:54
daemyung
daemyung suggested changes Oct 5, 2023
View reviewed changes
Copy link
Contributor

@daemyung daemyung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

지금의 커밋 메세지는 새로운 Attention with bias를 구현한 것으로 생각되요. Add support for bias in Attention로 변경하면 어떨까요? 기존의 Attention에 Bias를 지원하는 의미니까 더 적절하다고 생각되요.

tests/test_attention.py Outdated Show resolved Hide resolved
@kakao-steve-ai
Copy link
Contributor Author

지금의 커밋 메세지는 새로운 Attention with bias를 구현한 것으로 생각되요. Add support for bias in Attention로 변경하면 어떨까요? 기존의 Attention에 Bias를 지원하는 의미니까 더 적절하다고 생각되요.

Add, support가 중복되는거 같아 Support bias in Attention 로 수정하였습니다.

@kakao-steve-ai kakao-steve-ai changed the title Implement attention with bias Support bias in Attention Oct 5, 2023
daemyung
daemyung approved these changes Oct 5, 2023
View reviewed changes
Copy link
Contributor

@daemyung daemyung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

mejai1206
mejai1206 approved these changes Oct 5, 2023
View reviewed changes
Copy link
Contributor

@mejai1206 mejai1206 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kakao-steve-ai kakao-steve-ai merged commit c72e077 into main Oct 5, 2023
1 check passed
@kakao-steve-ai kakao-steve-ai deleted the attention branch October 5, 2023 08:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mejai1206 mejai1206 mejai1206 approved these changes

@daemyung daemyung daemyung approved these changes

Assignees

@kakao-steve-ai kakao-steve-ai

Labels
enhancement New feature or request
Projects
Trident
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kakao-steve-ai @daemyung @mejai1206