Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proposal for new Evaluation repo #79

Merged
merged 2 commits into from
Jun 11, 2024
Merged

Proposal for new Evaluation repo #79

merged 2 commits into from
Jun 11, 2024

Conversation

nathan-weinberg
Copy link
Member

Resolves #76

@mergify mergify bot added the backend InstructLab Backend Services label Jun 6, 2024
@nathan-weinberg nathan-weinberg force-pushed the new-eval branch 5 times, most recently from ec0937d to dfad7d9 Compare June 10, 2024 15:11
@nathan-weinberg nathan-weinberg changed the title [WIP] Proposal for new Evaluation repo Proposal for new Evaluation repo Jun 10, 2024
@nathan-weinberg nathan-weinberg marked this pull request as ready for review June 10, 2024 15:11
@nathan-weinberg nathan-weinberg force-pushed the new-eval branch 3 times, most recently from 2678fd5 to 88ed642 Compare June 10, 2024 17:08
russellb
russellb reviewed Jun 10, 2024
View reviewed changes
Copy link
Member

@russellb russellb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one suggested addition

docs/backend/eval-repo.md Outdated Show resolved Hide resolved
@nathan-weinberg nathan-weinberg requested a review from cdoern June 10, 2024 18:31
russellb
russellb reviewed Jun 10, 2024
View reviewed changes
Copy link
Member

@russellb russellb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - going to leave this open a while longer to gather more ACKs

@nathan-weinberg nathan-weinberg mentioned this pull request Jun 10, 2024
Open
@nathan-weinberg
Proposal for new Evaluation repo
1b40cb0 
Signed-off-by: Nathan Weinberg <[email protected]>
@danmcp
Copy link
Member

danmcp commented Jun 10, 2024

LGTM

@alimaredia
Copy link
Contributor

Besides the paragraph I wanted removed, this proposal in fantastic! Thanks Nathan!

@nathan-weinberg @alimaredia
Wording suggestion from Ali
686870e 
Co-authored-by: Ali Maredia <[email protected]>
Signed-off-by: Nathan Weinberg <[email protected]>
@nathan-weinberg nathan-weinberg merged commit 3ee5f52 into instructlab:main Jun 11, 2024
4 checks passed
@nathan-weinberg nathan-weinberg deleted the new-eval branch June 11, 2024 01:40
lhawthorn
lhawthorn reviewed Jun 11, 2024
View reviewed changes
Copy link
Member

@lhawthorn lhawthorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work @nathan-weinberg

@lhawthorn
Copy link
Member

@nathan-weinberg I believe our intent is to have proposals for new repos approved by the Oversight Committee, though I admit that if that is true it could be made more clear in our governance documentation.

Arrived here via ping from @russellb to the oversight committee to review this proposal, so will let him as pinger and Oversight Committee chair opine on my statement above.

@nathan-weinberg
Copy link
Member Author

@lhawthorn thanks! I assumed the approval from @russellb as a member of the Oversight Committee fulfilled that requirement, is that not the case?

The repo is already created so it's a moot point now, but if more process is needed someone will need to flesh that out in the doc you linked for next time.

@alinaryan
Copy link
Member

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lhawthorn lhawthorn lhawthorn left review comments

@russellb russellb russellb left review comments

@alimaredia alimaredia alimaredia approved these changes

@alinaryan alinaryan Awaiting requested review from alinaryan

@danmcp danmcp Awaiting requested review from danmcp

@xukai92 xukai92 Awaiting requested review from xukai92

@cdoern cdoern Awaiting requested review from cdoern

Assignees
No one assigned
Labels
backend InstructLab Backend Services
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Create proposal for new evaluation library repo
6 participants
@nathan-weinberg @danmcp @alimaredia @lhawthorn @alinaryan @russellb