Update tokenizer to fix some build issues

transitive-bullshit released this 18 Mar 13:55

· 348 commits to main since this release

c32d986

feat: update tokenizer 66be552
chore: update dev deps fa8012c
fix: CI 48ad624
feat: add gpt-4 demo fcc84b6

v5.1.1...v5.1.2

Assets 2