Skip to content

Commit

Permalink
Changed pfp and fixed alt text
Browse files Browse the repository at this point in the history
  • Loading branch information
waynchi committed Nov 12, 2024
1 parent 0505b27 commit a659847
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions _posts/2024-11-12-copilot-arena.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,8 +144,8 @@ Most current Copilot Arena users code in Python, followed by javascript/typescri
**What kind of context lengths are we looking at?**
The mean context length is 1002 tokens and the median is 560 tokens. This is much longer than tasks considered in existing static benchmarks. For example, human eval has a median length of ~100 tokens.

<img src="/assets/img/blog/copilot_arena/filetype_dist.png" alt="Copilot Arena filetype distribution" style="display:block; margin-top: auto; margin-left: auto; margin-right: auto; margin-bottom: auto; width: 90%">
<p style="color:gray; text-align: center;">Figure 3. Filetypes requested in Copilot Arena. Filetypes are determined based on file extension.</p>
<img src="/assets/img/blog/copilot_arena/context_length_dist.png" alt="Copilot Arena filetype distribution" style="display:block; margin-top: auto; margin-left: auto; margin-right: auto; margin-bottom: auto; width: 90%">
<p style="color:gray; text-align: center;">Figure 3. Context length of files requested in Copilot Arena.</p>

**Are people biased towards the top completion?** Yes. In fact, 82% of accepted completions were the top completion. We are still analyzing our data, but here are some of our insights.

Expand Down
Binary file modified assets/img/blog/copilot_arena/leaderboard_pfp.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit a659847

Please sign in to comment.