-
-
Notifications
You must be signed in to change notification settings - Fork 125
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add ben * apply patch --------- Co-authored-by: Ben Greenman <[email protected]>
- Loading branch information
1 parent
d6cc92f
commit e6fa11a
Showing
4 changed files
with
119 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
--- | ||
# Display name | ||
title: Ben Greenman | ||
|
||
# Username (this should match the folder name) | ||
authors: | ||
- bennn | ||
|
||
# Is this the primary user of the site? | ||
superuser: false | ||
|
||
# Role/position | ||
role: "PI at the University of Utah" | ||
|
||
# Organizations/Affiliations | ||
organizations: | ||
- name: Kahlert School of Computing | ||
url: "https://www.cs.utah.edu/" | ||
- name: University of Utah | ||
url: "https://www.utah.edu/" | ||
|
||
|
||
# Short bio (displayed in user profile at end of posts) | ||
bio: Ben is a professor at the University of Utah. He likes to go fishing. | ||
|
||
# Social/Academic Networking | ||
# For available icons, see: https://sourcethemes.com/academic/docs/widgets/#icons | ||
# For an email link, use "fas" icon pack, "envelope" icon, and a link in the | ||
# form "mailto:[email protected]" or "#contact" for contact widget. | ||
social: | ||
- icon: envelope | ||
icon_pack: fas | ||
link: mailto:[email protected] | ||
- icon: github | ||
icon_pack: fab | ||
link: https://github.com/bennn | ||
|
||
|
||
# Enter email to display Gravatar (if Gravatar enabled in Config) | ||
email: "" | ||
|
||
# Organizational groups that you belong to (for People widget) | ||
# Set this to `[]` or comment out if you are not using People widget. | ||
user_groups: | ||
- 2024 Mentors | ||
--- | ||
Ben Greenman is an assistant professor in the Kahlert School of Computing at the University of Utah. He earned his Ph.D. from Northeastern University in 2020 and was a CIFellows 2020 postdoc at Brown University. His research focus is the science of language design. His team develops methods to measure performance, prove guarantees, and understand human factors for languages and systems. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
--- | ||
title: "Static Python Perf: Measuring the Cost of Sound Gradual Types" | ||
authors: [bennn] | ||
author_notes: ["University of Utah"] | ||
tags: [osre24, reproducibility, "programming languages", python, types] | ||
date: 2024-01-06 | ||
lastmod: 2024-01-06 | ||
--- | ||
|
||
Gradual typing is a solution to the longstanding tension between typed and | ||
untyped languages: let programmers write code in any flexible language (such | ||
as Python), equip the language with a suitable type system that can describe | ||
invariants in part of a program, and use run-time checks to ensure soundness. | ||
|
||
For now, though, the cost of run-time checks can be enormous. | ||
Order-of-magnitude slowdowns are common. This high cost is a main reason why | ||
TypeScript is unsound by design --- its types are not trustworthy in order | ||
to avoid run-time costs. | ||
|
||
Recently, a team at Meta built a gradually-typed variant of Python called | ||
(*drumroll*) Static Python. They report an incredible 4% increase in CPU | ||
efficiency at Instagram thanks to the sound types in Static Python. This | ||
kind of speedup is unprecedented. | ||
|
||
Other languages may want to follow the Static Python approach to gradual types, | ||
but there are big reasons to doubt the Instagram numbers: | ||
* the experiment code is closed source, and | ||
* the experiment itself is not easily reproducible (even for Instagram!). | ||
|
||
Static Python needs a rigorous, reproducible performance evaluation to test | ||
whether it is indeed a fundamental advance for gradual typing. | ||
|
||
Related Work: | ||
* Gradual Soundness: Lessons from Static Python | ||
<https://programming-journal.org/2023/7/2/> | ||
* Producing Wrong Data Without Doing Anything Obviously Wrong! | ||
<https://users.cs.northwestern.edu/~robby/courses/322-2013-spring/mytkowicz-wrong-data.pdf> | ||
* On the Cost of Type-Tag Soundness | ||
<https://users.cs.utah.edu/~blg/resources/pdf/gm-pepm-2018.pdf> | ||
|
||
### Design and Run an Experiment | ||
|
||
* Topics: `performance`, `cluster computing`, `statistics` | ||
* Skills: Python AST parsing, program generation, scripting, measuring performance | ||
* Difficulty: Medium | ||
* Size: Medium (175 hours) | ||
* Mentor: {{% mention bennn %}} | ||
|
||
Design an experiment that covers the space of gradually-typed Static Python programs | ||
in a fair way. Since every variable in a program can have up to 3 different types, | ||
there are easily 3^20 possibilities in small programs --- far too many to measure | ||
exhaustively. | ||
|
||
Run the experiment on an existing set of benchmarks using a cluster such as CloudLab. | ||
Manage the cluster machines across potentially dozens of reservations and combine | ||
the results into one comprehensive view of Static Python performance. | ||
|
||
|
||
### Derive Benchmarks from Python Applications | ||
|
||
* Topics: `types`, `optimization`, `benchmark design` | ||
* Skills: Python | ||
* Difficulty: Medium | ||
* Size: Small to Large | ||
* Mentor: {{% mention bennn %}} | ||
|
||
Build or find realistic Python applications, equip them with rich types, | ||
and modify them to run a meaningful performance benchmark. Running a benchmark | ||
should produce timing information, and the timing should not be significantly | ||
influenced by random variables, I/O actions, or system events. | ||
|
||
|