Skip to content

Commit

Permalink
Merge #307 (#316)
Browse files Browse the repository at this point in the history
* add ben

* apply patch

---------

Co-authored-by: Ben Greenman <[email protected]>
  • Loading branch information
carlosmalt and bennn authored Jan 12, 2024
1 parent d6cc92f commit e6fa11a
Show file tree
Hide file tree
Showing 4 changed files with 119 additions and 0 deletions.
47 changes: 47 additions & 0 deletions content/authors/bennn/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
# Display name
title: Ben Greenman

# Username (this should match the folder name)
authors:
- bennn

# Is this the primary user of the site?
superuser: false

# Role/position
role: "PI at the University of Utah"

# Organizations/Affiliations
organizations:
- name: Kahlert School of Computing
url: "https://www.cs.utah.edu/"
- name: University of Utah
url: "https://www.utah.edu/"


# Short bio (displayed in user profile at end of posts)
bio: Ben is a professor at the University of Utah. He likes to go fishing.

# Social/Academic Networking
# For available icons, see: https://sourcethemes.com/academic/docs/widgets/#icons
# For an email link, use "fas" icon pack, "envelope" icon, and a link in the
# form "mailto:[email protected]" or "#contact" for contact widget.
social:
- icon: envelope
icon_pack: fas
link: mailto:[email protected]
- icon: github
icon_pack: fab
link: https://github.com/bennn


# Enter email to display Gravatar (if Gravatar enabled in Config)
email: ""

# Organizational groups that you belong to (for People widget)
# Set this to `[]` or comment out if you are not using People widget.
user_groups:
- 2024 Mentors
---
Ben Greenman is an assistant professor in the Kahlert School of Computing at the University of Utah. He earned his Ph.D. from Northeastern University in 2020 and was a CIFellows 2020 postdoc at Brown University. His research focus is the science of language design. His team develops methods to measure performance, prove guarantees, and understand human factors for languages and systems.
Binary file added content/authors/bennn/avatar.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
72 changes: 72 additions & 0 deletions content/project/osre24/uutah/static-python-perf/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
title: "Static Python Perf: Measuring the Cost of Sound Gradual Types"
authors: [bennn]
author_notes: ["University of Utah"]
tags: [osre24, reproducibility, "programming languages", python, types]
date: 2024-01-06
lastmod: 2024-01-06
---

Gradual typing is a solution to the longstanding tension between typed and
untyped languages: let programmers write code in any flexible language (such
as Python), equip the language with a suitable type system that can describe
invariants in part of a program, and use run-time checks to ensure soundness.

For now, though, the cost of run-time checks can be enormous.
Order-of-magnitude slowdowns are common. This high cost is a main reason why
TypeScript is unsound by design --- its types are not trustworthy in order
to avoid run-time costs.

Recently, a team at Meta built a gradually-typed variant of Python called
(*drumroll*) Static Python. They report an incredible 4% increase in CPU
efficiency at Instagram thanks to the sound types in Static Python. This
kind of speedup is unprecedented.

Other languages may want to follow the Static Python approach to gradual types,
but there are big reasons to doubt the Instagram numbers:
* the experiment code is closed source, and
* the experiment itself is not easily reproducible (even for Instagram!).

Static Python needs a rigorous, reproducible performance evaluation to test
whether it is indeed a fundamental advance for gradual typing.

Related Work:
* Gradual Soundness: Lessons from Static Python
<https://programming-journal.org/2023/7/2/>
* Producing Wrong Data Without Doing Anything Obviously Wrong!
<https://users.cs.northwestern.edu/~robby/courses/322-2013-spring/mytkowicz-wrong-data.pdf>
* On the Cost of Type-Tag Soundness
<https://users.cs.utah.edu/~blg/resources/pdf/gm-pepm-2018.pdf>

### Design and Run an Experiment

* Topics: `performance`, `cluster computing`, `statistics`
* Skills: Python AST parsing, program generation, scripting, measuring performance
* Difficulty: Medium
* Size: Medium (175 hours)
* Mentor: {{% mention bennn %}}

Design an experiment that covers the space of gradually-typed Static Python programs
in a fair way. Since every variable in a program can have up to 3 different types,
there are easily 3^20 possibilities in small programs --- far too many to measure
exhaustively.

Run the experiment on an existing set of benchmarks using a cluster such as CloudLab.
Manage the cluster machines across potentially dozens of reservations and combine
the results into one comprehensive view of Static Python performance.


### Derive Benchmarks from Python Applications

* Topics: `types`, `optimization`, `benchmark design`
* Skills: Python
* Difficulty: Medium
* Size: Small to Large
* Mentor: {{% mention bennn %}}

Build or find realistic Python applications, equip them with rich types,
and modify them to run a meaningful performance benchmark. Running a benchmark
should produce timing information, and the timing should not be significantly
influenced by random variables, I/O actions, or system events.


0 comments on commit e6fa11a

Please sign in to comment.