-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Deploying to gh-pages from @ 07072b7 🚀
- Loading branch information
1 parent
37f1b34
commit b22647f
Showing
5 changed files
with
6 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| {"componentChunkName":"component---src-templates-on-my-mind-thought-js","path":"/on-my-mind/2024-10-27/","result":{"data":{"site":{"siteMetadata":{"title":"DEFINITIVELY NOT JAMES","author":"James Ward"}},"thought":{"id":"bd4c2bb5-7946-5621-9608-95f70ecd72da","html":"<p>\n <img src='https://media.giphy.com/media/N2QQfEMAogAEQhJlkw/giphy.gif' alt='' loading='lazy' />\n </p>\n<p>There’s a <a href=\"https://commoncog.com/becoming-data-driven-in-business/\">fantastic set of essays</a> by Cedric Chin about becoming a\ndata driven business. It starts off on a really inspiring\n<a href=\"https://commoncog.com/goodharts-law-not-useful/\">essay dissecting Goodhart’s law</a>. Most of the essays are focused\naround Amazon and often reference the book that Cedric had helped with:\n<a href=\"https://www.goodreads.com/book/show/53138083\">“Working Backwards” by Colin Bryar and Bill Carr</a>.</p>\n<p>For those that don’t know it, Goodhart’s law is:</p>\n<blockquote>\n<p>When a measure becomes a target, it ceases to be a good measure.</p>\n</blockquote>\n<p>Every measure is a proxy for what you actually want — and is always an\nimperfect proxy for it. Even when you’re using a tape measure you’re not\nquite getting what you want - an exact measurement of a distance. Instead,\nyou’re getting something that’s “close enough” in precision to the real\ndeal that it works fine.</p>\n<p>In <a href=\"https://www.holistics.io/blog/four-types-goodharts-law/\">another essay</a>, Cedric summarized a <a href=\"https://arxiv.org/abs/1803.04585\">paper by David Manheim and Scott Garrabrant</a>\nthat breaks down Goodhart’s law into four “flavors”: regressive, extremal,\ncausal, and adversarial. This is reflected in Donald Wheeler’s take on Goodhart’s\nlaw from <a href=\"https://www.goodreads.com/book/show/63859\">Understanding Variation</a>:</p>\n<blockquote>\n<p>When people are pressured to meet a target value there are three ways they can proceed:</p>\n<ol>\n<li>They can work to improve the system</li>\n<li>They can distort the system</li>\n<li>Or they can distort the data</li>\n</ol>\n</blockquote>\n<p>To me, this is a fantastic way to frame Goodhart’s law. Instead of it being a\nproblem declared out into the ether it’s something that can be solutioned against.\nThe solutions would be to make it more difficult to distort the data or the system -\nor you can make it easier to improve the system.</p>\n<p>Wheeler notes that this can be seen between the Voice of the Customer - the\nexpectations - and the Voice of the Process - what can be done in the real world.\nFocusing entirely on the Voice of the Customer without understanding the\nVoice of the Process leads to gaming the measurements, inadvertently or not.</p>\n<p>The essay series goes into depth on how Amazon has approached this problem - via their\nWeekly Business Review (WBR). The goal of the WBR is to provide flexibility to make\nthe various systems work for them and also to drop measurements that aren’t useful.\nIt promotes iteration and learning, synchronizing leadership rather than promoting\nblindly following numbers.</p>\n<p>Proxy measures will always allow for some form of gaming the system. At the same time,\nyou can’t run a business without these kinds of proxy measures. Goodhart’s law isn’t wrong -\nbut instead of approaching it by choosing to avoid measuring, we instead need\nto find solutions to prevent it.</p>\n<p>The problem isn’t having targets - it’s failing to reconcile those\ntargets with the real world.</p>","excerpt":"There’s a fantastic set of essays by Cedric Chin about becoming a\ndata driven business. It starts off on a really inspiring\nessay…","fields":{"slug":"/2024-10-27"},"frontmatter":{"date":"2024/10/27"}},"previous":{"id":"d162715b-0125-5d0f-ad1d-63df3a4ef2fe","fields":{"slug":"/2024-04-02"},"frontmatter":{"date":"2024/04/02"}},"next":null},"pageContext":{"slug":"/2024-10-27","previous":"/2024-04-02","next":null}},"staticQueryHashes":["63159454"],"slicesMap":{}} | ||
| {"componentChunkName":"component---src-templates-on-my-mind-thought-js","path":"/on-my-mind/2024-10-27/","result":{"data":{"site":{"siteMetadata":{"title":"DEFINITIVELY NOT JAMES","author":"James Ward"}},"thought":{"id":"bd4c2bb5-7946-5621-9608-95f70ecd72da","html":"<p>\n <img src='https://media.giphy.com/media/N2QQfEMAogAEQhJlkw/giphy.gif' alt='' loading='lazy' />\n </p>\n<p>There’s a <a href=\"https://commoncog.com/becoming-data-driven-in-business/\">fantastic series of essays</a> by Cedric Chin about becoming a\ndata driven business. It starts off on a really inspiring\n<a href=\"https://commoncog.com/goodharts-law-not-useful/\">essay dissecting Goodhart’s law</a>. Most of the essays are focused\naround Amazon and often reference the book that Cedric had helped with:\n<a href=\"https://www.goodreads.com/book/show/53138083\">“Working Backwards” by Colin Bryar and Bill Carr</a>.</p>\n<p>For those that don’t know it, Goodhart’s law is:</p>\n<blockquote>\n<p>When a measure becomes a target, it ceases to be a good measure.</p>\n</blockquote>\n<p>Every measure is a proxy for what you actually want — and is always an\nimperfect proxy for it. Even when you’re using a tape measure you’re not\nquite getting what you want - an exact measurement of a distance. Instead,\nyou’re getting something that’s “close enough” in precision to the real\ndeal that it works fine.</p>\n<p>In <a href=\"https://www.holistics.io/blog/four-types-goodharts-law/\">another essay</a>, Cedric summarized a <a href=\"https://arxiv.org/abs/1803.04585\">paper by David Manheim and Scott Garrabrant</a>\nthat breaks down Goodhart’s law into four “flavors”: regressive, extremal,\ncausal, and adversarial. This is reflected in Donald Wheeler’s take on Goodhart’s\nlaw from <a href=\"https://www.goodreads.com/book/show/63859\">Understanding Variation</a>:</p>\n<blockquote>\n<p>When people are pressured to meet a target value there are three ways they can proceed:</p>\n<ol>\n<li>They can work to improve the system</li>\n<li>They can distort the system</li>\n<li>Or they can distort the data</li>\n</ol>\n</blockquote>\n<p>To me, this is a fantastic way to frame Goodhart’s law. Instead of it being a\nproblem declared out into the ether it’s something that can be solutioned against.\nThe solutions would be to make it more difficult to distort the data or the system -\nor you can make it easier to improve the system.</p>\n<p>Wheeler notes that this can be seen between the Voice of the Customer - the\nexpectations - and the Voice of the Process - what can be done in the real world.\nFocusing entirely on the Voice of the Customer without understanding the\nVoice of the Process leads to gaming the measurements, inadvertently or not.</p>\n<p>The essay series goes into depth on how Amazon has approached this problem - via their\nWeekly Business Review (WBR). The goal of the WBR is to provide flexibility to make\nthe various systems work for them and also to drop measurements that aren’t useful.\nIt promotes iteration and learning, synchronizing leadership rather than promoting\nblindly following numbers.</p>\n<p>Proxy measures will always allow for some form of gaming the system. At the same time,\nyou can’t run a business without these kinds of proxy measures. Goodhart’s law isn’t wrong -\nbut instead of approaching it by choosing to avoid measuring, we instead need\nto find solutions to prevent it.</p>\n<p>The problem isn’t having targets - it’s failing to reconcile those\ntargets with the real world.</p>","excerpt":"There’s a fantastic series of essays by Cedric Chin about becoming a\ndata driven business. It starts off on a really inspiring\nessay…","fields":{"slug":"/2024-10-27"},"frontmatter":{"date":"2024/10/27"}},"previous":{"id":"d162715b-0125-5d0f-ad1d-63df3a4ef2fe","fields":{"slug":"/2024-04-02"},"frontmatter":{"date":"2024/04/02"}},"next":null},"pageContext":{"slug":"/2024-10-27","previous":"/2024-04-02","next":null}},"staticQueryHashes":["63159454"],"slicesMap":{}} |
Large diffs are not rendered by default.
Oops, something went wrong.
Binary file not shown.