-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #14 from 4ARMED/readme_update
Release 1.6.4
- Loading branch information
Showing
7 changed files
with
215 additions
and
114 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,9 @@ | ||
| # SRI Checker | ||
|
|
||
| Ridiculously simple Python script for grabbing `<script>` tags from a remote URL and outputting any that don't have an `integrity` attribute for Subresource Integrity. | ||
| Ridiculously simple Python script for grabbing resource tags (script, link) from a remote URL and outputting any that don't have an `integrity` attribute for Subresource Integrity. | ||
|
|
||
| We use it for <a href="https://www.4armed.com/assess/penetration-testing/">application security reviews</a> but you can use it for whatever you like! | ||
|
|
||
| ## Docker | ||
| The easiest way to run the application is to build docker image provided in the repository `docker build -t sri-check .` and then run it as `docker run --rm sri-check --help`. See [Usage](#usage) section for more details. | ||
|
|
||
| ## Install | ||
|
|
||
| The easiest way is to install it from PyPi using: | ||
|
|
@@ -17,75 +14,164 @@ pip install sri-check | |
|
|
||
| Then you should have `sri-check` in your PATH. | ||
|
|
||
| If you want to install from source then clone this repo, then you need a few libraries. There's a `requirements.txt` file in this repo for use with pip. We recommend using a venv to isolate these dependencies. | ||
| If you want to install from source then clone this repo and install the required libraries. There's a `requirements.txt` file in this repo for use with pip. We recommend using a venv to isolate these dependencies. | ||
|
|
||
| If you run `make` it will set one up for you. | ||
|
|
||
| ## Usage | ||
|
|
||
| If you use the `Makefile` it will set one up for you. | ||
| ### Check | ||
|
|
||
| Now you can run the script which in its most basic form takes one argument, the URL you wish to check. | ||
| ```bash | ||
| $ make | ||
| Requirement already satisfied: beautifulsoup4==4.9.3 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (4.9.3) | ||
| Requirement already satisfied: certifi==2020.12.5 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (2020.12.5) | ||
| Requirement already satisfied: chardet==4.0.0 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (4.0.0) | ||
| Requirement already satisfied: idna==2.10 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (2.10) | ||
| Requirement already satisfied: requests==2.25.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (2.25.1) | ||
| Requirement already satisfied: soupsieve==2.1 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (2.1) | ||
| Requirement already satisfied: urllib3==1.26.3 in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (1.26.3) | ||
| Now run source venv/bin/activate | ||
| sri-check https://www.4armed.com/ | ||
| <script src="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.js"></script> | ||
| <link href="https://use.typekit.net/vlp2azz.css" rel="stylesheet"/> | ||
| <link href="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.css" rel="stylesheet"/> | ||
| ``` | ||
|
|
||
| Alternatively you can run the commands yourself. | ||
| Tut, tut, you can see above that on our website we have a few external dependencies without SRI. | ||
|
|
||
| If sri-check finds any tags missing SRI that it will output, it also returns a non-zero exit code (99). In this way you can use sri-check in pipelines or scripts if you wish. I've included the dollar signs here to differentiate the shell commands from the output. | ||
|
|
||
| ```bash | ||
| $ python3 -m venv venv | ||
| $ source venv/bin/activate | ||
| $ pip install -r requirements.txt | ||
| $ sri-check https://www.4armed.com/ | ||
| <script src="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.js"></script> | ||
| <link href="https://use.typekit.net/vlp2azz.css" rel="stylesheet"/> | ||
| <link href="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.css" rel="stylesheet"/> | ||
| $ echo $? | ||
| 99 | ||
| ``` | ||
|
|
||
| ## Usage | ||
| ### Allowlisting | ||
|
|
||
| ### Check | ||
| Allowlisting is a way to tell sri-check that this _host_ is ok. By default sri-check adds the target host to the allow list, as well as some common third-party analytics domains which do not use versioning (Google, Hubspot, etc). | ||
|
|
||
| Now you can run the script which in its most basic form takes one argument, the URL you wish to check. | ||
| In the example above, the 4ARMED website is using typekit CSS for its fonts. Since this isn't a versioned resource if we generate an integrity hash for it and implement SRI, it could legitimately change and break the fonts. We'd either have to monitor this and update the hash or decide to accept this. | ||
|
|
||
| To add this host to the allow list we use the `--ignore` or `-i` flag and sri-check won't report it. | ||
|
|
||
| ```bash | ||
| sri-check -i use.typekit.net https://www.4armed.com/ | ||
| <script src="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.js"></script> | ||
| <link href="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.css" rel="stylesheet"/> | ||
| ``` | ||
|
|
||
| That's a bit cleaner. But what if we want to ignore multiple hosts? You can specify multiple `-i` flags. | ||
|
|
||
| ```bash | ||
| sri-check -i use.typekit.net -i cdn.jsdelivr.net https://www.4armed.com/ | ||
| [*] No resource tags found without integrity attribute | ||
| ``` | ||
|
|
||
| In this case, there are no SRI concerns let to report so sri-check tells you. If you don't want output if there's nothing to worry about you can use the `-q` flag to tell sri-check to be quiet. | ||
|
|
||
| ```bash | ||
| sri-check -i use.typekit.net -i cdn.jsdelivr.net https://www.4armed.com/ -q | ||
| ``` | ||
|
|
||
| Let's have a look at another example now, the BBC News site. | ||
|
|
||
| ```bash | ||
| sri-check https://www.bbc.co.uk/news/ | ||
| <script nomodule="" src="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/js/polyfills.js" type="text/javascript"></script> | ||
| <script src="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/js/require.min.js"></script> | ||
| <script src="https://static.files.bbci.co.uk/cookies/1098fb404f038cacf92f0ee250c025a0/cookie-banner/cookie-library.bundle.js"></script> | ||
| <script async="" src="https://mybbc-analytics.files.bbci.co.uk/reverb-client-js/reverb-3.8.0.js" type="text/javascript"></script> | ||
| <script defer="" src="https://m.files.bbci.co.uk/modules/bbc-morph-news-breaking-news-banner/2.1.2/breakingNewsBanner.js"></script> | ||
| <script src="https://m.files.bbci.co.uk/modules/bbc-morph-news-local-slice/2.10.6/xss.min.js"></script> | ||
| <script class="js-asset-path" data-asset-path="//m.files.bbci.co.uk/modules/bbc-morph-news-local-slice/2.10.6/" data-slice-path="/news/local_news_slice/" defer="" src="https://m.files.bbci.co.uk/modules/bbc-morph-news-local-slice/2.10.6/main.min.js"></script> | ||
| <script async="" src="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/js/redirect.js" type="text/javascript"></script> | ||
| <script async="" data-release="3.0.1-208.5a547360" data-ux="v5" src="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/js/performance.js" type="text/javascript"></script> | ||
| <script src="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/js/more-drawer.mjs" type="module"></script> | ||
| <script async="" src="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/js/orbit.mjs" type="module"></script> | ||
| <script async="" nomodule="" src="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/js/orbit.js" type="text/javascript"></script> | ||
| <script async="" data-base="https://navpromo.api.bbci.co.uk" data-variant="default" src="https://nav.files.bbci.co.uk/navpromo/f67a9538931d75e28d1807a3daf6dc00/js/footerpromo.js" type="text/javascript"></script> | ||
| <script async="" src="https://static.files.bbci.co.uk/cookies/1098fb404f038cacf92f0ee250c025a0/cookie-banner/cookie-banners.bundle.js"></script> | ||
| <script async="" src="https://nav.files.bbci.co.uk//user-activity-helper/a029a4d9f7f005262e2e6043630347b3/js/detectview.bundle.js"></script> | ||
| <script defer="defer" src="https://mybbc.files.bbci.co.uk/notification-ui/5.0.25//js/NotificationsMain.js"></script> | ||
| <script async="" src="https://m.files.bbci.co.uk/modules/bbc-morph-news-front-page-js-bundle/1.56.4/newsFrontPagePersonalised.js"></script> | ||
| <link crossorigin="" href="https://static.bbc.co.uk" rel="preconnect"/> | ||
| <link crossorigin="" href="https://m.files.bbci.co.uk" rel="preconnect"/> | ||
| <link crossorigin="" href="https://nav.files.bbci.co.uk" rel="preconnect"/> | ||
| <link crossorigin="" href="https://ichef.bbci.co.uk" rel="preconnect"/> | ||
| <link href="https://mybbc.files.bbci.co.uk" rel="dns-prefetch"/> | ||
| <link href="https://ssl.bbc.co.uk/" rel="dns-prefetch"/> | ||
| <link href="https://sa.bbc.co.uk/" rel="dns-prefetch"/> | ||
| <link href="https://ichef.bbci.co.uk" rel="dns-prefetch"/> | ||
| <link as="style" href="https://m.files.bbci.co.uk/modules/bbc-morph-news-page-styles/2.4.25/enhanced.css" rel="preload"/> | ||
| <link href="https://www.bbc.com/news" hreflang="en" rel="alternate"/> | ||
| <link href="https://m.files.bbci.co.uk/modules/bbc-morph-news-waf-page-meta/5.3.0/apple-touch-icon-57x57-precomposed.png" rel="apple-touch-icon-precomposed" sizes="57x57"/> | ||
| <link href="https://m.files.bbci.co.uk/modules/bbc-morph-news-waf-page-meta/5.3.0/apple-touch-icon-72x72-precomposed.png" rel="apple-touch-icon-precomposed" sizes="72x72"/> | ||
| <link href="https://m.files.bbci.co.uk/modules/bbc-morph-news-waf-page-meta/5.3.0/apple-touch-icon-114x114-precomposed.png" rel="apple-touch-icon-precomposed" sizes="114x114"/> | ||
| <link href="https://m.files.bbci.co.uk/modules/bbc-morph-news-waf-page-meta/5.3.0/apple-touch-icon.png" rel="apple-touch-icon-precomposed" sizes="144x144"/> | ||
| <link href="https://m.files.bbci.co.uk/modules/bbc-morph-news-waf-page-meta/5.3.0/apple-touch-icon.png" rel="apple-touch-icon"/> | ||
| <link href="https://static.files.bbci.co.uk/orbit/33bd882d2f5b902e64a28e50d337afa4/css/orbit-v5-ltr.min.css" rel="stylesheet"/> | ||
| <link as="font" crossorigin="" href="https://static.files.bbci.co.uk/fonts/reith/2.512/BBCReithSans_W_Rg.woff2" rel="preload" type="font/woff2"/> | ||
| <link as="font" crossorigin="" href="https://static.files.bbci.co.uk/fonts/reith/2.512/BBCReithSans_W_Bd.woff2" rel="preload" type="font/woff2"/> | ||
| <link href="https://nav.files.bbci.co.uk/searchbox/521fdb102453edfd515ee5fca2a40eda/css/box.css" rel="stylesheet"/> | ||
| <link href="https://static.files.bbci.co.uk/account/id-cta/621/style/id-cta.css" rel="stylesheet"/> | ||
| <link href="https://m.files.bbci.co.uk/modules/bbc-morph-news-page-styles/2.4.25/core.css" rel="stylesheet"/> | ||
| ``` | ||
|
|
||
| In the following examples I will assume you have installed via PyPi and have the script in your PATH. If not, use `./sricheck/sricheck.py` instead of `sri-check`. | ||
| Woah, they've got a lot of resources without SRI. Although.... on closer inspection, a lot of those URLs seem to be their own hosts, so they're probably not too worried about putting SRI on them. So we can sri-check to ignore these hosts but there's quite a few. `nav.files.bbci.co.uk`, `sa.bbc.co.uk`, etc. To help with this, sri-check also supports the `--ignore-regex` or `-I` flag. | ||
|
|
||
| ```bash | ||
| $ sri-check https://kubernetes.io/ | ||
| <script async="" src="https://www.googletagmanager.com/gtag/js?id=UA-36037335-10"></script> | ||
| <script async="" src="https://www.google-analytics.com/analytics.js"></script> | ||
| <link href="https://kubernetes.io/zh/" hreflang="zh" rel="alternate"/> | ||
| <link href="https://kubernetes.io/ko/" hreflang="ko" rel="alternate"/> | ||
| <link href="https://kubernetes.io/ja/" hreflang="ja" rel="alternate"/> | ||
| <link href="https://kubernetes.io/fr/" hreflang="fr" rel="alternate"/> | ||
| <link href="https://kubernetes.io/it/" hreflang="it" rel="alternate"/> | ||
| <link href="https://kubernetes.io/de/" hreflang="de" rel="alternate"/> | ||
| <link href="https://kubernetes.io/es/" hreflang="es" rel="alternate"/> | ||
| <link href="https://kubernetes.io/pt/" hreflang="pt" rel="alternate"/> | ||
| <link href="https://kubernetes.io/id/" hreflang="id" rel="alternate"/> | ||
| <link href="https://kubernetes.io/vi/" hreflang="vi" rel="alternate"/> | ||
| <link href="https://kubernetes.io/ru/" hreflang="ru" rel="alternate"/> | ||
| <link href="https://kubernetes.io/pl/" hreflang="pl" rel="alternate"/> | ||
| <link href="https://kubernetes.io/uk/" hreflang="uk" rel="alternate"/> | ||
| <link href="https://kubernetes.io/feed.xml" rel="alternate" type="application/rss+xml"/> | ||
| <link href="https://cdn-images.mailchimp.com/embedcode/horizontal-slim-10_7.css" rel="stylesheet" type="text/css"/> | ||
| sri-check -I '.*\.bbci\.co\.uk' https://www.bbc.co.uk/news/ | ||
| <link crossorigin="" href="https://static.bbc.co.uk" rel="preconnect"/> | ||
| <link href="https://ssl.bbc.co.uk/" rel="dns-prefetch"/> | ||
| <link href="https://sa.bbc.co.uk/" rel="dns-prefetch"/> | ||
| <link href="https://www.bbc.com/news" hreflang="en" rel="alternate"/> | ||
| ``` | ||
|
|
||
| > Sidenote: Don't worry about tags which aren't versioned, like the analytics ones above. You will spin your wheels trying to track changes and update the SRI hash. | ||
| That's looking pretty good. The ones which are left are also BBC domains. We could tweak our regex or just add a couple more `-I` flags. | ||
|
|
||
| ```bash | ||
| sri-check -I '.*\.bbci\.co\.uk' -I '.*\.bbc\.co[m|\.uk]' https://www.bbc.co.uk/news/ | ||
| [*] No resource tags found without integrity attribute | ||
| ``` | ||
|
|
||
| Nice. | ||
|
|
||
| ### Headers | ||
|
|
||
| If the page you are checking requires some kind of authorisation, you can specify HTTP request headers using the `--header` or `-H` flag. | ||
|
|
||
| ```bash | ||
| $ sri-check -H "Authorization: Bearer mytoken" -H "More: Headers" https://kubernetes.io/ | ||
| $ sri-check -H "Authorization: Bearer mytoken" -H "More: Headers" https://www.4armed.com/ | ||
| ``` | ||
|
|
||
| In case you are working with SPA application, you can use headless browser to render page before running check by setting `--browser` or `-b` flag. | ||
| ### Headless Browser | ||
|
|
||
| By default, sri-check uses the Python requests library to fetch the remote resources. In case you are working with SPA application where external resources may be dynamically written into the DOM and therefore not be visible in the source HTML, you can use a headless Chrome browser to render page before running check by setting `--browser` or `-b` flag. | ||
|
|
||
| ### Generate | ||
| ### Generating Hashes | ||
|
|
||
| You can alternatively output updated script tags with the SRI hash calculated. You can do this by specifying the `--generate` flag. | ||
| If you decide you want to fix these resources and implement SRI, sri-check can output the required tags for you, computing a SHA384 integrity hash for each resource. Use the `--generate` or -`-g` flag. | ||
|
|
||
| ```bash | ||
| $ sri-check --generate https://kubernetes.io/ | ||
| <script async="" crossorigin="anonymous" integrity="sha384-ITXXO4YR2TnoUD5vtyrbRhklkt/Q9hFZILlBvnQfLCe4dFJzAofF5sflCksWtNRa" src="https://www.googletagmanager.com/gtag/js?id=UA-36037335-10"></script> | ||
| <script async="" crossorigin="anonymous" integrity="sha384-+Mufq/JopHTZLYFkTpT8TA9y0zY/A/VflmpyMsbjcGbcyUbfudhn5Et/w6EIFhoo" src="https://www.google-analytics.com/analytics.js"></script> | ||
| sri-check -i use.typekit.net https://www.4armed.com/ -g | ||
| <script crossorigin="anonymous" integrity="sha384-uZ9tjGJYBp5yJzyGlqrVDWn2thY23HUzI9DhkhNhYa+6xmG2kvy3S28S5r4yE7cN" src="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.js"></script> | ||
| <link crossorigin="anonymous" href="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.css" integrity="sha384-uh3y/WeAusiZJ76/oewz8WiTiOl0SIQXYOTNT42NZ65y/Hyo8j3qGw7dnVAmSSDn" rel="stylesheet"/> | ||
| ``` | ||
|
|
||
| ### STDIN | ||
|
|
||
| sri-check also supports reading input from STDIN, like all good CLI tools. :-) Use the `-s` or `--stdin` flag to do this. Note that you still need to specify a URL otherwise sri-check doesn't know what site you are auditing and can't add it to the allowlist. It means you could pipe the output of curl through it, for example. | ||
|
|
||
| ```bash | ||
| curl -s https://www.4armed.com/ | sri-check -s https://www.4armed.com/ | ||
| <script src="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.js"></script> | ||
| <link href="https://use.typekit.net/vlp2azz.css" rel="stylesheet"/> | ||
| <link href="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.css" rel="stylesheet"/> | ||
| ``` | ||
|
|
||
| Or you might have some HTML you've saved from somewhere into a file. | ||
|
|
||
| ```bash | ||
| cat /tmp/4armed.html | sri-check -s https://www.4armed.com | ||
| <script src="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.js"></script> | ||
| <link href="https://use.typekit.net/vlp2azz.css" rel="stylesheet"/> | ||
| <link href="https://cdn.jsdelivr.net/npm/[email protected]/swiper-bundle.min.css" rel="stylesheet"/> | ||
| ``` | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| beautifulsoup4>=4.0 | ||
| lxml>=4.8 | ||
| requests>=2.0 | ||
| selenium>=4.4 | ||
| selenium>=4.8 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| __version__ = "1.6.2" | ||
| __version__ = "1.6.4" |
Oops, something went wrong.