Updates to fuzzing hw

hws/hw-fuzzing-tabbed.html

@@ -61,7 +61,10 @@ <h4 id='background'>Background</h4>
 <li>You should have a recent version of Python, and you will need to be able to install packages, via pip</li>
 </ul>
 </div><div id='tchangelog' class='tabcontent'><h3 id='changelog'>Changelog</h3>
-<p>Any changes to this page will be put here for easy reference. Typo fixes and minor clarifications are not listed here. So far there aren’t any significant changes to report.</p>
+<p>Any changes to this page will be put here for easy reference. Typo fixes and minor clarifications are not listed here. </p>
+<ul>
+<li>Sat, Oct 14: improved the explanations in the advanced fuzzing section; no changes made to the requirements.</li>
+</ul>
 </div><div id='tsetup' class='tabcontent'><h3 id='setup'>Setup</h3>
 <p>This assignment must be done in a recent version of Python. The only additional package you will need is <code>uvicorn</code> – to install it, enter <code>pip install uvicorn</code> (or <code>pip3 install uvicorn</code>).</p>
 <p>You will need a number of files from this repository to work on this assignment:</p>
@@ -90,30 +93,31 @@ <h4 id='output'>Output</h4>
 <p>We are going to use this particular call the visible test when you submit your assignment (although we will use a smaller word list to save time, but it will contain both <code>class</code> and <code>development.log</code>). Other hidden tests will be used to grade your assignment.</p>
 <p><strong>DO NOT HAVE ANY OTHER OUTPUT!</strong> We are going to test it by doing a file comparison, so if you have any other output it will report as not the same, and you will fail that test. Again, the order of the lines in your output does not matter.</p>
 </div><div id='ttask-2-advanced-fuzzing' class='tabcontent'><h3 id='task-2-advanced-fuzzing'>Task 2: Advanced fuzzing</h3>
-<p>There are a number of command-line parameters that the fuzzer.py file will accept. You have to implement usage of the others. You can find the via <code>python3 fuzzer.py -h</code>. The remaining ones to implement are as follows. Note that these are already parsed for you; you just have to handle when those values are in the <code>args</code> parameter to the <code>fuzz()</code> function.</p>
+<p>There are a number of command-line parameters that the fuzzer.py file will accept. You have to implement usage of the others. You can find them all via <code>python3 fuzzer.py -h</code>. The remaining ones to implement are as follows. Note that these are already parsed for you; you just have to handle when those values are in the <code>args</code> parameter to the <code>fuzz()</code> function.</p>
 <ul>
-<li><code>-e EXTENSIONS</code> or <code>--extension EXTENSIONS</code>: One or more extensions to append (e.g. php, html, etc.). Multiple extensions may be provided. So if <code>-e php</code> and <code>-e html</code> is provided, and the wordlist contains <code>hello</code> and <code>world</code>, then you should be replacing <code>FUZZ</code> with six different values: <code>hello</code>, <code>hello.php</code>, <code>hello.html</code>, <code>world</code>, <code>world.php</code>, and <code>world.html</code>.
+<li><code>-e EXTENSIONS</code> or <code>--extension EXTENSIONS</code>: One or more extensions to append (e.g. php, html, etc.). Multiple extensions may be provided. So if <code>-e php -e html</code> is provided, and the wordlist contains <code>hello</code> and <code>world</code>, then you should be replacing <code>FUZZ</code> with six different values: <code>hello</code>, <code>hello.php</code>, <code>hello.html</code>, <code>world</code>, <code>world.php</code>, and <code>world.html</code>.
 <ul>
-<li>The value to the <code>-e</code> parameter assumes htat it will be prefixed with a period before being added to each word in the word list. So <code>-e html</code> means you will add <code>.html</code> to each word in the word list. However, note that the command line parameter inserts that period for you.</li>
+<li>The value to the <code>-e</code> parameter assumes that it will be prefixed with a period before being added to each word in the word list. So <code>-e html</code> means you will add <code>.html</code> to each word in the word list. However, note that the command line parameter inserts that period for you.</li>
 <li>Note that adding any number of extensions still means you try the base word as well. So adding <code>-e html</code> means your program will try <em>both</em> <code>alert</code> and <code>alert.html</code>.</li>
 </ul></li>
 <li><code>-X METHOD</code> or <code>--method METHOD</code>: HTTP method to use (GET, POST, or PUT) (default: GET)
 <ul>
-<li>Your code should allow both upper-case and lower-case values</li>
-<li>You can also use the <code>urllib.request.Request</code> class to set the method; to check the method is received correctly, print out the <code>scope</code> variable in <code>server.py</code></li>
+<li>Your code should allow both upper-case and lower-case values (“get” and “GET”)</li>
+<li>You can also use the <a href='https://docs.python.org/3/library/urllib.request.html#urllib.request.Request'>urllib.request.Request</a> class to set the method; you just pass that object into <a href='https://docs.python.org/3/library/urllib.request.html#urllib.request.urlopen'>urllib.request.urlopen()</a>. To check the method is received correctly, update <code>server.py</code> to print out the <code>scope</code> variable.</li>
 </ul></li>
-<li><code>-H HEADERS</code> or <code>--header HEADERS</code>: One or more HTTP headers to add to requests, in the form “HeaderName: HeaderValue” (e.g. “Content-Type: application/json” or “Host: FUZZ.example.com”). May be specified one or more times.
+<li><code>-H HEADERS</code> or <code>--header HEADERS</code>: One or more HTTP headers to add to requests, in the form “HeaderName:HeaderValue” (e.g. “Content-Type:application/json” or “Host:FUZZ.example.com”). This may be specified one or more times.
 <ul>
-<li>You can also use the <code>urllib.request.Request</code> class to set the headers; to check the header is received correctly, print out the <code>scope</code> variable in <code>server.py</code></li>
+<li>You can again use <a href='https://docs.python.org/3/library/urllib.request.html#urllib.request.Request'>urllib.request.Request</a> to set the headers; to check the header is received correctly, print out the <code>scope</code> variable in <code>server.py</code>. As before, you then past the Request object into <code>urllib.request.urlopen</code>.</li>
 <li>Example usage: adding <code>-H "MyHeader:foobarbaz"</code> will cause each request sent to the URL to include that header; note that there is no space after the colon</li>
-<li>Note that, in <code>fuzz()</code>, the headers are received as a string with a colon (<code>:</code>) separating the key and value. You have to <code>split()</code> that, as what is passed to the <code>urllib.request.Request</code> constructor is a dictionary of key-value pairs.</li>
+<li>Note that, in <code>fuzz()</code>, the headers are received as a string with a colon (<code>:</code>) separating the key and value. You have to <code>split()</code> that, as what needs to be passed to the <code>urllib.request.Request</code> constructor is a dictionary of key-value pairs.</li>
 </ul></li>
 <li><code>-d DATA</code> or <code>--data DATA</code>: Data to send in the body of the HTTP request.
 <ul>
-<li>To see how to read it from uvicorn, look <a href='https://www.uvicorn.org/'>here</a> – specifically, look at the <code>read_body()</code> function, which is called (in the <code>app()</code> function in server.py) as <code>body = await read_body(receive)</code>.</li>
-<li>You can also use the <code>urllib.request.Request</code> class to set the data; to check the data is received correctly, print out the <code>scope</code> variable in <code>server.py</code></li>
+<li>To see how to read it from uvicorn, look <a href='https://www.uvicorn.org/'>here</a> – specifically, look at the <code>read_body()</code> function, which is called (in the <code>app()</code> function in server.py) as <code>body = await read_body(receive)</code>. You can cut-and-paste that function right into server.py if you want to use it.</li>
+<li>You can also use the <code>urllib.request.Request</code> class to set the data; to check the data is received correctly, print out the <code>scope</code> variable in <code>server.py</code>. As before, you then past the Request object into <code>urllib.request.urlopen</code>.</li>
+<li>Warning: the data passed in to the <code>Request</code> object must be <code>bytes</code>, not a string or <code>None</code>.</li>
 </ul></li>
-<li><code>-mc MATCH_CODES</code>: Match HTTP response codes. May be specified multiple times. If let unspecified, defaults to the following response codes: [200, 301, 302, 401, 403]. Previously you printed out any URLs that did not return 404 (not found). That should now be modified to print out the URLs that return one of the escape codes in this list (which is parsed for you and passed into the <code>fuzz()</code> function).
+<li><code>-mc MATCH_CODES</code>: Match <a href='https://en.wikipedia.org/wiki/List_of_HTTP_status_codes'>HTTP response codes</a>. May be specified multiple times. If left unspecified, defaults to the following response codes: [200, 301, 302, 401, 403]. Previously you printed out any URLs that did not return 404 (not found). That should now be modified to print out the URLs that return one of the response codes in this list (which is parsed for you and passed into the <code>fuzz()</code> function in the <code>args</code> parameter).
 <ul>
 <li>Specifying just one response code via <code>-mc</code> will replace the default list with just that one. So <code>-mc 200</code> will not check for any of the defaults other than 200. Note that the command line argument parsing does this for you.</li>
 <li><strong>NOTE:</strong> this can be specified <em>multiple</em> times, at which point you would then have to check for <em>multiple</em> match codes.</li>

hws/hw-fuzzing.html

@@ -55,7 +55,10 @@ <h4 id="background">Background</h4>
 <li>You should have a recent version of Python, and you will need to be able to install packages, via pip</li>
 </ul>
 <h3 id="changelog">Changelog</h3>
-<p>Any changes to this page will be put here for easy reference. Typo fixes and minor clarifications are not listed here. So far there aren’t any significant changes to report.</p>
+<p>Any changes to this page will be put here for easy reference. Typo fixes and minor clarifications are not listed here. <!-- So far there aren't any significant changes to report. --></p>
+<ul>
+<li>Sat, Oct 14: improved the explanations in the advanced fuzzing section; no changes made to the requirements.</li>
+</ul>
 <h3 id="setup">Setup</h3>
 <p>This assignment must be done in a recent version of Python. The only additional package you will need is <code>uvicorn</code> – to install it, enter <code>pip install uvicorn</code> (or <code>pip3 install uvicorn</code>).</p>
 <p>You will need a number of files from this repository to work on this assignment:</p>
@@ -84,30 +87,31 @@ <h4 id="output">Output</h4>
 <p>We are going to use this particular call the visible test when you submit your assignment (although we will use a smaller word list to save time, but it will contain both <code>class</code> and <code>development.log</code>). Other hidden tests will be used to grade your assignment.</p>
 <p><strong>DO NOT HAVE ANY OTHER OUTPUT!</strong> We are going to test it by doing a file comparison, so if you have any other output it will report as not the same, and you will fail that test. Again, the order of the lines in your output does not matter.</p>
 <h3 id="task-2-advanced-fuzzing">Task 2: Advanced fuzzing</h3>
-<p>There are a number of command-line parameters that the fuzzer.py file will accept. You have to implement usage of the others. You can find the via <code>python3 fuzzer.py -h</code>. The remaining ones to implement are as follows. Note that these are already parsed for you; you just have to handle when those values are in the <code>args</code> parameter to the <code>fuzz()</code> function.</p>
+<p>There are a number of command-line parameters that the fuzzer.py file will accept. You have to implement usage of the others. You can find them all via <code>python3 fuzzer.py -h</code>. The remaining ones to implement are as follows. Note that these are already parsed for you; you just have to handle when those values are in the <code>args</code> parameter to the <code>fuzz()</code> function.</p>
 <ul>
-<li><code>-e EXTENSIONS</code> or <code>--extension EXTENSIONS</code>: One or more extensions to append (e.g. php, html, etc.). Multiple extensions may be provided. So if <code>-e php</code> and <code>-e html</code> is provided, and the wordlist contains <code>hello</code> and <code>world</code>, then you should be replacing <code>FUZZ</code> with six different values: <code>hello</code>, <code>hello.php</code>, <code>hello.html</code>, <code>world</code>, <code>world.php</code>, and <code>world.html</code>.
+<li><code>-e EXTENSIONS</code> or <code>--extension EXTENSIONS</code>: One or more extensions to append (e.g. php, html, etc.). Multiple extensions may be provided. So if <code>-e php -e html</code> is provided, and the wordlist contains <code>hello</code> and <code>world</code>, then you should be replacing <code>FUZZ</code> with six different values: <code>hello</code>, <code>hello.php</code>, <code>hello.html</code>, <code>world</code>, <code>world.php</code>, and <code>world.html</code>.
 <ul>
-<li>The value to the <code>-e</code> parameter assumes htat it will be prefixed with a period before being added to each word in the word list. So <code>-e html</code> means you will add <code>.html</code> to each word in the word list. However, note that the command line parameter inserts that period for you.</li>
+<li>The value to the <code>-e</code> parameter assumes that it will be prefixed with a period before being added to each word in the word list. So <code>-e html</code> means you will add <code>.html</code> to each word in the word list. However, note that the command line parameter inserts that period for you.</li>
 <li>Note that adding any number of extensions still means you try the base word as well. So adding <code>-e html</code> means your program will try <em>both</em> <code>alert</code> and <code>alert.html</code>.</li>
 </ul></li>
 <li><code>-X METHOD</code> or <code>--method METHOD</code>: HTTP method to use (GET, POST, or PUT) (default: GET)
 <ul>
-<li>Your code should allow both upper-case and lower-case values</li>
-<li>You can also use the <code>urllib.request.Request</code> class to set the method; to check the method is received correctly, print out the <code>scope</code> variable in <code>server.py</code></li>
+<li>Your code should allow both upper-case and lower-case values (“get” and “GET”)</li>
+<li>You can also use the <a href="https://docs.python.org/3/library/urllib.request.html#urllib.request.Request">urllib.request.Request</a> class to set the method; you just pass that object into <a href="https://docs.python.org/3/library/urllib.request.html#urllib.request.urlopen">urllib.request.urlopen()</a>. To check the method is received correctly, update <code>server.py</code> to print out the <code>scope</code> variable.</li>
 </ul></li>
-<li><code>-H HEADERS</code> or <code>--header HEADERS</code>: One or more HTTP headers to add to requests, in the form “HeaderName: HeaderValue” (e.g. “Content-Type: application/json” or “Host: FUZZ.example.com”). May be specified one or more times.
+<li><code>-H HEADERS</code> or <code>--header HEADERS</code>: One or more HTTP headers to add to requests, in the form “HeaderName:HeaderValue” (e.g. “Content-Type:application/json” or “Host:FUZZ.example.com”). This may be specified one or more times.
 <ul>
-<li>You can also use the <code>urllib.request.Request</code> class to set the headers; to check the header is received correctly, print out the <code>scope</code> variable in <code>server.py</code></li>
+<li>You can again use <a href="https://docs.python.org/3/library/urllib.request.html#urllib.request.Request">urllib.request.Request</a> to set the headers; to check the header is received correctly, print out the <code>scope</code> variable in <code>server.py</code>. As before, you then past the Request object into <code>urllib.request.urlopen</code>.</li>
 <li>Example usage: adding <code>-H "MyHeader:foobarbaz"</code> will cause each request sent to the URL to include that header; note that there is no space after the colon</li>
-<li>Note that, in <code>fuzz()</code>, the headers are received as a string with a colon (<code>:</code>) separating the key and value. You have to <code>split()</code> that, as what is passed to the <code>urllib.request.Request</code> constructor is a dictionary of key-value pairs.</li>
+<li>Note that, in <code>fuzz()</code>, the headers are received as a string with a colon (<code>:</code>) separating the key and value. You have to <code>split()</code> that, as what needs to be passed to the <code>urllib.request.Request</code> constructor is a dictionary of key-value pairs.</li>
 </ul></li>
 <li><code>-d DATA</code> or <code>--data DATA</code>: Data to send in the body of the HTTP request.
 <ul>
-<li>To see how to read it from uvicorn, look <a href="https://www.uvicorn.org/">here</a> – specifically, look at the <code>read_body()</code> function, which is called (in the <code>app()</code> function in server.py) as <code>body = await read_body(receive)</code>.</li>
-<li>You can also use the <code>urllib.request.Request</code> class to set the data; to check the data is received correctly, print out the <code>scope</code> variable in <code>server.py</code></li>
+<li>To see how to read it from uvicorn, look <a href="https://www.uvicorn.org/">here</a> – specifically, look at the <code>read_body()</code> function, which is called (in the <code>app()</code> function in server.py) as <code>body = await read_body(receive)</code>. You can cut-and-paste that function right into server.py if you want to use it.</li>
+<li>You can also use the <code>urllib.request.Request</code> class to set the data; to check the data is received correctly, print out the <code>scope</code> variable in <code>server.py</code>. As before, you then past the Request object into <code>urllib.request.urlopen</code>.</li>
+<li>Warning: the data passed in to the <code>Request</code> object must be <code>bytes</code>, not a string or <code>None</code>.</li>
 </ul></li>
-<li><code>-mc MATCH_CODES</code>: Match HTTP response codes. May be specified multiple times. If let unspecified, defaults to the following response codes: [200, 301, 302, 401, 403]. Previously you printed out any URLs that did not return 404 (not found). That should now be modified to print out the URLs that return one of the escape codes in this list (which is parsed for you and passed into the <code>fuzz()</code> function).
+<li><code>-mc MATCH_CODES</code>: Match <a href="https://en.wikipedia.org/wiki/List_of_HTTP_status_codes">HTTP response codes</a>. May be specified multiple times. If left unspecified, defaults to the following response codes: [200, 301, 302, 401, 403]. Previously you printed out any URLs that did not return 404 (not found). That should now be modified to print out the URLs that return one of the response codes in this list (which is parsed for you and passed into the <code>fuzz()</code> function in the <code>args</code> parameter).
 <ul>
 <li>Specifying just one response code via <code>-mc</code> will replace the default list with just that one. So <code>-mc 200</code> will not check for any of the defaults other than 200. Note that the command line argument parsing does this for you.</li>
 <li><strong>NOTE:</strong> this can be specified <em>multiple</em> times, at which point you would then have to check for <em>multiple</em> match codes.</li>