From ff3d7c66924a416f1331cee25b630a6fcab0c106 Mon Sep 17 00:00:00 2001 From: ptmcg Date: Sun, 15 Sep 2024 15:51:31 -0500 Subject: [PATCH] Added notes on adding a User-Agent header when importing from a URL --- CHANGES | 3 +++ how_to_use_littletable.md | 14 ++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/CHANGES b/CHANGES index 74ac2cb..7481ef9 100644 --- a/CHANGES +++ b/CHANGES @@ -10,6 +10,9 @@ Version 3.0.1 sending the output to a file. The width can now be overridden by passing an integer `width` argument to `Table.present()`. +- Updated `how_to_use_littletable.md` with notes on adding a `User-Agent` header when + importing data from HTTP/HTTPS urls. + Version 3.0.0 ------------- diff --git a/how_to_use_littletable.md b/how_to_use_littletable.md index f85503d..61a060b 100644 --- a/how_to_use_littletable.md +++ b/how_to_use_littletable.md @@ -333,6 +333,20 @@ _(Note: when passing username and password credentials, HTTPS URLs are strongly exposure of credentials in unencrypted requests. `littletable` will emit a warning when importing with authentication using an HTTP URL.)_ +Some sites will use the "User-Agent" HTTP header to filter out unrestricted automated access to their site. You can add this header to your +import statement using code as in the following: + + user_agent_string = ( + "Mozilla/5.0" + " (Windows NT 10.0; Win64; x64)" + " AppleWebKit/537.36" + " (KHTML, like Gecko)" + " Chrome/128.0.0.0" + " Safari/537.36" + " Edg/128.0.0.0" + ) + table = littletable.csv_import(url, headers={"User-Agent": user_agent_string}) + Tabular output -------------- To produce a nice tabular output for a table, you can use the embedded support for