Misc Tips, Recipes, and Observations

[thorwhalen]

To collect miscellaneous tips, recipes, and observations.

[thorwhalen]

GrazeReturningFilepaths

GrazeReturningFilepaths will do what Graze does, but return the local file path.

This can be particularly useful when you don't want to keep the bytes of some resources in memory.
For example, when downloading a zip (say from a dropbox url of a folder), do you really want to be holding all the contents of the folder in memory, when you might just be working with a few files?

from graze import Graze, GrazeReturningFilepaths

rootdir = '~/.config/imbed/graze'

g = Graze(rootdir)
gg = GrazeReturningFilepaths(rootdir)
url = 'https://drive.google.com/file/d/1g3K-wlixFxklTSUQNZKpEgN4WNTFTPIZ/view?usp=share_link'
# Note, contents of url are 7.55GB of zip file bytes (7.55GB is the compressed!)

[thorwhalen]

google drive downloads

Downloading from things like dropbox, google drive, etc. often needs some extra work over a simple request on the url.
I saw here that someone probably had a similar problem with google drive that I had, so I asked chatGPT to translate this to python.
Haven't verified or tried the code yet:

def gdrive_download(file_id, destination):
    """
    Download a file from Google Drive.

    Parameters:
    - file_id: str, Google Drive file ID
    - destination: str, local path to save the downloaded file
    """
    import requests

    # URL template for Google Drive
    base_url = "https://docs.google.com/uc?export=download"

    with requests.Session() as session:
        # Step 1: Initial request to get the confirmation token
        response = session.get(base_url, params={'id': file_id}, stream=True)
        for key, value in response.cookies.items():
            if key.startswith('download_warning'):
                # Step 2: Extract confirmation token
                params = {'id': file_id, 'confirm': value}
                break
        else:
            # No confirmation token, means direct download
            params = {'id': file_id}

        # Step 3: Download the file with the confirmation token
        response = session.get(base_url, params=params, stream=True)

        # Step 4: Save the file to the specified location
        with open(destination, 'wb') as f:
            for chunk in response.iter_content(chunk_size=32768):
                if chunk:
                    f.write(chunk)