Code

Integrate scraping#15

Code

Merged

freddyheppell merged 24 commits intodevGateNLP/wpextract:devfrom feature/scrapeGateNLP/wpextract:feature/scrapeCopy head branch name to clipboardJul 4, 2024

+2,197-79

Commits

Commits on Jul 3, 2024

• incorporate wp-json-scraper module

  freddyheppell

  committed
  47662ab

  Copy full SHA for 47662ab

• ruff autofix

  freddyheppell

  committed
  87a5e77

  Copy full SHA for 87a5e77

• docconvert

  freddyheppell

  committed
  00ee4d0

  Copy full SHA for 00ee4d0

• manual fixes

  freddyheppell

  committed
  d81d2bc

  Copy full SHA for d81d2bc

• fix noqa comment

  freddyheppell

  committed
  25f3820

  Copy full SHA for 25f3820

• refactor cli to support subcommands

  freddyheppell

  committed
  2e47067

  Copy full SHA for 2e47067

• Basic integration of downloader

  freddyheppell

  committed
  e2e47a1

  Copy full SHA for e2e47a1

• swap progress bar implementation

  freddyheppell

  committed
  acc546e

  Copy full SHA for acc546e

• ruff

  freddyheppell

  committed
  7f2a38e

  Copy full SHA for 7f2a38e

• remove object display code

  freddyheppell

  committed
  cf95bd6

  Copy full SHA for cf95bd6

• add media dl support

  freddyheppell

  committed
  f7912cd

  Copy full SHA for f7912cd

• lint

  freddyheppell

  committed
  f347e73

  Copy full SHA for f347e73

• remove totally unused modules

  freddyheppell

  committed
  88f7ce2

  Copy full SHA for 88f7ce2

• remove unused csv export code

  freddyheppell

  committed
  e49c8de

  Copy full SHA for e49c8de

• ruff

  freddyheppell

  committed
  af19107

  Copy full SHA for af19107

• implement proxy/cookies/auth on download client

  freddyheppell

  committed
  efe2404

  Copy full SHA for efe2404

• improve subparser documentation

  freddyheppell

  committed
  8c4af22

  Copy full SHA for 8c4af22

• ruff

  freddyheppell

  committed
  3d49c6a

  Copy full SHA for 3d49c6a

• add NOTICE file

  freddyheppell

  committed
  58e0ea5

  Copy full SHA for 58e0ea5

• remove plugin list

  freddyheppell

  committed
  a4e9dd3

  Copy full SHA for a4e9dd3

Commits on Jul 4, 2024

• remove console util

  freddyheppell

  committed
  9c87318

  Copy full SHA for 9c87318

• dl utils cleanup

  freddyheppell

  committed
  5c03dfb

  Copy full SHA for 5c03dfb

• implement request robustness features

  freddyheppell

  committed
  dba7a46

  Copy full SHA for dba7a46

• make media dl use request session

  freddyheppell

  committed
  fc23047

  Copy full SHA for fc23047