Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: jazzband/django-pipeline
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 2.0.7
Choose a base ref
...
head repository: jazzband/django-pipeline
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Dec 27, 2020

  1. Copy the full SHA
    406e014 View commit details

Commits on Nov 10, 2021

  1. Copy the full SHA
    fbab928 View commit details

Commits on Nov 23, 2021

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    c641b59 View commit details
  2. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    5dc1d14 View commit details

Commits on Nov 25, 2021

  1. Merge pull request #739 from gatsinski/master

    Add a note about removed cached storage in Django 3.1
    Buky authored Nov 25, 2021

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    616d804 View commit details

Commits on Jan 6, 2022

  1. Release Django-Pipeline 2.0.8

    Bump the package version and update the package HISTORY
    TheBuky authored and TheBuky committed Jan 6, 2022
    Copy the full SHA
    525dafa View commit details
  2. CoffeeScript on NPM has moved to "coffeescript"

    Update package.json due to the npm package coffee-script has moved coffeescript
    TheBuky authored and TheBuky committed Jan 6, 2022
    Copy the full SHA
    8628326 View commit details

Commits on Jan 10, 2022

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    9123d68 View commit details
  2. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    991bd91 View commit details
  3. Drop slimit support (#765)

    Remove slimit official package support due to not maintained package:
    no any release since 2013
    TheBuky authored and TheBuky committed Jan 10, 2022
    Copy the full SHA
    f961008 View commit details

Commits on Aug 6, 2022

  1. Improved packager.compile() to handle s3 storage when compiler genera…

    …tes output to the local one. This patch copies the generated file. (#502)
    thomasyip authored Aug 6, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    fafe97c View commit details
  2. docs: Fix a few typos (#769)

    There are small typos in:
    - CONTRIBUTING.rst
    - docs/using.rst
    
    Fixes:
    - Should read `recommend` rather than `reccomend`.
    - Should read `platform` rather than `plaform`.
    - Should read `dependencies` rather than `depencies`.
    
    Signed-off-by: Tim Gates <tim.gates@iress.com>
    timgates42 authored Aug 6, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    c85a257 View commit details

Commits on Jan 24, 2023

  1. Add allowlist_externals for npm.

    By default tox does not allow invoking external binaries, in order to
    prevent people from accidentally (or nefariously) deleting their entire
    system. The pipeline tests do need to run npm though, so this change
    adds that to the configuration.
    davidt committed Jan 24, 2023
    Copy the full SHA
    b5e35f2 View commit details
  2. Fix string encoding in compiler and compressor errors.

    The errors raised when a compiler or compressor failed were including
    the message as bytes instead of str. This meant that when printing the
    errors, or seeing them in the django-pipeline-error element that's
    rendered to the page, the error would be shown as the repr of the
    bytestring (which did not properly show newlines, and included extra b''
    junk). This change fixes things to decode to str when appropriate.
    davidt committed Jan 24, 2023
    Copy the full SHA
    69820b0 View commit details
  3. Update github test workflow.

    This updates the test workflow to fix several issues:
    
    - Python 3.6 is no longer available on the test hosts.
    - Node 12 is deprecated, and needs to be replaced with 16.
    - The name of the node step had a typo in it (Instal -> Install).
    davidt committed Jan 24, 2023
    Copy the full SHA
    d4ab608 View commit details
  4. Bump versions of github actions.

    This updates actions/checkout, actions/setup-python, actions/setup-node,
    and actions/cache to their most recent releases.
    davidt committed Jan 24, 2023
    Copy the full SHA
    bf67109 View commit details
  5. Copy the full SHA
    fa5aea3 View commit details
  6. Bump version of codecov action.

    This updates to codecov-action@v3.
    davidt committed Jan 24, 2023
    Copy the full SHA
    84350e4 View commit details
  7. Copy the full SHA
    54d7c15 View commit details
  8. Copy the full SHA
    93b55fe View commit details

Commits on Feb 9, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    f0b78a2 View commit details

Commits on Feb 10, 2023

  1. Fix a typo (#775)

    streeter authored Feb 10, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    8fada29 View commit details
  2. Add flake8 and isort to pre-commit (#777)

    * Add flake8 and isort to pre-commit
    
    * [pre-commit.ci] auto fixes from pre-commit.com hooks
    
    for more information, see https://pre-commit.ci
    
    ---------
    
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    peymanslh and pre-commit-ci[bot] authored Feb 10, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    a642568 View commit details
  3. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    52b8024 View commit details

Commits on Feb 17, 2023

  1. Copy the full SHA
    b638b7a View commit details
  2. Make svg file transparent

    peymanslh committed Feb 17, 2023
    Copy the full SHA
    24050e1 View commit details

Commits on Feb 20, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    455649d View commit details

Commits on Feb 21, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    b5aa4a9 View commit details
  2. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    413c921 View commit details

Commits on Feb 23, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    b4ba452 View commit details

Commits on Mar 1, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    54ccf68 View commit details

Commits on Mar 7, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    9af3eeb View commit details

Commits on Mar 9, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    3d828d2 View commit details

Commits on Mar 10, 2023

  1. Lint Python with Ruff (#790)

    cclauss authored Mar 10, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    275c4b5 View commit details

Commits on Apr 3, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    38deeb5 View commit details

Commits on Apr 5, 2023

  1. [pre-commit.ci] pre-commit autoupdate (#767)

    updates:
    - [github.com/pre-commit/pre-commit-hooks: v4.0.1 → v4.4.0](pre-commit/pre-commit-hooks@v4.0.1...v4.4.0)
    
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    pre-commit-ci[bot] authored Apr 5, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    ccfafcc View commit details

Commits on Apr 11, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    e459d87 View commit details

Commits on Nov 28, 2023

  1. Use Pypy 3.10, drop Python 3.7 and Django 2

    Upgrade pypy to 3.10
    
    Drop support for Python 3.7 and django 2,3.0,3.1
    
    Update history file
    peymanslh committed Nov 28, 2023
    Copy the full SHA
    bc2a5f5 View commit details
  2. Copy the full SHA
    bb7a4a9 View commit details

Commits on Dec 3, 2023

  1. Copy the full SHA
    3a0bb76 View commit details
  2. [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci
    pre-commit-ci[bot] authored and peymanslh committed Dec 3, 2023
    Copy the full SHA
    9aa8be1 View commit details
  3. Add noqa positions after applying black format

    peymanslh committed Dec 3, 2023
    Copy the full SHA
    0dd41a3 View commit details

Commits on Dec 13, 2023

  1. avoid the use of pkg_resources when importlib.metadata is available (#…

    …799)
    
    * feat: avoid the use of pkg_resources when importlib.metadata, since pkg_resources is removed from Python 3.12
    
    * feat: avoid the use of pkg_resources when importlib.metadata, since pkg_resources is removed from Python 3.12
    
    * [pre-commit.ci] auto fixes from pre-commit.com hooks
    
    for more information, see https://pre-commit.ci
    
    * fix: remove flake8 errors
    
    * [pre-commit.ci] auto fixes from pre-commit.com hooks
    
    for more information, see https://pre-commit.ci
    
    ---------
    
    Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
    d9pouces and pre-commit-ci[bot] authored Dec 13, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    c286019 View commit details

Commits on Dec 14, 2023

  1. Add Python 3.12 support (#802)

    * Add Python 3.12 support
    
    * Use shutil in tests after deprecating distutils in python 3.12
    peymanslh authored Dec 14, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    b9e6356 View commit details

Commits on Dec 21, 2023

  1. Add django 5 support (#803)

    * Add django 5 support
    
    * exclude Python 3.9 and django 5
    peymanslh authored Dec 21, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    7b4eea1 View commit details

Commits on Dec 27, 2023

  1. Update history and docs conf to release v3 (#804)

    peymanslh authored Dec 27, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    14b6d2c View commit details

Commits on Jan 15, 2024

  1. Replace deprecated .warn method with .warning (#807)

    - `logging.Logger.warn()` has been deprecated since Python 3.3 and
    will not be available in Python 3.13 as per this pull request
    (python/cpython#105377).
    
    Co-authored-by: Lewis M. Kabui <lewisemm@users.noreply.github.com>
    lewisemm and lewisemm authored Jan 15, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    2018c11 View commit details

Commits on Mar 19, 2024

  1. Update sourcemap paths when concatenating source files.

    When building a package from source files, the built source files get
    concatenated together before being post-processed by Django. Prior to
    Django 4.0, the post-processing step would normalize `url(...)` entries
    in CSS by looking it up in storage and replacing the path with the
    hashed version.
    
    Starting in Django 4.0, post-processing would do the same for
    sourcemaps. This can break when concatenating either CSS or JavaScript
    files, since Pipeline may produce a built package file that's in a
    different directory from one or more built source files. Django would
    fail to find the file and raise an error.
    
    We now include sourcemap normalization as part of the concatenation
    process. This is using a similar approach to `url(...)` normalization,
    but now consolidated into the `Compressor.concatenate()` function. This
    has been updated to take arguments controlling the concatenation
    process, such as a regex for capturing paths to normalize.
    
    The regex for capturing sourcemap lines is built to be spec-compliant,
    and is currently more broad than what Django looks for during
    post-processing. This will help avoid potential issues as Django makes
    changes to their process.
    
    The old functions (`concatenate_and_rewrite()`) and old default behavior
    has been left intact, but with runtime deprecation warnings, so that any
    code specializing Pipeline will continue to work. This helps ensure this
    change is API-compatible and non-breaking.
    
    See issue #808 for more details on the problem and the solution.
    chipx86 committed Mar 19, 2024
    Copy the full SHA
    fd1c033 View commit details
  2. [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci
    pre-commit-ci[bot] committed Mar 19, 2024
    Copy the full SHA
    58f9f99 View commit details

Commits on Apr 17, 2024

  1. Merge pull request #809 from chipx86/3.x/fix-sourcemap-paths

    Update sourcemap paths when concatenating source files.
    davidt authored Apr 17, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    193cc20 View commit details
Showing with 2,585 additions and 1,377 deletions.
  1. +6 −0 .flake8
  2. +6 −0 .github/dependabot.yml
  3. +8 −8 .github/workflows/release.yml
  4. +37 −12 .github/workflows/test.yml
  5. +1 −0 .gitignore
  6. +24 −0 .pre-commit-config.yaml
  7. +13 −0 .readthedocs.yaml
  8. +1 −0 AUTHORS
  9. +46 −0 CODE_OF_CONDUCT.md
  10. +1 −1 CONTRIBUTING.rst
  11. +60 −2 HISTORY.rst
  12. +107 −2 README.rst
  13. +26 −0 docs/compilers.rst
  14. +0 −15 docs/compressors.rst
  15. +61 −56 docs/conf.py
  16. +3 −0 docs/storages.rst
  17. +0 −8 docs/usage.rst
  18. +1 −1 docs/using.rst
  19. +118 −0 img/django-pipeline.svg
  20. +4 −3 package.json
  21. +27 −6 pipeline/__init__.py
  22. +16 −12 pipeline/collector.py
  23. +35 −21 pipeline/compilers/__init__.py
  24. +3 −3 pipeline/compilers/coffee.py
  25. +4 −4 pipeline/compilers/es6.py
  26. +6 −4 pipeline/compilers/less.py
  27. +3 −3 pipeline/compilers/livescript.py
  28. +4 −9 pipeline/compilers/sass.py
  29. +4 −8 pipeline/compilers/stylus.py
  30. +21 −0 pipeline/compilers/typescript.py
  31. +263 −81 pipeline/compressors/__init__.py
  32. +1 −1 pipeline/compressors/closure.py
  33. +3 −0 pipeline/compressors/csshtmljsminify.py
  34. +1 −1 pipeline/compressors/cssmin.py
  35. +3 −3 pipeline/compressors/csstidy.py
  36. +2 −0 pipeline/compressors/jsmin.py
  37. +0 −11 pipeline/compressors/slimit.py
  38. +2 −4 pipeline/compressors/terser.py
  39. +2 −2 pipeline/compressors/uglifyjs.py
  40. +4 −8 pipeline/compressors/yuglify.py
  41. +4 −8 pipeline/compressors/yui.py
  42. +60 −79 pipeline/conf.py
  43. +1 −3 pipeline/exceptions.py
  44. +45 −37 pipeline/finders.py
  45. +26 −27 pipeline/forms.py
  46. +5 −4 pipeline/glob.py
  47. +30 −26 pipeline/jinja2/__init__.py
  48. +0 −59 pipeline/manifest.py
  49. +8 −6 pipeline/middleware.py
  50. +68 −23 pipeline/packager.py
  51. +0 −1 pipeline/signals.py
  52. +28 −23 pipeline/storage.py
  53. +1 −1 pipeline/templates/pipeline/compile_error.html
  54. +71 −51 pipeline/templatetags/pipeline.py
  55. +3 −4 pipeline/utils.py
  56. +6 −5 pipeline/views.py
  57. +81 −0 pyproject.toml
  58. +0 −5 requirements.txt
  59. +0 −2 setup.cfg
  60. +0 −46 setup.py
  61. +6 −2 tests/assets/compilers/scss/expected.css
  62. +4 −0 tests/assets/compilers/typescript/expected.js
  63. +13 −0 tests/assets/compilers/typescript/input.ts
  64. +24 −0 tests/assets/css/sourcemap.css
  65. +18 −0 tests/assets/js/sourcemap.js
  66. +166 −156 tests/settings.py
  67. +12 −14 tests/tests/__init__.py
  68. +38 −24 tests/tests/test_collector.py
  69. +115 −73 tests/tests/test_compiler.py
  70. +462 −120 tests/tests/test_compressor.py
  71. +8 −8 tests/tests/test_conf.py
  72. +106 −100 tests/tests/test_forms.py
  73. +34 −33 tests/tests/test_glob.py
  74. +20 −15 tests/tests/test_middleware.py
  75. +21 −19 tests/tests/test_packager.py
  76. +38 −33 tests/tests/test_storage.py
  77. +94 −39 tests/tests/test_template.py
  78. +3 −5 tests/tests/test_utils.py
  79. +9 −9 tests/tests/test_views.py
  80. +4 −5 tests/urls.py
  81. +3 −4 tests/utils.py
  82. +23 −19 tox.ini
6 changes: 6 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[flake8]
max-line-length = 88
exclude =
.tox
node_modules
*env/
6 changes: 6 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
16 changes: 8 additions & 8 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -11,29 +11,29 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: '3.9'

- name: Install dependencies
run: |
python -m pip install -U pip
python -m pip install -U setuptools twine wheel
python -m pip install -U twine build setuptools-scm
- name: Build package
run: |
python setup.py --version
python setup.py sdist --format=gztar bdist_wheel
twine check dist/*
python -m setuptools_scm
python -m build
twine check --strict dist/*
- name: Upload packages to Jazzband
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@master
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: jazzband
password: ${{ secrets.JAZZBAND_RELEASE_KEY }}
49 changes: 37 additions & 12 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -9,36 +9,54 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.6', '3.7', '3.8', '3.9', 'pypy3']
django-version: ['2.2', '3.0', '3.1', '3.2', 'main']
python-version: ['3.9', '3.10', '3.11', '3.12', 'pypy-3.10']
django-version: ['4.1', '4.2', '5.0', '5.1', 'main']
exclude:
- python-version: '3.9'
django-version: '5.0'
- python-version: '3.9'
django-version: '5.1'
- python-version: '3.9'
django-version: 'main'
- python-version: 'pypy-3.10'
django-version: '4.1'
- python-version: 'pypy-3.10'
django-version: '4.2'
- python-version: 'pypy-3.10'
django-version: '5.0'
- python-version: 'pypy-3.10'
django-version: '5.1'
- python-version: 'pypy-3.10'
django-version: 'main'
- python-version: '3.12'
django-version: '4.1'

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Set up Node
uses: actions/setup-node@v1
uses: actions/setup-node@v4
with:
node-version: '12'
node-version: '16'

- name: Instal Node dependencies
- name: Install Node dependencies
run: npm install

- name: Get pip cache dir
id: pip-cache
run: |
echo "::set-output name=dir::$(pip cache dir)"
run: echo "dir=$(pip cache dir)" >> $GITHUB_OUTPUT

- name: Cache
uses: actions/cache@v2
uses: actions/cache@v4
with:
path: ${{ steps.pip-cache.outputs.dir }}
key:
${{ matrix.python-version }}-v1-${{ hashFiles('**/setup.py') }}-${{ hashFiles('**/tox.ini') }}
${{ matrix.python-version }}-v1-${{ hashFiles('**/pyproject.toml') }}-${{ hashFiles('**/tox.ini') }}
restore-keys: |
${{ matrix.python-version }}-v1-
@@ -54,6 +72,13 @@ jobs:
DJANGO: ${{ matrix.django-version }}

- name: Upload coverage
uses: codecov/codecov-action@v1
uses: codecov/codecov-action@v5
with:
name: Python ${{ matrix.python-version }}

ruff:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pip install --user ruff
- run: ruff check . --extend-select=C4,C9,I,PLC,PLE,PLR,U --ignore=C414,I001,PLR0913,UP007,UP032 --target-version=py39
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -29,3 +29,4 @@ tests/npm-cache
django-pipeline-*/
.tags
node_modules/
package-lock.json
24 changes: 24 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
repos:
- repo: https://github.com/psf/black
rev: 24.8.0
hooks:
- id: black

- repo: https://github.com/PyCQA/isort
rev: 5.13.2
hooks:
- id: isort

- repo: https://github.com/PyCQA/flake8
rev: 7.1.1
hooks:
- id: flake8

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: check-merge-conflict
- id: check-yaml

ci:
autoupdate_schedule: quarterly
13 changes: 13 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

version: 2

build:
os: ubuntu-22.04
tools:
python: "3.10"

sphinx:
configuration: docs/conf.py
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
@@ -90,6 +90,7 @@ or just made Pipeline more awesome.
* Nathan Shafer <nate@torzo.com>
* Patrick Altman <paltman@gmail.com>
* Peter Baumgartner <pete@lincolnloop.com>
* Peyman Salehi <slh.peyman@gmail.com>
* Philipp Wollermann <philipp.wollermann@gmail.com>
* Pierre Drescher <pierre.drescher@gmail.com>
* Rajiv Bose <nerd.bose@gmail.com>
46 changes: 46 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Code of Conduct

As contributors and maintainers of the Jazzband projects, and in the interest of
fostering an open and welcoming community, we pledge to respect all people who
contribute through reporting issues, posting feature requests, updating documentation,
submitting pull requests or patches, and other activities.

We are committed to making participation in the Jazzband a harassment-free experience
for everyone, regardless of the level of experience, gender, gender identity and
expression, sexual orientation, disability, personal appearance, body size, race,
ethnicity, age, religion, or nationality.

Examples of unacceptable behavior by participants include:

- The use of sexualized language or imagery
- Personal attacks
- Trolling or insulting/derogatory comments
- Public or private harassment
- Publishing other's private information, such as physical or electronic addresses,
without explicit permission
- Other unethical or unprofessional conduct

The Jazzband roadies have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are not
aligned to this Code of Conduct, or to ban temporarily or permanently any contributor
for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

By adopting this Code of Conduct, the roadies commit themselves to fairly and
consistently applying these principles to every aspect of managing the jazzband
projects. Roadies who do not follow or enforce the Code of Conduct may be permanently
removed from the Jazzband roadies.

This code of conduct applies both within project spaces and in public spaces when an
individual is representing the project or its community.

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by
contacting the roadies at `roadies@jazzband.co`. All complaints will be reviewed and
investigated and will result in a response that is deemed necessary and appropriate to
the circumstances. Roadies are obligated to maintain confidentiality with regard to the
reporter of an incident.

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version
1.3.0, available at [https://contributor-covenant.org/version/1/3/0/][version]

[homepage]: https://contributor-covenant.org
[version]: https://contributor-covenant.org/version/1/3/0/
2 changes: 1 addition & 1 deletion CONTRIBUTING.rst
Original file line number Diff line number Diff line change
@@ -31,7 +31,7 @@ tox and django installed::
python3.7 -m pip install tox

Since we use a number of node.js tools, one should first install the node
depencies. We reccomend using [nvm](https://github.com/nvm-sh/nvm#installation-and-update) , tl;dr::
dependencies. We recommend using [nvm](https://github.com/nvm-sh/nvm#installation-and-update) , tl;dr::

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.2/install.sh | bash
nvm install node
62 changes: 60 additions & 2 deletions HISTORY.rst
Original file line number Diff line number Diff line change
@@ -3,8 +3,66 @@
History
=======

Unreleased
==========
4.0.0
=====
* Drop support for Python 3.8
* Confirm support for Django 5.1 and drop support for Django 3.2
* Use pyproject.toml

3.1.0
=====

* Replace deprecated .warn method with .warning
* Update sourcemap paths when concatenating source files
* Ensure correct compiler error styling and strip ANSI escape sequences

3.0.0
=====

* Use Pypy 3.10
* Drop support for Python 3.7
* Drop support for Django 2
* Add Python 3.12 support
* Add Django 4.2 support
* Add Django 5.0 support

2.1.0
=====

* Update README.rst and add Pipeline overview image.
* Add TypeScript compiler support.
* Drop support for ``manifesto`` package.
* Add support for Python 3.11 and Django 4.1


2.0.9
=====

* Fixed some typos in the docs.
* Fixed string type of errors reported from compilers and compressors.
* Updated github actions matrix for host and django support.
* Updated github actions configuration to use modern versions of third-party
actions.
* Improved the packager to copy files to (S3) storage if it does not exist
(#502).


2.0.8
=====

* Added **Django 4.0** compatibility. Thanks to @kevinmarsh (#760)
* Add tests for **Django 4.0**, **Python 3.9** and **Python 3.10**.
Thank to @kevinmarsh (#739)
* Introduce CODE_OF_CONDUCT.md for the project. Thank to @hugovk (#758)
* Add precision in the documentation for PipelineCachedStorage.
Thank to @gatsinski (#739)
* Drop support for slimit compressor (#765) due to package not released
an official version for Python 3 and not any new package release from 2013.
* Edit github actions matrix: django 3.2.9 support python 3.10, remove
python 4.0 (doesn't exist) and exclude pypy-3.8 for django-main.
* Add .pre-commit-config.yaml. Thanks to @hugovk (#762)
* Update package.json due to CoffeeScript on NPM has moved to "coffeescript"
* Update setup.py with Django 4.0 and Python 3.10

2.0.7
=====
109 changes: 107 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
@@ -22,17 +22,122 @@ Pipeline is an asset packaging library for Django, providing both CSS and
JavaScript concatenation and compression, built-in JavaScript template support,
and optional data-URI image and font embedding.

.. image:: https://github.com/jazzband/django-pipeline/raw/master/img/django-pipeline.svg
:alt: Django Pipeline Overview


Installation
------------

To install it, simply: ::
To install it, simply:

.. code-block:: bash
pip install django-pipeline
Quickstart
----------

Pipeline compiles and compress your assets files from
``STATICFILES_DIRS`` to your ``STATIC_ROOT`` when you run Django's
``collectstatic`` command.

These simple steps add Pipeline to your project to compile multiple ``.js`` and
``.css`` file into one and compress them.

Add Pipeline to your installed apps:

.. code-block:: python
# settings.py
INSTALLED_APPS = [
...
'pipeline',
]
Use Pipeline specified classes for ``STATICFILES_FINDERS`` and ``STATICFILES_STORAGE``:

.. code-block:: python
STATICFILES_STORAGE = 'pipeline.storage.PipelineManifestStorage'
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'pipeline.finders.PipelineFinder',
)
Configure Pipeline:

.. code-block:: python
# The folowing config merges CSS files(main.css, normalize.css)
# and JavaScript files(app.js, script.js) and compress them using
# `yuglify` into `css/styles.css` and `js/main.js`
# NOTE: Pipeline only works when DEBUG is False
PIPELINE = {
'STYLESHEETS': {
'css_files': {
'source_filenames': (
'css/main.css',
'css/normalize.css',
),
'output_filename': 'css/styles.css',
'extra_context': {
'media': 'screen,projection',
},
},
},
'JAVASCRIPT': {
'js_files': {
'source_filenames': (
'js/app.js',
'js/script.js',
),
'output_filename': 'js/main.js',
}
}
}
Then, you have to install compilers and compressors binary manually.

For example, you can install them using `NPM <https://www.npmjs.com/>`_
and address them from ``node_modules`` directory in your project path:

.. code-block:: python
PIPELINE.update({
'YUGLIFY_BINARY': path.join(BASE_DIR, 'node_modules/.bin/yuglify'),
})
# For a list of all supported compilers and compressors see documentation
Load static files in your template:

.. code-block::
{% load pipeline %}
{% stylesheet 'css_files' %}
{% javascript 'js_files' %}
Documentation
-------------

For documentation, usage, and examples, see :
For documentation, usage, and examples, see:
https://django-pipeline.readthedocs.io


Issues
------
You can report bugs and discuss features on the `issues page <https://github.com/jazzband/django-pipeline/issues>`_.


Changelog
---------

See `HISTORY.rst <https://github.com/jazzband/django-pipeline/blob/master/HISTORY.rst>`_.
26 changes: 26 additions & 0 deletions docs/compilers.rst
Original file line number Diff line number Diff line change
@@ -4,6 +4,32 @@
Compilers
=========

TypeScript compiler
======================

The TypeScript compiler uses `TypeScript <https://www.typescriptlang.org/>`_
to compile your TypeScript code to JavaScript.

To use it add this to your ``PIPELINE['COMPILERS']`` ::

PIPELINE['COMPILERS'] = (
'pipeline.compilers.typescript.TypeScriptCompiler',
)

``TYPE_SCRIPT_BINARY``
---------------------------------

Command line to execute for TypeScript program.
You will most likely change this to the location of ``tsc`` on your system.

Defaults to ``'/usr/bin/env tsc'``.

``TYPE_SCRIPT_ARGUMENTS``
------------------------------------

Additional arguments to use when ``tsc`` is called.

Defaults to ``''``.

Coffee Script compiler
======================
15 changes: 0 additions & 15 deletions docs/compressors.rst
Original file line number Diff line number Diff line change
@@ -158,21 +158,6 @@ Install the jsmin library with your favorite Python package manager ::
pip install jsmin


SlimIt compressor
=================

The slimit compressor uses `SlimIt <https://slimit.readthedocs.io>`_ to
compress javascripts.

To use it add this to your ``PIPELINE['JS_COMPRESSOR']`` ::

PIPELINE['JS_COMPRESSOR'] = 'pipeline.compressors.slimit.SlimItCompressor'

Install the slimit library with your favorite Python package manager ::

pip install slimit


Terser compressor
===================

117 changes: 61 additions & 56 deletions docs/conf.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# -*- coding: utf-8 -*-
#
# Pipeline documentation build configuration file, created by
# sphinx-quickstart on Sat Apr 30 17:47:55 2011.
#
@@ -10,206 +8,213 @@
#
# All configuration values have a default; values that are commented out
# serve to show the default.
from pkg_resources import get_distribution

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
# sys.path.insert(0, os.path.abspath('.'))
from datetime import datetime

from pipeline import __version__ as pipeline_version

# -- General configuration -----------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# needs_sphinx = '1.0'

# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = []

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
templates_path = ["_templates"]

# The suffix of source filenames.
source_suffix = '.rst'
source_suffix = ".rst"

# The encoding of source files.
#source_encoding = 'utf-8-sig'
# source_encoding = 'utf-8-sig'

# The master toctree document.
master_doc = 'index'
master_doc = "index"

# General information about the project.
project = u'django-pipeline'
copyright = u'2011-2014, Timothée Peignier'
project = "django-pipeline"
current_year = datetime.now().year
copyright = "2011-{}, Timothée Peignier".format(current_year)

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The full version, including alpha/beta/rc tags.
release = get_distribution("django-pipeline").version
release = pipeline_version
# The short X.Y version.
version = ".".join(release.split(".")[:2])

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# language = None

# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# today_fmt = '%B %d, %Y'

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build']
exclude_patterns = ["_build"]

# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# default_role = None

# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# add_function_parentheses = True

# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# add_module_names = True

# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# show_authors = False

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
pygments_style = "sphinx"

# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# modindex_common_prefix = []


# -- Options for HTML output ---------------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'default'
html_theme = "default"

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# html_theme_options = {}

# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# html_theme_path = []

# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# html_title = None

# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# html_short_title = None

# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# html_logo = None

# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# html_favicon = None

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
#html_static_path = ['_static']
# html_static_path = ['_static']

# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# html_last_updated_fmt = '%b %d, %Y'

# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# html_use_smartypants = True

# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# html_sidebars = {}

# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# html_additional_pages = {}

# If false, no module index is generated.
#html_domain_indices = True
# html_domain_indices = True

# If false, no index is generated.
#html_use_index = True
# html_use_index = True

# If true, the index is split into individual pages for each letter.
#html_split_index = False
# html_split_index = False

# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# html_show_sourcelink = True

# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# html_show_sphinx = True

# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# html_show_copyright = True

# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# html_use_opensearch = ''

# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# html_file_suffix = None

# Output file base name for HTML help builder.
htmlhelp_basename = 'django-pipelinedoc'
htmlhelp_basename = "django-pipelinedoc"


# -- Options for LaTeX output --------------------------------------------------

# The paper size ('letter' or 'a4').
#latex_paper_size = 'letter'
# latex_paper_size = 'letter'

# The font size ('10pt', '11pt' or '12pt').
#latex_font_size = '10pt'
# latex_font_size = '10pt'

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'django-pipeline.tex', u'Pipeline Documentation',
u'Timothée Peignier', 'manual'),
(
"index",
"django-pipeline.tex",
"Pipeline Documentation",
"Timothée Peignier",
"manual",
),
]

# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# latex_logo = None

# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# latex_use_parts = False

# If true, show page references after internal links.
#latex_show_pagerefs = False
# latex_show_pagerefs = False

# If true, show URL addresses after external links.
#latex_show_urls = False
# latex_show_urls = False

# Additional stuff for the LaTeX preamble.
#latex_preamble = ''
# latex_preamble = ''

# Documents to append as an appendix to all manuals.
#latex_appendices = []
# latex_appendices = []

# If false, no module index is generated.
#latex_domain_indices = True
# latex_domain_indices = True


# -- Options for manual page output --------------------------------------------

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'django-pipeline', u'Pipeline Documentation',
[u'Timothée Peignier'], 1)
("index", "django-pipeline", "Pipeline Documentation", ["Timothée Peignier"], 1)
]
3 changes: 3 additions & 0 deletions docs/storages.rst
Original file line number Diff line number Diff line change
@@ -39,6 +39,9 @@ that allows staticfiles to locate your outputted assets : ::
If you use ``PipelineCachedStorage`` you may also like the ``CachedFileFinder``,
which allows you to use integration tests with cached file URLs.

Keep in mind that ``PipelineCachedStorage`` is only available for Django versions
before 3.1.

If you want to exclude Pipelinable content from your collected static files,
you can also use Pipeline's ``FileSystemFinder`` and ``AppDirectoriesFinder``.
These finders will also exclude `unwanted` content like READMEs, tests and
8 changes: 0 additions & 8 deletions docs/usage.rst
Original file line number Diff line number Diff line change
@@ -117,14 +117,6 @@ Ensure that it comes after any middleware which modifies your HTML, like ``GZipM
'pipeline.middleware.MinifyHTMLMiddleware',
)

Cache manifest
==============

Pipeline provide a way to add your javascripts and stylesheets files to a
cache-manifest via `Manifesto <https://manifesto.readthedocs.io/>`_.

To do so, you just need to add manifesto app to your ``INSTALLED_APPS``.


Jinja
=====
2 changes: 1 addition & 1 deletion docs/using.rst
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@ For their internal tools: http://www.20minutes.fr
Borsala
-------

Borsala is the social investment plaform. You can follow stock markets that are traded in Turkey: http://borsala.com
Borsala is the social investment platform. You can follow stock markets that are traded in Turkey: http://borsala.com


Croisé dans le Métro
118 changes: 118 additions & 0 deletions img/django-pipeline.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 4 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -13,16 +13,17 @@
"dependencies": {
"babel-cli": "latest",
"babel-preset-es2015": "latest",
"coffee-script": "latest",
"coffeescript": "latest",
"less": "latest",
"livescript": "latest",
"node-sass": "latest",
"sass": "latest",
"stylus": "latest",
"cssmin": "latest",
"google-closure-compiler": "latest",
"terser": "latest",
"uglify-js": "latest",
"yuglify": "1.0.x",
"yuicompressor": "latest"
"yuicompressor": "latest",
"typescript": "latest"
}
}
33 changes: 27 additions & 6 deletions pipeline/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,28 @@
from pkg_resources import get_distribution, DistributionNotFound

PackageNotFoundError = None
DistributionNotFound = None
try:
__version__ = get_distribution("django-pipeline").version
except DistributionNotFound:
# package is not installed
__version__ = None
from importlib.metadata import PackageNotFoundError
from importlib.metadata import version as get_version
except ImportError:
get_version = None
PackageNotFoundError = None
if get_version is None:
try:
from pkg_resources import DistributionNotFound, get_distribution

def get_version(x):
return get_distribution(x).version

except ImportError:
get_version = None
DistributionNotFound = None
get_distribution = None

__version__ = None
if get_version is not None:
try:
__version__ = get_version("django-pipeline")
except PackageNotFoundError:
pass
except DistributionNotFound:
pass
28 changes: 16 additions & 12 deletions pipeline/collector.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
import os

from collections import OrderedDict

import django
from django.contrib.staticfiles import finders
from django.contrib.staticfiles.storage import staticfiles_storage

from pipeline.finders import PipelineFinder


class Collector(object):
class Collector:
request = None

def __init__(self, storage=None):
@@ -34,20 +32,21 @@ def collect(self, request=None, files=[]):
if self.request and self.request is request:
return
self.request = request
found_files = OrderedDict()
found_files = {}
for finder in finders.get_finders():
# Ignore our finder to avoid looping
if isinstance(finder, PipelineFinder):
continue
for path, storage in finder.list(['CVS', '.*', '*~']):
for path, storage in finder.list(["CVS", ".*", "*~"]):
# Prefix the relative path if the source storage contains it
if getattr(storage, 'prefix', None):
if getattr(storage, "prefix", None):
prefixed_path = os.path.join(storage.prefix, path)
else:
prefixed_path = path

if (prefixed_path not in found_files and
(not files or prefixed_path in files)):
if prefixed_path not in found_files and (
not files or prefixed_path in files
):
found_files[prefixed_path] = (storage, path)
self.copy_file(path, prefixed_path, storage)

@@ -68,7 +67,10 @@ def delete_file(self, path, prefixed_path, source_storage):
if self.storage.exists(prefixed_path):
try:
# When was the target file modified last time?
target_last_modified = self._get_modified_time(self.storage, prefixed_path)
target_last_modified = self._get_modified_time(
self.storage,
prefixed_path,
)
except (OSError, NotImplementedError, AttributeError):
# The storage doesn't support ``modified_time`` or failed
pass
@@ -81,11 +83,13 @@ def delete_file(self, path, prefixed_path, source_storage):
else:
# Skip the file if the source file is younger
# Avoid sub-second precision
if (target_last_modified.replace(microsecond=0)
>= source_last_modified.replace(microsecond=0)):
return False
if target_last_modified.replace(
microsecond=0
) >= source_last_modified.replace(microsecond=0):
return False
# Then delete the existing file if really needed
self.storage.delete(prefixed_path)
return True


default_collector = Collector()
56 changes: 35 additions & 21 deletions pipeline/compilers/__init__.py
Original file line number Diff line number Diff line change
@@ -6,13 +6,14 @@
from django.contrib.staticfiles import finders
from django.contrib.staticfiles.storage import staticfiles_storage
from django.core.files.base import ContentFile
from django.utils.encoding import force_str

from pipeline.conf import settings
from pipeline.exceptions import CompilerError
from pipeline.utils import to_class, set_std_streams_blocking
from pipeline.utils import set_std_streams_blocking, to_class


class Compiler(object):
class Compiler:
def __init__(self, storage=None, verbose=False):
if storage is None:
storage = staticfiles_storage
@@ -35,9 +36,13 @@ def _compile(input_path):
project_infile = finders.find(input_path)
outfile = compiler.output_path(infile, compiler.output_extension)
outdated = compiler.is_outdated(project_infile, outfile)
compiler.compile_file(project_infile, outfile,
outdated=outdated, force=force,
**compiler_options)
compiler.compile_file(
project_infile,
outfile,
outdated=outdated,
force=force,
**compiler_options,
)

return compiler.output_path(input_path, compiler.output_extension)
else:
@@ -49,11 +54,13 @@ def _compile(input_path):
except ImportError:
return list(map(_compile, paths))
else:
with futures.ThreadPoolExecutor(max_workers=multiprocessing.cpu_count()) as executor:
with futures.ThreadPoolExecutor(
max_workers=multiprocessing.cpu_count()
) as executor:
return list(executor.map(_compile, paths))


class CompilerBase(object):
class CompilerBase:
def __init__(self, verbose, storage):
self.verbose = verbose
self.storage = storage
@@ -68,14 +75,14 @@ def save_file(self, path, content):
return self.storage.save(path, ContentFile(content))

def read_file(self, path):
file = self.storage.open(path, 'rb')
file = self.storage.open(path, "rb")
content = file.read()
file.close()
return content

def output_path(self, path, extension):
path = os.path.splitext(path)
return '.'.join((path[0], extension))
return ".".join((path[0], extension))

def is_outdated(self, infile, outfile):
if not os.path.exists(outfile):
@@ -111,17 +118,22 @@ def execute_command(self, command, cwd=None, stdout_captured=None):
else:
argument_list.extend(flattening_arg)

# The first element in argument_list is the program that will be executed; if it is '', then
# a PermissionError will be raised. Thus empty arguments are filtered out from argument_list
# The first element in argument_list is the program that will be
# executed; if it is '', then a PermissionError will be raised.
# Thus empty arguments are filtered out from argument_list
argument_list = list(filter(None, argument_list))
stdout = None
try:
# We always catch stdout in a file, but we may not have a use for it.
temp_file_container = cwd or os.path.dirname(stdout_captured or "") or os.getcwd()
with NamedTemporaryFile('wb', delete=False, dir=temp_file_container) as stdout:
compiling = subprocess.Popen(argument_list, cwd=cwd,
stdout=stdout,
stderr=subprocess.PIPE)
temp_file_container = (
cwd or os.path.dirname(stdout_captured or "") or os.getcwd()
)
with NamedTemporaryFile(
"wb", delete=False, dir=temp_file_container
) as stdout:
compiling = subprocess.Popen(
argument_list, cwd=cwd, stdout=stdout, stderr=subprocess.PIPE
)
_, stderr = compiling.communicate()
set_std_streams_blocking()

@@ -130,21 +142,23 @@ def execute_command(self, command, cwd=None, stdout_captured=None):
raise CompilerError(
f"{argument_list!r} exit code {compiling.returncode}\n{stderr}",
command=argument_list,
error_output=stderr)
error_output=force_str(stderr),
)

# User wants to see everything that happened.
if self.verbose:
with open(stdout.name, 'rb') as out:
with open(stdout.name, "rb") as out:
print(out.read())
print(stderr)
except OSError as e:
stdout_captured = None # Don't save erroneous result.
raise CompilerError(e, command=argument_list,
error_output=str(e))
raise CompilerError(e, command=argument_list, error_output=str(e))
finally:
# Decide what to do with captured stdout.
if stdout:
if stdout_captured:
shutil.move(stdout.name, os.path.join(cwd or os.curdir, stdout_captured))
shutil.move(
stdout.name, os.path.join(cwd or os.curdir, stdout_captured)
)
else:
os.remove(stdout.name)
6 changes: 3 additions & 3 deletions pipeline/compilers/coffee.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
from pipeline.conf import settings
from pipeline.compilers import SubProcessCompiler
from pipeline.conf import settings


class CoffeeScriptCompiler(SubProcessCompiler):
output_extension = 'js'
output_extension = "js"

def match_file(self, path):
return path.endswith('.coffee') or path.endswith('.litcoffee')
return path.endswith(".coffee") or path.endswith(".litcoffee")

def compile_file(self, infile, outfile, outdated=False, force=False):
if not outdated and not force:
8 changes: 4 additions & 4 deletions pipeline/compilers/es6.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
from pipeline.conf import settings
from pipeline.compilers import SubProcessCompiler
from pipeline.conf import settings


class ES6Compiler(SubProcessCompiler):
output_extension = 'js'
output_extension = "js"

def match_file(self, path):
return path.endswith('.es6')
return path.endswith(".es6")

def compile_file(self, infile, outfile, outdated=False, force=False):
if not outdated and not force:
@@ -16,6 +16,6 @@ def compile_file(self, infile, outfile, outdated=False, force=False):
settings.BABEL_ARGUMENTS,
infile,
"-o",
outfile
outfile,
)
return self.execute_command(command)
10 changes: 6 additions & 4 deletions pipeline/compilers/less.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
from os.path import dirname

from pipeline.conf import settings
from pipeline.compilers import SubProcessCompiler
from pipeline.conf import settings


class LessCompiler(SubProcessCompiler):
output_extension = 'css'
output_extension = "css"

def match_file(self, filename):
return filename.endswith('.less')
return filename.endswith(".less")

def compile_file(self, infile, outfile, outdated=False, force=False):
# Pipe to file rather than provide outfile arg due to a bug in lessc
@@ -17,4 +17,6 @@ def compile_file(self, infile, outfile, outdated=False, force=False):
settings.LESS_ARGUMENTS,
infile,
)
return self.execute_command(command, cwd=dirname(infile), stdout_captured=outfile)
return self.execute_command(
command, cwd=dirname(infile), stdout_captured=outfile
)
6 changes: 3 additions & 3 deletions pipeline/compilers/livescript.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
from pipeline.conf import settings
from pipeline.compilers import SubProcessCompiler
from pipeline.conf import settings


class LiveScriptCompiler(SubProcessCompiler):
output_extension = 'js'
output_extension = "js"

def match_file(self, path):
return path.endswith('.ls')
return path.endswith(".ls")

def compile_file(self, infile, outfile, outdated=False, force=False):
if not outdated and not force:
13 changes: 4 additions & 9 deletions pipeline/compilers/sass.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
from os.path import dirname

from pipeline.conf import settings
from pipeline.compilers import SubProcessCompiler
from pipeline.conf import settings


class SASSCompiler(SubProcessCompiler):
output_extension = 'css'
output_extension = "css"

def match_file(self, filename):
return filename.endswith(('.scss', '.sass'))
return filename.endswith((".scss", ".sass"))

def compile_file(self, infile, outfile, outdated=False, force=False):
command = (
settings.SASS_BINARY,
settings.SASS_ARGUMENTS,
infile,
outfile
)
command = (settings.SASS_BINARY, settings.SASS_ARGUMENTS, infile, outfile)
return self.execute_command(command, cwd=dirname(infile))
12 changes: 4 additions & 8 deletions pipeline/compilers/stylus.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,15 @@
from os.path import dirname

from pipeline.conf import settings
from pipeline.compilers import SubProcessCompiler
from pipeline.conf import settings


class StylusCompiler(SubProcessCompiler):
output_extension = 'css'
output_extension = "css"

def match_file(self, filename):
return filename.endswith('.styl')
return filename.endswith(".styl")

def compile_file(self, infile, outfile, outdated=False, force=False):
command = (
settings.STYLUS_BINARY,
settings.STYLUS_ARGUMENTS,
infile
)
command = (settings.STYLUS_BINARY, settings.STYLUS_ARGUMENTS, infile)
return self.execute_command(command, cwd=dirname(infile))
21 changes: 21 additions & 0 deletions pipeline/compilers/typescript.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
from pipeline.compilers import SubProcessCompiler
from pipeline.conf import settings


class TypeScriptCompiler(SubProcessCompiler):
output_extension = "js"

def match_file(self, path):
return path.endswith(".ts")

def compile_file(self, infile, outfile, outdated=False, force=False):
if not outdated and not force:
return
command = (
settings.TYPE_SCRIPT_BINARY,
settings.TYPE_SCRIPT_ARGUMENTS,
infile,
"--outFile",
outfile,
)
return self.execute_command(command)
344 changes: 263 additions & 81 deletions pipeline/compressors/__init__.py

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion pipeline/compressors/closure.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from pipeline.conf import settings
from pipeline.compressors import SubProcessCompressor
from pipeline.conf import settings


class ClosureCompressor(SubProcessCompressor):
3 changes: 3 additions & 0 deletions pipeline/compressors/csshtmljsminify.py
Original file line number Diff line number Diff line change
@@ -6,10 +6,13 @@ class CssHtmlJsMinifyCompressor(CompressorBase):
CSS, HTML and JS compressor based on the Python library css-html-js-minify
(https://pypi.org/project/css-html-js-minify/).
"""

def compress_css(self, css):
from css_html_js_minify import css_minify

return css_minify(css)

def compress_js(self, js):
from css_html_js_minify import js_minify

return js_minify(js)
2 changes: 1 addition & 1 deletion pipeline/compressors/cssmin.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from pipeline.conf import settings
from pipeline.compressors import SubProcessCompressor
from pipeline.conf import settings


class CSSMinCompressor(SubProcessCompressor):
6 changes: 3 additions & 3 deletions pipeline/compressors/csstidy.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
from django.core.files import temp as tempfile

from pipeline.conf import settings
from pipeline.compressors import SubProcessCompressor
from pipeline.conf import settings


class CSSTidyCompressor(SubProcessCompressor):
def compress_css(self, css):
output_file = tempfile.NamedTemporaryFile(suffix='.pipeline')
output_file = tempfile.NamedTemporaryFile(suffix=".pipeline")

command = (
settings.CSSTIDY_BINARY,
"-",
settings.CSSTIDY_ARGUMENTS,
output_file.name
output_file.name,
)
self.execute_command(command, css)

2 changes: 2 additions & 0 deletions pipeline/compressors/jsmin.py
Original file line number Diff line number Diff line change
@@ -6,6 +6,8 @@ class JSMinCompressor(CompressorBase):
JS compressor based on the Python library jsmin
(http://pypi.python.org/pypi/jsmin/).
"""

def compress_js(self, js):
from jsmin import jsmin

return jsmin(js)
11 changes: 0 additions & 11 deletions pipeline/compressors/slimit.py

This file was deleted.

6 changes: 2 additions & 4 deletions pipeline/compressors/terser.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
from __future__ import unicode_literals

from pipeline.conf import settings
from pipeline.compressors import SubProcessCompressor
from pipeline.conf import settings


class TerserCompressor(SubProcessCompressor):
def compress_js(self, js):
command = (settings.TERSER_BINARY, settings.TERSER_ARGUMENTS)
if self.verbose:
command += ' --verbose'
command += " --verbose"
return self.execute_command(command, js)
4 changes: 2 additions & 2 deletions pipeline/compressors/uglifyjs.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
from pipeline.conf import settings
from pipeline.compressors import SubProcessCompressor
from pipeline.conf import settings


class UglifyJSCompressor(SubProcessCompressor):
def compress_js(self, js):
command = (settings.UGLIFYJS_BINARY, settings.UGLIFYJS_ARGUMENTS)
if self.verbose:
command += ' --verbose'
command += " --verbose"
return self.execute_command(command, js)
12 changes: 4 additions & 8 deletions pipeline/compressors/yuglify.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
from pipeline.conf import settings
from pipeline.compressors import SubProcessCompressor
from pipeline.conf import settings


class YuglifyCompressor(SubProcessCompressor):
def compress_common(self, content, compress_type, arguments):
command = (
settings.YUGLIFY_BINARY,
f"--type={compress_type}",
arguments
)
command = (settings.YUGLIFY_BINARY, f"--type={compress_type}", arguments)
return self.execute_command(command, content)

def compress_js(self, js):
return self.compress_common(js, 'js', settings.YUGLIFY_JS_ARGUMENTS)
return self.compress_common(js, "js", settings.YUGLIFY_JS_ARGUMENTS)

def compress_css(self, css):
return self.compress_common(css, 'css', settings.YUGLIFY_CSS_ARGUMENTS)
return self.compress_common(css, "css", settings.YUGLIFY_CSS_ARGUMENTS)
12 changes: 4 additions & 8 deletions pipeline/compressors/yui.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
from pipeline.conf import settings
from pipeline.compressors import SubProcessCompressor
from pipeline.conf import settings


class YUICompressor(SubProcessCompressor):
def compress_common(self, content, compress_type, arguments):
command = (
settings.YUI_BINARY,
f"--type={compress_type}",
arguments
)
command = (settings.YUI_BINARY, f"--type={compress_type}", arguments)
return self.execute_command(command, content)

def compress_js(self, js):
return self.compress_common(js, 'js', settings.YUI_JS_ARGUMENTS)
return self.compress_common(js, "js", settings.YUI_JS_ARGUMENTS)

def compress_css(self, css):
return self.compress_common(css, 'css', settings.YUI_CSS_ARGUMENTS)
return self.compress_common(css, "css", settings.YUI_CSS_ARGUMENTS)
139 changes: 60 additions & 79 deletions pipeline/conf.py
Original file line number Diff line number Diff line change
@@ -1,95 +1,76 @@
import os
from collections.abc import MutableMapping
import shlex
from collections.abc import MutableMapping

from django.conf import settings as _settings
from django.core.signals import setting_changed
from django.dispatch import receiver


DEFAULTS = {
'PIPELINE_ENABLED': not _settings.DEBUG,

'PIPELINE_COLLECTOR_ENABLED': True,

'PIPELINE_ROOT': _settings.STATIC_ROOT,
'PIPELINE_URL': _settings.STATIC_URL,

'SHOW_ERRORS_INLINE': _settings.DEBUG,

'CSS_COMPRESSOR': 'pipeline.compressors.yuglify.YuglifyCompressor',
'JS_COMPRESSOR': 'pipeline.compressors.yuglify.YuglifyCompressor',
'COMPILERS': [],

'STYLESHEETS': {},
'JAVASCRIPT': {},

'TEMPLATE_NAMESPACE': "window.JST",
'TEMPLATE_EXT': ".jst",
'TEMPLATE_FUNC': "template",
'TEMPLATE_SEPARATOR': "_",

'DISABLE_WRAPPER': False,
'JS_WRAPPER': "(function() {\n%s\n}).call(this);",

'CSSTIDY_BINARY': '/usr/bin/env csstidy',
'CSSTIDY_ARGUMENTS': '--template=highest',

'YUGLIFY_BINARY': '/usr/bin/env yuglify',
'YUGLIFY_CSS_ARGUMENTS': '--terminal',
'YUGLIFY_JS_ARGUMENTS': '--terminal',

'YUI_BINARY': '/usr/bin/env yuicompressor',
'YUI_CSS_ARGUMENTS': '',
'YUI_JS_ARGUMENTS': '',

'CLOSURE_BINARY': '/usr/bin/env closure',
'CLOSURE_ARGUMENTS': '',

'UGLIFYJS_BINARY': '/usr/bin/env uglifyjs',
'UGLIFYJS_ARGUMENTS': '',

'TERSER_BINARY': '/usr/bin/env terser',
'TERSER_ARGUMENTS': '--compress',

'CSSMIN_BINARY': '/usr/bin/env cssmin',
'CSSMIN_ARGUMENTS': '',

'COFFEE_SCRIPT_BINARY': '/usr/bin/env coffee',
'COFFEE_SCRIPT_ARGUMENTS': '',

'BABEL_BINARY': '/usr/bin/env babel',
'BABEL_ARGUMENTS': '',

'LIVE_SCRIPT_BINARY': '/usr/bin/env lsc',
'LIVE_SCRIPT_ARGUMENTS': '',

'SASS_BINARY': '/usr/bin/env sass',
'SASS_ARGUMENTS': '',

'STYLUS_BINARY': '/usr/bin/env stylus',
'STYLUS_ARGUMENTS': '',

'LESS_BINARY': '/usr/bin/env lessc',
'LESS_ARGUMENTS': '',

'MIMETYPES': (
(('text/coffeescript'), ('.coffee')),
(('text/less'), ('.less')),
(('text/javascript'), ('.js')),
(('text/x-sass'), ('.sass')),
(('text/x-scss'), ('.scss'))
"PIPELINE_ENABLED": not _settings.DEBUG,
"PIPELINE_COLLECTOR_ENABLED": True,
"PIPELINE_ROOT": _settings.STATIC_ROOT,
"PIPELINE_URL": _settings.STATIC_URL,
"SHOW_ERRORS_INLINE": _settings.DEBUG,
"CSS_COMPRESSOR": "pipeline.compressors.yuglify.YuglifyCompressor",
"JS_COMPRESSOR": "pipeline.compressors.yuglify.YuglifyCompressor",
"COMPILERS": [],
"STYLESHEETS": {},
"JAVASCRIPT": {},
"TEMPLATE_NAMESPACE": "window.JST",
"TEMPLATE_EXT": ".jst",
"TEMPLATE_FUNC": "template",
"TEMPLATE_SEPARATOR": "_",
"DISABLE_WRAPPER": False,
"JS_WRAPPER": "(function() {\n%s\n}).call(this);",
"CSSTIDY_BINARY": "/usr/bin/env csstidy",
"CSSTIDY_ARGUMENTS": "--template=highest",
"YUGLIFY_BINARY": "/usr/bin/env yuglify",
"YUGLIFY_CSS_ARGUMENTS": "--terminal",
"YUGLIFY_JS_ARGUMENTS": "--terminal",
"YUI_BINARY": "/usr/bin/env yuicompressor",
"YUI_CSS_ARGUMENTS": "",
"YUI_JS_ARGUMENTS": "",
"CLOSURE_BINARY": "/usr/bin/env closure",
"CLOSURE_ARGUMENTS": "",
"UGLIFYJS_BINARY": "/usr/bin/env uglifyjs",
"UGLIFYJS_ARGUMENTS": "",
"TERSER_BINARY": "/usr/bin/env terser",
"TERSER_ARGUMENTS": "--compress",
"CSSMIN_BINARY": "/usr/bin/env cssmin",
"CSSMIN_ARGUMENTS": "",
"COFFEE_SCRIPT_BINARY": "/usr/bin/env coffee",
"COFFEE_SCRIPT_ARGUMENTS": "",
"BABEL_BINARY": "/usr/bin/env babel",
"BABEL_ARGUMENTS": "",
"LIVE_SCRIPT_BINARY": "/usr/bin/env lsc",
"LIVE_SCRIPT_ARGUMENTS": "",
"TYPE_SCRIPT_BINARY": "/usr/bin/env tsc",
"TYPE_SCRIPT_ARGUMENTS": "",
"SASS_BINARY": "/usr/bin/env sass",
"SASS_ARGUMENTS": "",
"STYLUS_BINARY": "/usr/bin/env stylus",
"STYLUS_ARGUMENTS": "",
"LESS_BINARY": "/usr/bin/env lessc",
"LESS_ARGUMENTS": "",
"MIMETYPES": (
(("text/coffeescript"), (".coffee")),
(("text/less"), (".less")),
(("text/javascript"), (".js")),
(("text/typescript"), (".ts")),
(("text/x-sass"), (".sass")),
(("text/x-scss"), (".scss")),
),

'EMBED_MAX_IMAGE_SIZE': 32700,
'EMBED_PATH': r'[/]?embed/',
"EMBED_MAX_IMAGE_SIZE": 32700,
"EMBED_PATH": r"[/]?embed/",
}


class PipelineSettings(MutableMapping):
"""
Container object for pipeline settings
"""

def __init__(self, wrapped_settings):
self.settings = DEFAULTS.copy()
self.settings.update(wrapped_settings)
@@ -98,7 +79,7 @@ def __getitem__(self, key):
value = self.settings[key]
if key.endswith(("_BINARY", "_ARGUMENTS")):
if isinstance(value, (str,)):
return tuple(shlex.split(value, posix=(os.name == 'posix')))
return tuple(shlex.split(value, posix=(os.name == "posix")))
return tuple(value)
return value

@@ -123,5 +104,5 @@ def __getattr__(self, name):

@receiver(setting_changed)
def reload_settings(**kwargs):
if kwargs['setting'] == 'PIPELINE':
settings.update(kwargs['value'])
if kwargs["setting"] == "PIPELINE":
settings.update(kwargs["value"])
4 changes: 1 addition & 3 deletions pipeline/exceptions.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@


class PipelineException(Exception):
pass

@@ -10,7 +8,7 @@ class PackageNotFound(PipelineException):

class CompilerError(PipelineException):
def __init__(self, msg, command=None, error_output=None):
super(CompilerError, self).__init__(msg)
super().__init__(msg)

self.command = command
self.error_output = error_output.strip()
82 changes: 45 additions & 37 deletions pipeline/finders.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
from itertools import chain
from os.path import normpath

from django.contrib.staticfiles.finders import (
AppDirectoriesFinder as DjangoAppDirectoriesFinder,
)
from django.contrib.staticfiles.finders import BaseFinder, BaseStorageFinder
from django.contrib.staticfiles.finders import (
FileSystemFinder as DjangoFileSystemFinder,
)
from django.contrib.staticfiles.finders import find
from django.contrib.staticfiles.storage import staticfiles_storage
from django.contrib.staticfiles.finders import BaseFinder, BaseStorageFinder, find, \
AppDirectoriesFinder as DjangoAppDirectoriesFinder, FileSystemFinder as DjangoFileSystemFinder
from django.utils._os import safe_join
from os.path import normpath

from pipeline.conf import settings

@@ -14,7 +20,7 @@ class PipelineFinder(BaseStorageFinder):

def find(self, path, all=False):
if not settings.PIPELINE_ENABLED:
return super(PipelineFinder, self).find(path, all)
return super().find(path, all)
else:
return []

@@ -29,7 +35,7 @@ def find(self, path, all=False):
"""
matches = []
for elem in chain(settings.STYLESHEETS.values(), settings.JAVASCRIPT.values()):
if normpath(elem['output_filename']) == normpath(path):
if normpath(elem["output_filename"]) == normpath(path):
match = safe_join(settings.PIPELINE_ROOT, path)
if not all:
return match
@@ -46,17 +52,17 @@ def find(self, path, all=False):
Work out the uncached name of the file and look that up instead
"""
try:
start, _, extn = path.rsplit('.', 2)
start, _, extn = path.rsplit(".", 2)
except ValueError:
return []
path = '.'.join((start, extn))
path = ".".join((start, extn))
return find(path, all=all) or []

def list(self, *args):
return []


class PatternFilterMixin(object):
class PatternFilterMixin:
ignore_patterns = []

def get_ignored_patterns(self):
@@ -65,7 +71,7 @@ def get_ignored_patterns(self):
def list(self, ignore_patterns):
if ignore_patterns:
ignore_patterns = ignore_patterns + self.get_ignored_patterns()
return super(PatternFilterMixin, self).list(ignore_patterns)
return super().list(ignore_patterns)


class AppDirectoriesFinder(PatternFilterMixin, DjangoAppDirectoriesFinder):
@@ -76,12 +82,13 @@ class AppDirectoriesFinder(PatternFilterMixin, DjangoAppDirectoriesFinder):
This allows us to concentrate/compress our components without dragging
the raw versions in via collectstatic.
"""

ignore_patterns = [
'*.js',
'*.css',
'*.less',
'*.scss',
'*.styl',
"*.js",
"*.css",
"*.less",
"*.scss",
"*.styl",
]


@@ -92,28 +99,29 @@ class FileSystemFinder(PatternFilterMixin, DjangoFileSystemFinder):
This allows us to concentrate/compress our components without dragging
the raw versions in too.
"""

ignore_patterns = [
'*.js',
'*.css',
'*.less',
'*.scss',
'*.styl',
'*.sh',
'*.html',
'*.md',
'*.markdown',
'*.php',
'*.txt',
'README*',
'LICENSE*',
'*examples*',
'*test*',
'*bin*',
'*samples*',
'*docs*',
'*build*',
'*demo*',
'Makefile*',
'Gemfile*',
'node_modules',
"*.js",
"*.css",
"*.less",
"*.scss",
"*.styl",
"*.sh",
"*.html",
"*.md",
"*.markdown",
"*.php",
"*.txt",
"README*",
"LICENSE*",
"*examples*",
"*test*",
"*bin*",
"*samples*",
"*docs*",
"*build*",
"*demo*",
"Makefile*",
"Gemfile*",
"node_modules",
]
53 changes: 26 additions & 27 deletions pipeline/forms.py
Original file line number Diff line number Diff line change
@@ -8,7 +8,7 @@
from .packager import Packager


class PipelineFormMediaProperty(object):
class PipelineFormMediaProperty:
"""A property that converts Pipeline packages to lists of files.
This is used behind the scenes for any Media classes that subclass
@@ -122,8 +122,7 @@ def __new__(cls, name, bases, attrs):
type:
The new class.
"""
new_class = super(PipelineFormMediaMetaClass, cls).__new__(
cls, name, bases, attrs)
new_class = super().__new__(cls, name, bases, attrs)

# If we define any packages, we'll need to use our special
# PipelineFormMediaProperty class. We use this instead of intercepting
@@ -132,13 +131,15 @@ def __new__(cls, name, bases, attrs):
# and accesses them from there. By using these special properties, we
# can handle direct access (Media.css) and dictionary-based access
# (Media.__dict__['css']).
if 'css_packages' in attrs:
if "css_packages" in attrs:
new_class.css = PipelineFormMediaProperty(
cls._get_css_files, new_class, attrs.get('css') or {})
cls._get_css_files, new_class, attrs.get("css") or {}
)

if 'js_packages' in attrs:
if "js_packages" in attrs:
new_class.js = PipelineFormMediaProperty(
cls._get_js_files, new_class, attrs.get('js') or [])
cls._get_js_files, new_class, attrs.get("js") or []
)

return new_class

@@ -156,17 +157,17 @@ def _get_css_files(cls, extra_files):
attribute.
"""
packager = Packager()
css_packages = getattr(cls, 'css_packages', {})

return dict(
(media_target,
cls._get_media_files(packager=packager,
media_packages=media_packages,
media_type='css',
extra_files=extra_files.get(media_target,
[])))
css_packages = getattr(cls, "css_packages", {})

return {
media_target: cls._get_media_files(
packager=packager,
media_packages=media_packages,
media_type="css",
extra_files=extra_files.get(media_target, []),
)
for media_target, media_packages in css_packages.items()
)
}

def _get_js_files(cls, extra_files):
"""Return all JavaScript files from the Media class.
@@ -182,12 +183,12 @@ def _get_js_files(cls, extra_files):
"""
return cls._get_media_files(
packager=Packager(),
media_packages=getattr(cls, 'js_packages', {}),
media_type='js',
extra_files=extra_files)
media_packages=getattr(cls, "js_packages", {}),
media_type="js",
extra_files=extra_files,
)

def _get_media_files(cls, packager, media_packages, media_type,
extra_files):
def _get_media_files(cls, packager, media_packages, media_type, extra_files):
"""Return source or output media files for a list of packages.
This will go through the media files belonging to the provided list
@@ -214,23 +215,21 @@ def _get_media_files(cls, packager, media_packages, media_type,
"""
source_files = list(extra_files)

if (not settings.PIPELINE_ENABLED and
settings.PIPELINE_COLLECTOR_ENABLED):
if not settings.PIPELINE_ENABLED and settings.PIPELINE_COLLECTOR_ENABLED:
default_collector.collect()

for media_package in media_packages:
package = packager.package_for(media_type, media_package)

if settings.PIPELINE_ENABLED:
source_files.append(
staticfiles_storage.url(package.output_filename))
source_files.append(staticfiles_storage.url(package.output_filename))
else:
source_files += packager.compile(package.paths)

return source_files


class PipelineFormMedia(object, metaclass=PipelineFormMediaMetaClass):
class PipelineFormMedia(metaclass=PipelineFormMediaMetaClass):
"""Base class for form or widget Media classes that use Pipeline packages.
Forms or widgets that need custom CSS or JavaScript media on a page can
9 changes: 5 additions & 4 deletions pipeline/glob.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import fnmatch
import os
import re
import fnmatch

from django.contrib.staticfiles.storage import staticfiles_storage

@@ -42,6 +42,7 @@ def iglob(pathname):
for name in glob_in_dir(dirname, basename):
yield os.path.join(dirname, name)


# These 2 helper functions non-recursively glob inside a literal directory.
# They return a list of basenames. `glob1` accepts a pattern while `glob0`
# takes a literal basename (so it only has to check for its existence).
@@ -55,8 +56,8 @@ def glob1(dirname, pattern):
# We are not sure that dirname is a real directory
# and storage implementations are really exotic.
return []
if pattern[0] != '.':
names = [x for x in names if x[0] != '.']
if pattern[0] != ".":
names = [x for x in names if x[0] != "."]
return fnmatch.filter(names, pattern)


@@ -66,7 +67,7 @@ def glob0(dirname, basename):
return []


magic_check = re.compile('[*?[]')
magic_check = re.compile("[*?[]")


def has_magic(s):
56 changes: 30 additions & 26 deletions pipeline/jinja2/__init__.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
from jinja2 import nodes, TemplateSyntaxError
from jinja2.ext import Extension

from django.contrib.staticfiles.storage import staticfiles_storage
from jinja2 import TemplateSyntaxError, nodes
from jinja2.ext import Extension

from ..packager import PackageNotFound
from ..utils import guess_type
from ..templatetags.pipeline import PipelineMixin
from ..utils import guess_type


class PipelineExtension(PipelineMixin, Extension):
tags = set(['stylesheet', 'javascript'])
tags = {"stylesheet", "javascript"}

def parse(self, parser):
tag = next(parser.stream)
@@ -20,61 +19,66 @@ def parse(self, parser):

args = [package_name]
if tag.value == "stylesheet":
return nodes.CallBlock(self.call_method('package_css', args), [], [], []).set_lineno(tag.lineno)
return nodes.CallBlock(
self.call_method("package_css", args), [], [], []
).set_lineno(tag.lineno)

if tag.value == "javascript":
return nodes.CallBlock(self.call_method('package_js', args), [], [], []).set_lineno(tag.lineno)
return nodes.CallBlock(
self.call_method("package_js", args), [], [], []
).set_lineno(tag.lineno)

return []

def package_css(self, package_name, *args, **kwargs):
try:
package = self.package_for(package_name, 'css')
package = self.package_for(package_name, "css")
except PackageNotFound:
return '' # fail silently, do not return anything if an invalid group is specified
return self.render_compressed(package, package_name, 'css')
# fail silently, do not return anything if an invalid group is specified
return ""
return self.render_compressed(package, package_name, "css")

def render_css(self, package, path):
template_name = package.template_name or "pipeline/css.jinja"
context = package.extra_context
context.update({
'type': guess_type(path, 'text/css'),
'url': staticfiles_storage.url(path)
})
context.update(
{"type": guess_type(path, "text/css"), "url": staticfiles_storage.url(path)}
)
template = self.environment.get_template(template_name)
return template.render(context)

def render_individual_css(self, package, paths, **kwargs):
tags = [self.render_css(package, path) for path in paths]
return '\n'.join(tags)
return "\n".join(tags)

def package_js(self, package_name, *args, **kwargs):
try:
package = self.package_for(package_name, 'js')
package = self.package_for(package_name, "js")
except PackageNotFound:
return '' # fail silently, do not return anything if an invalid group is specified
return self.render_compressed(package, package_name, 'js')
# fail silently, do not return anything if an invalid group is specified
return ""
return self.render_compressed(package, package_name, "js")

def render_js(self, package, path):
template_name = package.template_name or "pipeline/js.jinja"
context = package.extra_context
context.update({
'type': guess_type(path, 'text/javascript'),
'url': staticfiles_storage.url(path)
})
context.update(
{
"type": guess_type(path, "text/javascript"),
"url": staticfiles_storage.url(path),
}
)
template = self.environment.get_template(template_name)
return template.render(context)

def render_inline(self, package, js):
context = package.extra_context
context.update({
'source': js
})
context.update({"source": js})
template = self.environment.get_template("pipeline/inline_js.jinja")
return template.render(context)

def render_individual_js(self, package, paths, templates=None):
tags = [self.render_js(package, js) for js in paths]
if templates:
tags.append(self.render_inline(package, templates))
return '\n'.join(tags)
return "\n".join(tags)
59 changes: 0 additions & 59 deletions pipeline/manifest.py

This file was deleted.

14 changes: 8 additions & 6 deletions pipeline/middleware.py
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
from django.core.exceptions import MiddlewareNotUsed
from django.utils.deprecation import MiddlewareMixin
from django.utils.encoding import DjangoUnicodeDecodeError
from django.utils.html import strip_spaces_between_tags as minify_html

from pipeline.conf import settings

from django.utils.deprecation import MiddlewareMixin


class MinifyHTMLMiddleware(MiddlewareMixin):
def __init__(self, *args, **kwargs):
super(MinifyHTMLMiddleware, self).__init__(*args, **kwargs)
super().__init__(*args, **kwargs)
if not settings.PIPELINE_ENABLED:
raise MiddlewareNotUsed

def process_response(self, request, response):
if response.has_header('Content-Type') and 'text/html' in response['Content-Type']:
if (
response.has_header("Content-Type")
and "text/html" in response["Content-Type"]
):
try:
response.content = minify_html(response.content.decode('utf-8').strip())
response['Content-Length'] = str(len(response.content))
response.content = minify_html(response.content.decode("utf-8").strip())
response["Content-Length"] = str(len(response.content))
except DjangoUnicodeDecodeError:
pass
return response
91 changes: 68 additions & 23 deletions pipeline/packager.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from django.contrib.staticfiles.finders import find, get_finders
from django.contrib.staticfiles.storage import staticfiles_storage
from django.contrib.staticfiles.finders import find
from django.core.files.base import ContentFile
from django.utils.encoding import smart_bytes

@@ -11,7 +11,7 @@
from pipeline.signals import css_compressed, js_compressed


class Package(object):
class Package:
def __init__(self, config):
self.config = config
self._sources = []
@@ -20,7 +20,7 @@ def __init__(self, config):
def sources(self):
if not self._sources:
paths = []
for pattern in self.config.get('source_filenames', []):
for pattern in self.config.get("source_filenames", []):
for path in glob(pattern):
if path not in paths and find(path):
paths.append(str(path))
@@ -29,41 +29,47 @@ def sources(self):

@property
def paths(self):
return [path for path in self.sources
if not path.endswith(settings.TEMPLATE_EXT)]
return [
path for path in self.sources if not path.endswith(settings.TEMPLATE_EXT)
]

@property
def templates(self):
return [path for path in self.sources
if path.endswith(settings.TEMPLATE_EXT)]
return [path for path in self.sources if path.endswith(settings.TEMPLATE_EXT)]

@property
def output_filename(self):
return self.config.get('output_filename')
return self.config.get("output_filename")

@property
def extra_context(self):
return self.config.get('extra_context', {})
return self.config.get("extra_context", {})

@property
def template_name(self):
return self.config.get('template_name')
return self.config.get("template_name")

@property
def variant(self):
return self.config.get('variant')
return self.config.get("variant")

@property
def manifest(self):
return self.config.get('manifest', True)
return self.config.get("manifest", True)

@property
def compiler_options(self):
return self.config.get('compiler_options', {})
return self.config.get("compiler_options", {})


class Packager(object):
def __init__(self, storage=None, verbose=False, css_packages=None, js_packages=None):
class Packager:
def __init__(
self,
storage=None,
verbose=False,
css_packages=None,
js_packages=None,
):
if storage is None:
storage = staticfiles_storage
self.storage = storage
@@ -75,16 +81,16 @@ def __init__(self, storage=None, verbose=False, css_packages=None, js_packages=N
if js_packages is None:
js_packages = settings.JAVASCRIPT
self.packages = {
'css': self.create_packages(css_packages),
'js': self.create_packages(js_packages),
"css": self.create_packages(css_packages),
"js": self.create_packages(js_packages),
}

def package_for(self, kind, package_name):
try:
return self.packages[kind][package_name]
except KeyError:
raise PackageNotFound(
"No corresponding package for %s package name : %s" % (
"No corresponding package for {} package name : {}".format(
kind, package_name
)
)
@@ -93,16 +99,39 @@ def individual_url(self, filename):
return self.storage.url(filename)

def pack_stylesheets(self, package, **kwargs):
return self.pack(package, self.compressor.compress_css, css_compressed,
output_filename=package.output_filename,
variant=package.variant, **kwargs)
return self.pack(
package,
self.compressor.compress_css,
css_compressed,
output_filename=package.output_filename,
variant=package.variant,
**kwargs,
)

def compile(self, paths, compiler_options={}, force=False):
return self.compiler.compile(
paths = self.compiler.compile(
paths,
compiler_options=compiler_options,
force=force,
)
for path in paths:
if not self.storage.exists(path):
if self.verbose:
e = (
"Compiled file '%s' cannot be "
"found with packager's storage. Locating it."
)
print(e % path)

source_storage = self.find_source_storage(path)
if source_storage is not None:
with source_storage.open(path) as source_file:
if self.verbose:
print(f"Saving: {path}")
self.storage.save(path, source_file)
else:
raise OSError(f"File does not exist: {path}")
return paths

def pack(self, package, compress, signal, **kwargs):
output_filename = package.output_filename
@@ -119,14 +148,30 @@ def pack(self, package, compress, signal, **kwargs):
return output_filename

def pack_javascripts(self, package, **kwargs):
return self.pack(package, self.compressor.compress_js, js_compressed, templates=package.templates, **kwargs)
return self.pack(
package,
self.compressor.compress_js,
js_compressed,
output_filename=package.output_filename,
templates=package.templates,
**kwargs,
)

def pack_templates(self, package):
return self.compressor.compile_templates(package.templates)

def save_file(self, path, content):
return self.storage.save(path, ContentFile(smart_bytes(content)))

def find_source_storage(self, path):
for finder in get_finders():
for short_path, storage in finder.list(""):
if short_path == path:
if self.verbose:
print(f"Found storage: {str(self.storage)}")
return storage
return None

def create_packages(self, config):
packages = {}
if not config:
1 change: 0 additions & 1 deletion pipeline/signals.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
from django.dispatch import Signal


css_compressed = Signal()
js_compressed = Signal()
51 changes: 28 additions & 23 deletions pipeline/storage.py
Original file line number Diff line number Diff line change
@@ -1,69 +1,72 @@
import gzip

from io import BytesIO

from django import get_version as django_version
from django.contrib.staticfiles.storage import (
ManifestStaticFilesStorage,
StaticFilesStorage,
)
from django.contrib.staticfiles.utils import matches_patterns
from django.core.files.base import File

_CACHED_STATIC_FILES_STORAGE_AVAILABLE = django_version() < '3.1'
_CACHED_STATIC_FILES_STORAGE_AVAILABLE = django_version() < "3.1"

if _CACHED_STATIC_FILES_STORAGE_AVAILABLE:
from django.contrib.staticfiles.storage import CachedStaticFilesStorage
from django.contrib.staticfiles.storage import ManifestStaticFilesStorage, StaticFilesStorage
from django.contrib.staticfiles.utils import matches_patterns

from django.core.files.base import File


class PipelineMixin(object):
class PipelineMixin:
packing = True

def post_process(self, paths, dry_run=False, **options):
if dry_run:
return

from pipeline.packager import Packager

packager = Packager(storage=self)
for package_name in packager.packages['css']:
package = packager.package_for('css', package_name)
for package_name in packager.packages["css"]:
package = packager.package_for("css", package_name)
output_file = package.output_filename
if self.packing:
packager.pack_stylesheets(package)
paths[output_file] = (self, output_file)
yield output_file, output_file, True
for package_name in packager.packages['js']:
package = packager.package_for('js', package_name)
for package_name in packager.packages["js"]:
package = packager.package_for("js", package_name)
output_file = package.output_filename
if self.packing:
packager.pack_javascripts(package)
paths[output_file] = (self, output_file)
yield output_file, output_file, True

super_class = super(PipelineMixin, self)
if hasattr(super_class, 'post_process'):
for name, hashed_name, processed in super_class.post_process(paths.copy(), dry_run, **options):
yield name, hashed_name, processed
super_class = super()
if hasattr(super_class, "post_process"):
yield from super_class.post_process(paths.copy(), dry_run, **options)

def get_available_name(self, name, max_length=None):
if self.exists(name):
self.delete(name)
return name


class GZIPMixin(object):
class GZIPMixin:
gzip_patterns = ("*.css", "*.js")

def _compress(self, original_file):
content = BytesIO()
gzip_file = gzip.GzipFile(mode='wb', fileobj=content)
gzip_file = gzip.GzipFile(mode="wb", fileobj=content)
gzip_file.write(original_file.read())
gzip_file.close()
content.seek(0)
return File(content)

def post_process(self, paths, dry_run=False, **options):
super_class = super(GZIPMixin, self)
if hasattr(super_class, 'post_process'):
for name, hashed_name, processed in super_class.post_process(paths.copy(), dry_run, **options):
super_class = super()
if hasattr(super_class, "post_process"):
for name, hashed_name, processed in super_class.post_process(
paths.copy(), dry_run, **options
):
if hashed_name != name:
paths[hashed_name] = (self, hashed_name)
yield name, hashed_name, processed
@@ -84,7 +87,7 @@ def post_process(self, paths, dry_run=False, **options):
yield gzipped_path, gzipped_path, True


class NonPackagingMixin(object):
class NonPackagingMixin:
packing = False


@@ -97,12 +100,12 @@ class NonPackagingPipelineStorage(NonPackagingMixin, PipelineStorage):


if _CACHED_STATIC_FILES_STORAGE_AVAILABLE:

class PipelineCachedStorage(PipelineMixin, CachedStaticFilesStorage):
# Deprecated since Django 2.2
# Removed in Django 3.1
pass


class NonPackagingPipelineCachedStorage(NonPackagingMixin, PipelineCachedStorage):
# Deprecated since Django 2.2
# Removed in Django 3.1
@@ -113,5 +116,7 @@ class PipelineManifestStorage(PipelineMixin, ManifestStaticFilesStorage):
pass


class NonPackagingPipelineManifestStorage(NonPackagingMixin, ManifestStaticFilesStorage):
class NonPackagingPipelineManifestStorage(
NonPackagingMixin, ManifestStaticFilesStorage
):
pass
2 changes: 1 addition & 1 deletion pipeline/templates/pipeline/compile_error.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<div id="django-pipeline-error-{{package_name}}" class="django-pipeline-error"
style="display: none; border: 2px #DD0000 solid; margin: 1em; padding: 1em; background: white;">
style="display: none; border: 2px #DD0000 solid; margin: 1em; padding: 1em; background: white; color: black;">
<h1>Error compiling {{package_type}} package "{{package_name}}"</h1>
<p><strong>Command:</strong></p>
<pre style="white-space: pre-wrap;">{{command}}</pre>
122 changes: 71 additions & 51 deletions pipeline/templatetags/pipeline.py
Original file line number Diff line number Diff line change
@@ -1,46 +1,46 @@
import logging
import re
import subprocess

from django.contrib.staticfiles.storage import staticfiles_storage

from django import template
from django.contrib.staticfiles.storage import staticfiles_storage
from django.template.base import VariableDoesNotExist
from django.template.loader import render_to_string
from django.utils.safestring import mark_safe

from ..collector import default_collector
from ..conf import settings
from ..exceptions import CompilerError
from ..packager import Packager, PackageNotFound
from ..packager import PackageNotFound, Packager
from ..utils import guess_type

logger = logging.getLogger(__name__)

register = template.Library()


class PipelineMixin(object):
class PipelineMixin:
request = None
_request_var = None

@property
def request_var(self):
if not self._request_var:
self._request_var = template.Variable('request')
self._request_var = template.Variable("request")
return self._request_var

def package_for(self, package_name, package_type):
package = {
'js': getattr(settings, 'JAVASCRIPT', {}).get(package_name, {}),
'css': getattr(settings, 'STYLESHEETS', {}).get(package_name, {}),
"js": getattr(settings, "JAVASCRIPT", {}).get(package_name, {}),
"css": getattr(settings, "STYLESHEETS", {}).get(package_name, {}),
}[package_type]

if package:
package = {package_name: package}

packager = {
'js': Packager(css_packages={}, js_packages=package),
'css': Packager(css_packages=package, js_packages={}),
"js": Packager(css_packages={}, js_packages=package),
"css": Packager(css_packages=package, js_packages={}),
}[package_type]

return packager.package_for(package_type, package_name)
@@ -63,19 +63,17 @@ def render_compressed(self, package, package_name, package_type):
determining what to render.
"""
if settings.PIPELINE_ENABLED:
return self.render_compressed_output(package, package_name,
package_type)
return self.render_compressed_output(package, package_name, package_type)
else:
return self.render_compressed_sources(package, package_name,
package_type)
return self.render_compressed_sources(package, package_name, package_type)

def render_compressed_output(self, package, package_name, package_type):
"""Render HTML for using the package's output file.
Subclasses can override this method to provide custom behavior for
rendering the output file.
"""
method = getattr(self, f'render_{package_type}')
method = getattr(self, f"render_{package_type}")

return method(package, package.output_filename)

@@ -96,13 +94,13 @@ def render_compressed_sources(self, package, package_name, package_type):
default_collector.collect(self.request)

packager = Packager()
method = getattr(self, f'render_individual_{package_type}')
method = getattr(self, f"render_individual_{package_type}")

try:
paths = packager.compile(package.paths)
except CompilerError as e:
if settings.SHOW_ERRORS_INLINE:
method = getattr(self, f'render_error_{package_type}')
method = getattr(self, f"render_error_{package_type}")
return method(package_name, e)
else:
raise
@@ -112,95 +110,113 @@ def render_compressed_sources(self, package, package_name, package_type):
return method(package, paths, templates=templates)

def render_error(self, package_type, package_name, e):
return render_to_string('pipeline/compile_error.html', {
'package_type': package_type,
'package_name': package_name,
'command': subprocess.list2cmdline(e.command),
'errors': e.error_output,
})
# Remove any ANSI escape sequences in the output.
error_output = re.sub(
re.compile(r"(?:\x1B[@-_]|[\x80-\x9F])[0-?]*[ -/]*[@-~]"),
"",
e.error_output,
)

return render_to_string(
"pipeline/compile_error.html",
{
"package_type": package_type,
"package_name": package_name,
"command": subprocess.list2cmdline(e.command),
"errors": error_output,
},
)


class StylesheetNode(PipelineMixin, template.Node):
def __init__(self, name):
self.name = name

def render(self, context):
super(StylesheetNode, self).render(context)
super().render(context)
package_name = template.Variable(self.name).resolve(context)

try:
package = self.package_for(package_name, 'css')
package = self.package_for(package_name, "css")
except PackageNotFound:
logger.warn("Package %r is unknown. Check PIPELINE['STYLESHEETS'] in your settings.", package_name)
return '' # fail silently, do not return anything if an invalid group is specified
return self.render_compressed(package, package_name, 'css')
w = "Package %r is unknown. Check PIPELINE['STYLESHEETS'] in your settings."
logger.warning(w, package_name)
# fail silently, do not return anything if an invalid group is specified
return ""
return self.render_compressed(package, package_name, "css")

def render_css(self, package, path):
template_name = package.template_name or "pipeline/css.html"
context = package.extra_context
context.update({
'type': guess_type(path, 'text/css'),
'url': mark_safe(staticfiles_storage.url(path))
})
context.update(
{
"type": guess_type(path, "text/css"),
"url": mark_safe(staticfiles_storage.url(path)),
}
)
return render_to_string(template_name, context)

def render_individual_css(self, package, paths, **kwargs):
tags = [self.render_css(package, path) for path in paths]
return '\n'.join(tags)
return "\n".join(tags)

def render_error_css(self, package_name, e):
return super(StylesheetNode, self).render_error(
'CSS', package_name, e)
return super().render_error("CSS", package_name, e)


class JavascriptNode(PipelineMixin, template.Node):
def __init__(self, name):
self.name = name

def render(self, context):
super(JavascriptNode, self).render(context)
super().render(context)
package_name = template.Variable(self.name).resolve(context)

try:
package = self.package_for(package_name, 'js')
package = self.package_for(package_name, "js")
except PackageNotFound:
logger.warn("Package %r is unknown. Check PIPELINE['JAVASCRIPT'] in your settings.", package_name)
return '' # fail silently, do not return anything if an invalid group is specified
return self.render_compressed(package, package_name, 'js')
w = "Package %r is unknown. Check PIPELINE['JAVASCRIPT'] in your settings."
logger.warning(w, package_name)
# fail silently, do not return anything if an invalid group is specified
return ""
return self.render_compressed(package, package_name, "js")

def render_js(self, package, path):
template_name = package.template_name or "pipeline/js.html"
context = package.extra_context
context.update({
'type': guess_type(path, 'text/javascript'),
'url': mark_safe(staticfiles_storage.url(path))
})
context.update(
{
"type": guess_type(path, "text/javascript"),
"url": mark_safe(staticfiles_storage.url(path)),
}
)
return render_to_string(template_name, context)

def render_inline(self, package, js):
context = package.extra_context
context.update({
'source': js
})
context.update({"source": js})
return render_to_string("pipeline/inline_js.html", context)

def render_individual_js(self, package, paths, templates=None):
tags = [self.render_js(package, js) for js in paths]
if templates:
tags.append(self.render_inline(package, templates))
return '\n'.join(tags)
return "\n".join(tags)

def render_error_js(self, package_name, e):
return super(JavascriptNode, self).render_error(
'JavaScript', package_name, e)
return super().render_error("JavaScript", package_name, e)


@register.tag
def stylesheet(parser, token):
try:
tag_name, name = token.split_contents()
except ValueError:
raise template.TemplateSyntaxError('%r requires exactly one argument: the name of a group in the PIPELINE.STYLESHEETS setting' % token.split_contents()[0])
e = (
"%r requires exactly one argument: the name "
"of a group in the PIPELINE.STYLESHEETS setting"
)
raise template.TemplateSyntaxError(e % token.split_contents()[0])
return StylesheetNode(name)


@@ -209,5 +225,9 @@ def javascript(parser, token):
try:
tag_name, name = token.split_contents()
except ValueError:
raise template.TemplateSyntaxError('%r requires exactly one argument: the name of a group in the PIPELINE.JAVASVRIPT setting' % token.split_contents()[0])
e = (
"%r requires exactly one argument: the name "
"of a group in the PIPELINE.JAVASVRIPT setting"
)
raise template.TemplateSyntaxError(e % token.split_contents()[0])
return JavascriptNode(name)
7 changes: 3 additions & 4 deletions pipeline/utils.py
Original file line number Diff line number Diff line change
@@ -6,10 +6,9 @@

import importlib
import mimetypes
import posixpath
import os
import posixpath
import sys

from urllib.parse import quote

from django.utils.encoding import smart_str
@@ -21,8 +20,8 @@ def to_class(class_str):
if not class_str:
return None

module_bits = class_str.split('.')
module_path, class_name = '.'.join(module_bits[:-1]), module_bits[-1]
module_bits = class_str.split(".")
module_path, class_name = ".".join(module_bits[:-1]), module_bits[-1]
module = importlib.import_module(module_path)
return getattr(module, class_name, None)

11 changes: 6 additions & 5 deletions pipeline/views.py
Original file line number Diff line number Diff line change
@@ -22,15 +22,16 @@ def serve_static(request, path, insecure=False, **kwargs):
# Follow the same logic Django uses for determining access to the
# static-serving view.
if not django_settings.DEBUG and not insecure:
raise ImproperlyConfigured("The staticfiles view can only be used in "
"debug mode or if the --insecure "
"option of 'runserver' is used")
raise ImproperlyConfigured(
"The staticfiles view can only be used in "
"debug mode or if the --insecure "
"option of 'runserver' is used"
)

if not settings.PIPELINE_ENABLED and settings.PIPELINE_COLLECTOR_ENABLED:
# Collect only the requested file, in order to serve the result as
# fast as possible. This won't interfere with the template tags in any
# way, as those will still cause Django to collect all media.
default_collector.collect(request, files=[path])

return serve(request, path, document_root=django_settings.STATIC_ROOT,
**kwargs)
return serve(request, path, document_root=django_settings.STATIC_ROOT, **kwargs)
81 changes: 81 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
[build-system]
requires = ["setuptools>=64", "setuptools_scm[toml]>=8"]
build-backend = "setuptools.build_meta"

[project]
name = "django-pipeline"
requires-python = ">=3.9"
version = "4.0.0"
description = "Pipeline is an asset packaging library for Django."
readme = "README.rst"
authors = [{ "name" = "Timothée Peignier", "email" = "timothee.peignier@tryphon.org" }]
license = { text = "MIT" }
classifiers = [
"Development Status :: 5 - Production/Stable",
"Environment :: Web Environment",
"Framework :: Django",
"Framework :: Django :: 4.0",
"Framework :: Django :: 4.1",
"Framework :: Django :: 4.2",
"Framework :: Django :: 5.0",
"Framework :: Django :: 5.1",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: Implementation :: PyPy",
"Topic :: Utilities",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Internet :: WWW/HTTP :: Dynamic Content",
]
keywords = [
"django",
"pipeline",
"asset",
"compiling",
"concatenation",
"compression",
"packaging",
]
dependencies = [
# indirect dependencies
"setuptools",
"wheel",
]

[project.optional-dependencies]
testing = [
"coveralls",
"tox",
"wheel",
"django",
]

[project.urls]
homepage = "https://github.com/jazzband/django-pipeline/"
documentation = "https://django-pipeline.readthedocs.io/"
repository = "https://github.com/jazzband/django-pipeline"
changelog = "https://github.com/jazzband/django-pipeline/blob/master/HISTORY.rst"

[tool.setuptools]
include-package-data = true

[tool.setuptools.packages.find]
exclude = ["tests", "tests.tests"]

[tool.setuptools_scm]
local_scheme = "dirty-tag"

[tool.black]
line-length = 88
target-version = ["py39"]

[tool.isort]
profile = "black"
5 changes: 0 additions & 5 deletions requirements.txt

This file was deleted.

2 changes: 0 additions & 2 deletions setup.cfg

This file was deleted.

46 changes: 0 additions & 46 deletions setup.py

This file was deleted.

8 changes: 6 additions & 2 deletions tests/assets/compilers/scss/expected.css
4 changes: 4 additions & 0 deletions tests/assets/compilers/typescript/expected.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
function getName(u) {
return "".concat(u.firstName, " ").concat(u.lastName);
}
var userName = getName({ firstName: "Django", lastName: "Pipeline" });
13 changes: 13 additions & 0 deletions tests/assets/compilers/typescript/input.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
type FullName = string;

interface User {
firstName: string;
lastName: string;
}


function getName(u: User): FullName {
return `${u.firstName} ${u.lastName}`;
}

let userName: FullName = getName({firstName: "Django", lastName: "Pipeline"});
24 changes: 24 additions & 0 deletions tests/assets/css/sourcemap.css
18 changes: 18 additions & 0 deletions tests/assets/js/sourcemap.js
322 changes: 166 additions & 156 deletions tests/settings.py
Original file line number Diff line number Diff line change
@@ -1,227 +1,237 @@
import glob
import os
import distutils.spawn
import shutil


def local_path(path):
return os.path.join(os.path.dirname(__file__), path)


DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'TEST_NAME': ':memory:'
}
"default": {"ENGINE": "django.db.backends.sqlite3", "TEST_NAME": ":memory:"}
}

DEBUG = False

SITE_ID = 1

INSTALLED_APPS = [
'django.contrib.contenttypes',
'django.contrib.messages',
'django.contrib.sites',
'django.contrib.sessions',
'django.contrib.staticfiles',
'django.contrib.auth',
'django.contrib.admin',
'pipeline',
'tests.tests'
"django.contrib.contenttypes",
"django.contrib.messages",
"django.contrib.sites",
"django.contrib.sessions",
"django.contrib.staticfiles",
"django.contrib.auth",
"django.contrib.admin",
"pipeline",
"tests.tests",
]


ROOT_URLCONF = 'tests.urls'
ROOT_URLCONF = "tests.urls"

MIDDLEWARE = [
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
"django.contrib.sessions.middleware.SessionMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.common.CommonMiddleware",
"django.middleware.csrf.CsrfViewMiddleware",
]

MEDIA_URL = '/media/'
MEDIA_URL = "/media/"

MEDIA_ROOT = local_path('media')
MEDIA_ROOT = local_path("media")

STATICFILES_STORAGE = 'pipeline.storage.PipelineStorage'
STATIC_ROOT = local_path('static/')
STATIC_URL = '/static/'
STATICFILES_DIRS = (
('pipeline', local_path('assets/')),
)
STATICFILES_STORAGE = "pipeline.storage.PipelineStorage"
STATIC_ROOT = local_path("static/")
STATIC_URL = "/static/"
STATICFILES_DIRS = (("pipeline", local_path("assets/")),)
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'pipeline.finders.PipelineFinder',
"django.contrib.staticfiles.finders.FileSystemFinder",
"django.contrib.staticfiles.finders.AppDirectoriesFinder",
"pipeline.finders.PipelineFinder",
)

SECRET_KEY = "django-pipeline"

PIPELINE = {
'PIPELINE_ENABLED': True,
'JS_COMPRESSOR': None,
'CSS_COMPRESSOR': None,
'STYLESHEETS': {
'screen': {
'source_filenames': (
'pipeline/css/first.css',
'pipeline/css/second.css',
'pipeline/css/urls.css'
"PIPELINE_ENABLED": True,
"JS_COMPRESSOR": None,
"CSS_COMPRESSOR": None,
"STYLESHEETS": {
"screen": {
"source_filenames": (
"pipeline/css/first.css",
"pipeline/css/second.css",
"pipeline/css/urls.css",
),
'output_filename': 'screen.css'
"output_filename": "screen.css",
},
'screen_media': {
'source_filenames': (
'pipeline/css/first.css',
'pipeline/css/second.css',
'pipeline/css/urls.css'
"screen_media": {
"source_filenames": (
"pipeline/css/first.css",
"pipeline/css/second.css",
"pipeline/css/urls.css",
),
'output_filename': 'screen_media.css',
'extra_context': {
'media': 'screen and (min-width:500px)',
"output_filename": "screen_media.css",
"extra_context": {
"media": "screen and (min-width:500px)",
},
},
'screen_title': {
'source_filenames': (
'pipeline/css/first.css',
'pipeline/css/second.css',
'pipeline/css/urls.css'
"screen_title": {
"source_filenames": (
"pipeline/css/first.css",
"pipeline/css/second.css",
"pipeline/css/urls.css",
),
'output_filename': 'screen_title.css',
'extra_context': {
'title': 'Default Style',
"output_filename": "screen_title.css",
"extra_context": {
"title": "Default Style",
},
}
},
},
'JAVASCRIPT': {
'scripts': {
'source_filenames': (
'pipeline/js/first.js',
'pipeline/js/second.js',
'pipeline/js/application.js',
'pipeline/templates/**/*.jst'
"JAVASCRIPT": {
"scripts": {
"source_filenames": (
"pipeline/js/first.js",
"pipeline/js/second.js",
"pipeline/js/application.js",
"pipeline/templates/**/*.jst",
),
'output_filename': 'scripts.js'
"output_filename": "scripts.js",
},
'scripts_async': {
'source_filenames': (
'pipeline/js/first.js',
'pipeline/js/second.js',
'pipeline/js/application.js',
'pipeline/templates/**/*.jst'
"scripts_async": {
"source_filenames": (
"pipeline/js/first.js",
"pipeline/js/second.js",
"pipeline/js/application.js",
"pipeline/templates/**/*.jst",
),
'output_filename': 'scripts_async.js',
'extra_context': {
'async': True,
}
"output_filename": "scripts_async.js",
"extra_context": {
"async": True,
},
},
'scripts_defer': {
'source_filenames': (
'pipeline/js/first.js',
'pipeline/js/second.js',
'pipeline/js/application.js',
'pipeline/templates/**/*.jst'
"scripts_defer": {
"source_filenames": (
"pipeline/js/first.js",
"pipeline/js/second.js",
"pipeline/js/application.js",
"pipeline/templates/**/*.jst",
),
'output_filename': 'scripts_defer.js',
'extra_context': {
'defer': True,
}
"output_filename": "scripts_defer.js",
"extra_context": {
"defer": True,
},
},
'scripts_async_defer': {
'source_filenames': (
'pipeline/js/first.js',
'pipeline/js/second.js',
'pipeline/js/application.js',
'pipeline/templates/**/*.jst'
"scripts_async_defer": {
"source_filenames": (
"pipeline/js/first.js",
"pipeline/js/second.js",
"pipeline/js/application.js",
"pipeline/templates/**/*.jst",
),
'output_filename': 'scripts_async_defer.js',
'extra_context': {
'async': True,
'defer': True,
}
}
}
"output_filename": "scripts_async_defer.js",
"extra_context": {
"async": True,
"defer": True,
},
},
},
}

NODE_MODULES_PATH = local_path('../node_modules')
NODE_BIN_PATH = os.path.join(NODE_MODULES_PATH, '.bin')
NODE_EXE_PATH = distutils.spawn.find_executable('node')
JAVA_EXE_PATH = distutils.spawn.find_executable('java')
CSSTIDY_EXE_PATH = distutils.spawn.find_executable('csstidy')
NODE_MODULES_PATH = local_path("../node_modules")
NODE_BIN_PATH = os.path.join(NODE_MODULES_PATH, ".bin")
NODE_EXE_PATH = shutil.which("node")
JAVA_EXE_PATH = shutil.which("java")
CSSTIDY_EXE_PATH = shutil.which("csstidy")
HAS_NODE = bool(NODE_EXE_PATH)
HAS_JAVA = bool(JAVA_EXE_PATH)
HAS_CSSTIDY = bool(CSSTIDY_EXE_PATH)

if HAS_NODE:

def node_exe_path(command):
exe_ext = '.cmd' if os.name == 'nt' else ''
return os.path.join(NODE_BIN_PATH, "%s%s" % (command, exe_ext))

PIPELINE.update({
'SASS_BINARY': node_exe_path('node-sass'),
'COFFEE_SCRIPT_BINARY': node_exe_path('coffee'),
'COFFEE_SCRIPT_ARGUMENTS': ['--no-header'],
'LESS_BINARY': node_exe_path('lessc'),
'BABEL_BINARY': node_exe_path('babel'),
'BABEL_ARGUMENTS': ['--presets', 'es2015'],
'STYLUS_BINARY': node_exe_path('stylus'),
'LIVE_SCRIPT_BINARY': node_exe_path('lsc'),
'LIVE_SCRIPT_ARGUMENTS': ['--no-header'],
'YUGLIFY_BINARY': node_exe_path('yuglify'),
'UGLIFYJS_BINARY': node_exe_path('uglifyjs'),
'TERSER_BINARY': node_exe_path('terser'),
'CSSMIN_BINARY': node_exe_path('cssmin'),
})
exe_ext = ".cmd" if os.name == "nt" else ""
return os.path.join(NODE_BIN_PATH, "{}{}".format(command, exe_ext))

PIPELINE.update(
{
"SASS_BINARY": node_exe_path("sass"),
"COFFEE_SCRIPT_BINARY": node_exe_path("coffee"),
"COFFEE_SCRIPT_ARGUMENTS": ["--no-header"],
"LESS_BINARY": node_exe_path("lessc"),
"BABEL_BINARY": node_exe_path("babel"),
"BABEL_ARGUMENTS": ["--presets", "es2015"],
"STYLUS_BINARY": node_exe_path("stylus"),
"LIVE_SCRIPT_BINARY": node_exe_path("lsc"),
"LIVE_SCRIPT_ARGUMENTS": ["--no-header"],
"YUGLIFY_BINARY": node_exe_path("yuglify"),
"UGLIFYJS_BINARY": node_exe_path("uglifyjs"),
"TERSER_BINARY": node_exe_path("terser"),
"CSSMIN_BINARY": node_exe_path("cssmin"),
"TYPE_SCRIPT_BINARY": node_exe_path("tsc"),
}
)

if HAS_NODE and HAS_JAVA:
PIPELINE.update({
'CLOSURE_BINARY': [
JAVA_EXE_PATH, '-jar',
os.path.join(NODE_MODULES_PATH, 'google-closure-compiler-java', 'compiler.jar')],
'YUI_BINARY': [
JAVA_EXE_PATH, '-jar',
glob.glob(os.path.join(NODE_MODULES_PATH, 'yuicompressor', 'build', '*.jar'))[0]]
})
PIPELINE.update(
{
"CLOSURE_BINARY": [
JAVA_EXE_PATH,
"-jar",
os.path.join(
NODE_MODULES_PATH,
"google-closure-compiler-java",
"compiler.jar",
),
],
"YUI_BINARY": [
JAVA_EXE_PATH,
"-jar",
glob.glob(
os.path.join(NODE_MODULES_PATH, "yuicompressor", "build", "*.jar")
)[0],
],
}
)

if HAS_CSSTIDY:
PIPELINE.update({'CSSTIDY_BINARY': CSSTIDY_EXE_PATH})
PIPELINE.update({"CSSTIDY_BINARY": CSSTIDY_EXE_PATH})

TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'APP_DIRS': True,
'DIRS': [local_path('templates')],
'OPTIONS': {
'context_processors': [
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
"BACKEND": "django.template.backends.django.DjangoTemplates",
"APP_DIRS": True,
"DIRS": [local_path("templates")],
"OPTIONS": {
"context_processors": [
"django.template.context_processors.request",
"django.contrib.auth.context_processors.auth",
"django.contrib.messages.context_processors.messages",
]
}
},
},
{
'BACKEND': 'django.template.backends.jinja2.Jinja2',
'APP_DIRS': True,
'DIRS': [local_path('templates')],
'OPTIONS': {
'extensions': ['pipeline.jinja2.PipelineExtension']
}
}
"BACKEND": "django.template.backends.jinja2.Jinja2",
"APP_DIRS": True,
"DIRS": [local_path("templates")],
"OPTIONS": {"extensions": ["pipeline.jinja2.PipelineExtension"]},
},
]

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
"version": 1,
"disable_existing_loggers": False,
"handlers": {
"console": {
"class": "logging.StreamHandler",
},
},
'loggers': {
'pipeline.templatetags.pipeline': {
'handlers': ['console'],
'level': 'ERROR',
"loggers": {
"pipeline.templatetags.pipeline": {
"handlers": ["console"],
"level": "ERROR",
},
},
}
26 changes: 12 additions & 14 deletions tests/tests/__init__.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
# -*- coding: utf-8 flake8: noqa -*-
import os
import sys

if sys.platform.startswith("win"):
os.environ.setdefault("NUMBER_OF_PROCESSORS", "1")

if sys.platform.startswith('win'):
os.environ.setdefault('NUMBER_OF_PROCESSORS', '1')


from .test_collector import *
from .test_compiler import *
from .test_compressor import *
from .test_template import *
from .test_glob import *
from .test_middleware import *
from .test_packager import *
from .test_storage import *
from .test_utils import *
from .test_views import *
from .test_collector import * # noqa
from .test_compiler import * # noqa
from .test_compressor import * # noqa
from .test_glob import * # noqa
from .test_middleware import * # noqa
from .test_packager import * # noqa
from .test_storage import * # noqa
from .test_template import * # noqa
from .test_utils import * # noqa
from .test_views import * # noqa
62 changes: 38 additions & 24 deletions tests/tests/test_collector.py
Original file line number Diff line number Diff line change
@@ -9,54 +9,68 @@


def local_path(path):
return os.path.abspath(os.path.join(os.path.dirname(__file__), '..', path))
return os.path.abspath(os.path.join(os.path.dirname(__file__), "..", path))


class CollectorTest(TestCase):
def tearDown(self):
super(CollectorTest, self).tearDown()
super().tearDown()

default_collector.clear()

def test_collect(self):
self.assertEqual(
set(default_collector.collect()),
set(self._get_collectable_files()))
set(default_collector.collect()), set(self._get_collectable_files())
)

def test_collect_with_files(self):
self.assertEqual(
set(default_collector.collect(files=[
'pipeline/js/first.js',
'pipeline/js/second.js',
])),
set([
'pipeline/js/first.js',
'pipeline/js/second.js',
]))
set(
default_collector.collect(
files=[
"pipeline/js/first.js",
"pipeline/js/second.js",
]
)
),
{
"pipeline/js/first.js",
"pipeline/js/second.js",
},
)

def test_delete_file_with_modified(self):
list(default_collector.collect())

storage = FileSystemStorage(local_path('assets'))
new_mtime = os.path.getmtime(storage.path('js/first.js')) - 1000
os.utime(default_collector.storage.path('pipeline/js/first.js'),
(new_mtime, new_mtime))
storage = FileSystemStorage(local_path("assets"))
new_mtime = os.path.getmtime(storage.path("js/first.js")) - 1000
os.utime(
default_collector.storage.path("pipeline/js/first.js"),
(new_mtime, new_mtime),
)

self.assertTrue(default_collector.delete_file(
'js/first.js', 'pipeline/js/first.js', storage))
self.assertTrue(
default_collector.delete_file(
"js/first.js", "pipeline/js/first.js", storage
)
)

def test_delete_file_with_unmodified(self):
list(default_collector.collect(files=['pipeline/js/first.js']))
list(default_collector.collect(files=["pipeline/js/first.js"]))

self.assertFalse(default_collector.delete_file(
'js/first.js', 'pipeline/js/first.js',
FileSystemStorage(local_path('assets'))))
self.assertFalse(
default_collector.delete_file(
"js/first.js",
"pipeline/js/first.js",
FileSystemStorage(local_path("assets")),
)
)

def _get_collectable_files(self):
for finder in finders.get_finders():
if not isinstance(finder, PipelineFinder):
for path, storage in finder.list(['CVS', '.*', '*~']):
if getattr(storage, 'prefix', None):
for path, storage in finder.list(["CVS", ".*", "*~"]):
if getattr(storage, "prefix", None):
yield os.path.join(storage.prefix, path)
else:
yield path
188 changes: 115 additions & 73 deletions tests/tests/test_compiler.py
Original file line number Diff line number Diff line change
@@ -10,93 +10,102 @@
from pipeline.compilers import Compiler, CompilerBase, SubProcessCompiler
from pipeline.exceptions import CompilerError
from pipeline.utils import to_class

from tests.utils import _, pipeline_settings


class FailingCompiler(SubProcessCompiler):
output_extension = 'junk'
output_extension = "junk"

def match_file(self, path):
return path.endswith('.coffee')
return path.endswith(".coffee")

def compile_file(self, infile, outfile, outdated=False, force=False):
command = (("/usr/bin/env", "false",),)
command = (
(
"/usr/bin/env",
"false",
),
)
return self.execute_command(command)


class InvalidCompiler(SubProcessCompiler):
output_extension = 'junk'
output_extension = "junk"

def match_file(self, path):
return path.endswith('.coffee')
return path.endswith(".coffee")

def compile_file(self, infile, outfile, outdated=False, force=False):
command = (
("this-exists-nowhere-as-a-command-and-should-fail",),
infile,
outfile
outfile,
)
return self.execute_command(command)


class CompilerWithEmptyFirstArg(SubProcessCompiler):
output_extension = 'junk'
output_extension = "junk"

def match_file(self, path):
return path.endswith('.coffee')
return path.endswith(".coffee")

def compile_file(self, infile, outfile, outdated=False, force=False):
command = (
('', '/usr/bin/env', 'cat'),
("", "/usr/bin/env", "cat"),
infile,
)
return self.execute_command(command, stdout_captured=outfile)


class CopyingCompiler(SubProcessCompiler):
output_extension = 'junk'
output_extension = "junk"

def match_file(self, path):
return path.endswith('.coffee')
return path.endswith(".coffee")

def compile_file(self, infile, outfile, outdated=False, force=False):
command = (
"cp",
infile,
outfile
)
command = ("cp", infile, outfile)
return self.execute_command(command)


class LineNumberingCompiler(SubProcessCompiler):
output_extension = 'junk'
output_extension = "junk"

def match_file(self, path):
return path.endswith('.coffee')
return path.endswith(".coffee")

def compile_file(self, infile, outfile, outdated=False, force=False):
command = (("/usr/bin/env", "cat"), ("-n",), infile,)
command = (
("/usr/bin/env", "cat"),
("-n",),
infile,
)
return self.execute_command(command, stdout_captured=outfile)


class DummyCompiler(CompilerBase):
output_extension = 'js'
output_extension = "js"

def match_file(self, path):
return path.endswith('.coffee')
return path.endswith(".coffee")

def compile_file(self, infile, outfile, outdated=False, force=False):
return


@pipeline_settings(COMPILERS=['tests.tests.test_compiler.DummyCompiler'])
@pipeline_settings(COMPILERS=["tests.tests.test_compiler.DummyCompiler"])
class DummyCompilerTest(TestCase):
def setUp(self):
default_collector.collect()
self.compiler = Compiler()

def test_output_path(self):
compiler_class = self.compiler.compilers[0]
compiler = compiler_class(verbose=self.compiler.verbose, storage=self.compiler.storage)
compiler = compiler_class(
verbose=self.compiler.verbose,
storage=self.compiler.storage,
)
output_path = compiler.output_path("js/helpers.coffee", "js")
self.assertEqual(output_path, "js/helpers.js")

@@ -105,119 +114,133 @@ def test_compilers_class(self):
self.assertEqual(compilers_class[0], DummyCompiler)

def test_compile(self):
paths = self.compiler.compile([
_('pipeline/js/dummy.coffee'),
_('pipeline/js/application.js'),
])
self.assertEqual([_('pipeline/js/dummy.js'), _('pipeline/js/application.js')], list(paths))
paths = self.compiler.compile(
[
_("pipeline/js/dummy.coffee"),
_("pipeline/js/application.js"),
]
)
self.assertEqual(
[_("pipeline/js/dummy.js"), _("pipeline/js/application.js")],
list(paths),
)

def tearDown(self):
default_collector.clear()


@skipIf(sys.platform.startswith("win"), "requires posix platform")
@pipeline_settings(COMPILERS=['tests.tests.test_compiler.LineNumberingCompiler'])
@pipeline_settings(COMPILERS=["tests.tests.test_compiler.LineNumberingCompiler"])
class CompilerStdoutTest(TestCase):
def setUp(self):
default_collector.collect()
self.compiler = Compiler()

def test_output_path(self):
compiler_class = self.compiler.compilers[0]
compiler = compiler_class(verbose=self.compiler.verbose, storage=self.compiler.storage)
compiler = compiler_class(
verbose=self.compiler.verbose,
storage=self.compiler.storage,
)
output_path = compiler.output_path("js/helpers.coffee", "js")
self.assertEqual(output_path, "js/helpers.js")

def test_compile(self):
paths = self.compiler.compile([_('pipeline/js/dummy.coffee')])
self.assertEqual([_('pipeline/js/dummy.junk')], list(paths))
paths = self.compiler.compile([_("pipeline/js/dummy.coffee")])
self.assertEqual([_("pipeline/js/dummy.junk")], list(paths))

def tearDown(self):
default_collector.clear()


@skipIf(sys.platform.startswith("win"), "requires posix platform")
@pipeline_settings(COMPILERS=['tests.tests.test_compiler.CopyingCompiler'])
@pipeline_settings(COMPILERS=["tests.tests.test_compiler.CopyingCompiler"])
class CompilerSelfWriterTest(TestCase):
def setUp(self):
default_collector.collect()
self.compiler = Compiler()

def test_output_path(self):
compiler_class = self.compiler.compilers[0]
compiler = compiler_class(verbose=self.compiler.verbose, storage=self.compiler.storage)
compiler = compiler_class(
verbose=self.compiler.verbose,
storage=self.compiler.storage,
)
output_path = compiler.output_path("js/helpers.coffee", "js")
self.assertEqual(output_path, "js/helpers.js")

def test_compile(self):
paths = self.compiler.compile([_('pipeline/js/dummy.coffee')])
paths = self.compiler.compile([_("pipeline/js/dummy.coffee")])
default_collector.collect()
self.assertEqual([_('pipeline/js/dummy.junk')], list(paths))
self.assertEqual([_("pipeline/js/dummy.junk")], list(paths))

def tearDown(self):
default_collector.clear()


@pipeline_settings(COMPILERS=['tests.tests.test_compiler.CompilerWithEmptyFirstArg'])
@pipeline_settings(COMPILERS=["tests.tests.test_compiler.CompilerWithEmptyFirstArg"])
class CompilerWithEmptyFirstArgTest(TestCase):
def setUp(self):
default_collector.collect()
self.compiler = Compiler()

def test_compile(self):
paths = self.compiler.compile([_('pipeline/js/dummy.coffee')])
default_collector.collect()
self.assertEqual([_('pipeline/js/dummy.junk')], list(paths))
paths = self.compiler.compile([_("pipeline/js/dummy.coffee")])
default_collector.collect()
self.assertEqual([_("pipeline/js/dummy.junk")], list(paths))

def tearDown(self):
default_collector.clear()

@pipeline_settings(COMPILERS=['tests.tests.test_compiler.InvalidCompiler'])

@pipeline_settings(COMPILERS=["tests.tests.test_compiler.InvalidCompiler"])
class InvalidCompilerTest(TestCase):
def setUp(self):
default_collector.collect()
self.compiler = Compiler()

def test_compile(self):
with self.assertRaises(CompilerError) as cm:
self.compiler.compile([_('pipeline/js/dummy.coffee')])
self.compiler.compile([_("pipeline/js/dummy.coffee")])
e = cm.exception
self.assertEqual(
e.command,
['this-exists-nowhere-as-a-command-and-should-fail',
'pipeline/js/dummy.coffee',
'pipeline/js/dummy.junk'])
self.assertEqual(e.error_output, '')
[
"this-exists-nowhere-as-a-command-and-should-fail",
"pipeline/js/dummy.coffee",
"pipeline/js/dummy.junk",
],
)
self.assertEqual(e.error_output, "")

def tearDown(self):
default_collector.clear()


@skipIf(sys.platform.startswith("win"), "requires posix platform")
@pipeline_settings(COMPILERS=['tests.tests.test_compiler.FailingCompiler'])
@pipeline_settings(COMPILERS=["tests.tests.test_compiler.FailingCompiler"])
class FailingCompilerTest(TestCase):
def setUp(self):
default_collector.collect()
self.compiler = Compiler()

def test_compile(self):
with self.assertRaises(CompilerError) as cm:
self.compiler.compile([_('pipeline/js/dummy.coffee')])
self.compiler.compile([_("pipeline/js/dummy.coffee")])

e = cm.exception
self.assertEqual(e.command, ['/usr/bin/env', 'false'])
self.assertEqual(e.error_output, '')
self.assertEqual(e.command, ["/usr/bin/env", "false"])
self.assertEqual(e.error_output, "")

def tearDown(self):
default_collector.clear()


@skipUnless(settings.HAS_NODE, "requires node")
class CompilerImplementation(TestCase):

def setUp(self):
self.compiler = Compiler()
default_collector.collect(RequestFactory().get('/'))
default_collector.collect(RequestFactory().get("/"))

def tearDown(self):
default_collector.clear()
@@ -228,38 +251,57 @@ def _test_compiler(self, compiler_cls_str, infile, expected):
infile_path = staticfiles_storage.path(infile)
outfile_path = compiler.output_path(infile_path, compiler.output_extension)
compiler.compile_file(_(infile_path), _(outfile_path), force=True)
with open(outfile_path, 'r') as f:
with open(outfile_path) as f:
result = f.read()
with staticfiles_storage.open(expected, 'r') as f:
with staticfiles_storage.open(expected, "r") as f:
expected = f.read()
self.assertEqual(result, expected)

def test_sass(self):
self._test_compiler('pipeline.compilers.sass.SASSCompiler',
'pipeline/compilers/scss/input.scss',
'pipeline/compilers/scss/expected.css')
self._test_compiler(
"pipeline.compilers.sass.SASSCompiler",
"pipeline/compilers/scss/input.scss",
"pipeline/compilers/scss/expected.css",
)

def test_coffeescript(self):
self._test_compiler('pipeline.compilers.coffee.CoffeeScriptCompiler',
'pipeline/compilers/coffee/input.coffee',
'pipeline/compilers/coffee/expected.js')
self._test_compiler(
"pipeline.compilers.coffee.CoffeeScriptCompiler",
"pipeline/compilers/coffee/input.coffee",
"pipeline/compilers/coffee/expected.js",
)

def test_less(self):
self._test_compiler('pipeline.compilers.less.LessCompiler',
'pipeline/compilers/less/input.less',
'pipeline/compilers/less/expected.css')
self._test_compiler(
"pipeline.compilers.less.LessCompiler",
"pipeline/compilers/less/input.less",
"pipeline/compilers/less/expected.css",
)

def test_es6(self):
self._test_compiler('pipeline.compilers.es6.ES6Compiler',
'pipeline/compilers/es6/input.es6',
'pipeline/compilers/es6/expected.js')
self._test_compiler(
"pipeline.compilers.es6.ES6Compiler",
"pipeline/compilers/es6/input.es6",
"pipeline/compilers/es6/expected.js",
)

def test_typescript(self):
self._test_compiler(
"pipeline.compilers.typescript.TypeScriptCompiler",
"pipeline/compilers/typescript/input.ts",
"pipeline/compilers/typescript/expected.js",
)

def test_stylus(self):
self._test_compiler('pipeline.compilers.stylus.StylusCompiler',
'pipeline/compilers/stylus/input.styl',
'pipeline/compilers/stylus/expected.css')
self._test_compiler(
"pipeline.compilers.stylus.StylusCompiler",
"pipeline/compilers/stylus/input.styl",
"pipeline/compilers/stylus/expected.css",
)

def test_livescript(self):
self._test_compiler('pipeline.compilers.livescript.LiveScriptCompiler',
'pipeline/compilers/livescript/input.ls',
'pipeline/compilers/livescript/expected.js')
self._test_compiler(
"pipeline.compilers.livescript.LiveScriptCompiler",
"pipeline/compilers/livescript/input.ls",
"pipeline/compilers/livescript/expected.js",
)
582 changes: 462 additions & 120 deletions tests/tests/test_compressor.py

Large diffs are not rendered by default.

16 changes: 8 additions & 8 deletions tests/tests/test_conf.py
Original file line number Diff line number Diff line change
@@ -9,30 +9,30 @@
class TestSettings(TestCase):
def test_3unicode(self):
s = PipelineSettings({"FOO_BINARY": "env actualprogram"})
self.assertEqual(s.FOO_BINARY, ('env', 'actualprogram'))
self.assertEqual(s.FOO_BINARY, ("env", "actualprogram"))

def test_2unicode(self):
s = PipelineSettings({"FOO_BINARY": u"env actualprogram"})
self.assertEqual(s.FOO_BINARY, ('env', 'actualprogram'))
s = PipelineSettings({"FOO_BINARY": "env actualprogram"})
self.assertEqual(s.FOO_BINARY, ("env", "actualprogram"))

def test_2bytes(self):
s = PipelineSettings({"FOO_BINARY": "env actualprogram"})
self.assertEqual(s.FOO_BINARY, ('env', 'actualprogram'))
self.assertEqual(s.FOO_BINARY, ("env", "actualprogram"))

def test_expected_splitting(self):
s = PipelineSettings({"FOO_BINARY": "env actualprogram"})
self.assertEqual(s.FOO_BINARY, ('env', 'actualprogram'))
self.assertEqual(s.FOO_BINARY, ("env", "actualprogram"))

@skipIf(sys.platform.startswith("win"), "requires posix platform")
def test_expected_preservation(self):
s = PipelineSettings({"FOO_BINARY": r"actual\ program"})
self.assertEqual(s.FOO_BINARY, ('actual program',))
self.assertEqual(s.FOO_BINARY, ("actual program",))

@skipUnless(sys.platform.startswith("win"), "requires windows")
def test_win_path_preservation(self):
s = PipelineSettings({"FOO_BINARY": "C:\\Test\\ActualProgram.exe argument"})
self.assertEqual(s.FOO_BINARY, ('C:\\Test\\ActualProgram.exe', 'argument'))
self.assertEqual(s.FOO_BINARY, ("C:\\Test\\ActualProgram.exe", "argument"))

def test_tuples_are_normal(self):
s = PipelineSettings({"FOO_ARGUMENTS": ("explicit", "with", "args")})
self.assertEqual(s.FOO_ARGUMENTS, ('explicit', 'with', 'args'))
self.assertEqual(s.FOO_ARGUMENTS, ("explicit", "with", "args"))
206 changes: 106 additions & 100 deletions tests/tests/test_forms.py
Original file line number Diff line number Diff line change
@@ -3,203 +3,209 @@
from django.test import TestCase

from pipeline.forms import PipelineFormMedia

from ..utils import pipeline_settings


@pipeline_settings(
PIPELINE_COLLECTOR_ENABLED=False,
STYLESHEETS={
'styles1': {
'source_filenames': (
'pipeline/css/first.css',
'pipeline/css/second.css',
"styles1": {
"source_filenames": (
"pipeline/css/first.css",
"pipeline/css/second.css",
),
'output_filename': 'styles1.min.css',
"output_filename": "styles1.min.css",
},
'styles2': {
'source_filenames': (
'pipeline/css/unicode.css',
),
'output_filename': 'styles2.min.css',
"styles2": {
"source_filenames": ("pipeline/css/unicode.css",),
"output_filename": "styles2.min.css",
},
'print': {
'source_filenames': (
'pipeline/css/urls.css',
),
'output_filename': 'print.min.css',
"print": {
"source_filenames": ("pipeline/css/urls.css",),
"output_filename": "print.min.css",
},
},
JAVASCRIPT={
'scripts1': {
'source_filenames': (
'pipeline/js/first.js',
'pipeline/js/second.js',
"scripts1": {
"source_filenames": (
"pipeline/js/first.js",
"pipeline/js/second.js",
),
'output_filename': 'scripts1.min.js',
"output_filename": "scripts1.min.js",
},
'scripts2': {
'source_filenames': (
'pipeline/js/application.js',
),
'output_filename': 'scripts2.min.js',
"scripts2": {
"source_filenames": ("pipeline/js/application.js",),
"output_filename": "scripts2.min.js",
},
})
},
)
class PipelineFormMediaTests(TestCase):
"""Unit tests for pipeline.forms.PipelineFormMedia."""

@pipeline_settings(PIPELINE_ENABLED=True)
def test_css_packages_with_pipeline_enabled(self):
"""Testing PipelineFormMedia.css_packages with PIPELINE_ENABLED=True"""

class MyMedia(PipelineFormMedia):
css_packages = {
'all': ('styles1', 'styles2'),
'print': ('print',),
"all": ("styles1", "styles2"),
"print": ("print",),
}

css = {
'all': ('extra1.css', 'extra2.css')
}
css = {"all": ("extra1.css", "extra2.css")}

media = Media(MyMedia)

self.assertEqual(
MyMedia.css,
{
'all': [
'extra1.css',
'extra2.css',
'/static/styles1.min.css',
'/static/styles2.min.css',
"all": [
"extra1.css",
"extra2.css",
"/static/styles1.min.css",
"/static/styles2.min.css",
],
'print': ['/static/print.min.css'],
})
"print": ["/static/print.min.css"],
},
)
self.assertEqual(MyMedia.css, media._css)
expected_regex = [
r'<link href="%s" type="text/css" media="all" '
'rel="stylesheet"( /)?>' % path
r'<link href="{}"( type="text/css")? media="all" '
'rel="stylesheet"( /)?>'.format(path)
for path in (
'/static/extra1.css',
'/static/extra2.css',
'/static/styles1.min.css',
'/static/styles2.min.css',
"/static/extra1.css",
"/static/extra2.css",
"/static/styles1.min.css",
"/static/styles2.min.css",
)
] + [
r'<link href="/static/print.min.css" type="text/css" '
r'<link href="/static/print.min.css" (type="text/css" )?'
'media="print" rel="stylesheet"( /)?>'
]
for rendered_node, expected_node in zip(
media.render_css(), expected_regex
):
for rendered_node, expected_node in zip(media.render_css(), expected_regex):
self.assertRegex(rendered_node, expected_node)

@pipeline_settings(PIPELINE_ENABLED=False)
def test_css_packages_with_pipeline_disabled(self):
"""Testing PipelineFormMedia.css_packages with PIPELINE_ENABLED=False"""

class MyMedia(PipelineFormMedia):
css_packages = {
'all': ('styles1', 'styles2'),
'print': ('print',),
"all": ("styles1", "styles2"),
"print": ("print",),
}

css = {
'all': ('extra1.css', 'extra2.css')
}
css = {"all": ("extra1.css", "extra2.css")}

media = Media(MyMedia)

self.assertEqual(
MyMedia.css,
{
'all': [
'extra1.css',
'extra2.css',
'pipeline/css/first.css',
'pipeline/css/second.css',
'pipeline/css/unicode.css',
"all": [
"extra1.css",
"extra2.css",
"pipeline/css/first.css",
"pipeline/css/second.css",
"pipeline/css/unicode.css",
],
'print': ['pipeline/css/urls.css'],
})
"print": ["pipeline/css/urls.css"],
},
)
self.assertEqual(MyMedia.css, media._css)

expected_regex = [
'<link href="%s" type="text/css" media="all" '
'rel="stylesheet"( /)?>' % path
'<link href="{}"( type="text/css")? media="all" '
'rel="stylesheet"( /)?>'.format(path)
for path in (
'/static/extra1.css',
'/static/extra2.css',
'/static/pipeline/css/first.css',
'/static/pipeline/css/second.css',
'/static/pipeline/css/unicode.css',
"/static/extra1.css",
"/static/extra2.css",
"/static/pipeline/css/first.css",
"/static/pipeline/css/second.css",
"/static/pipeline/css/unicode.css",
)
] + [
'<link href="/static/pipeline/css/urls.css" type="text/css" '
'<link href="/static/pipeline/css/urls.css" (type="text/css" )?'
'media="print" rel="stylesheet"( /)?>'
]
for rendered_node, expected_node in zip(
media.render_css(), expected_regex
):
for rendered_node, expected_node in zip(media.render_css(), expected_regex):
self.assertRegex(rendered_node, expected_node)

@pipeline_settings(PIPELINE_ENABLED=True)
def test_js_packages_with_pipeline_enabled(self):
"""Testing PipelineFormMedia.js_packages with PIPELINE_ENABLED=True"""

class MyMedia(PipelineFormMedia):
js_packages = ('scripts1', 'scripts2')
js = ('extra1.js', 'extra2.js')
js_packages = ("scripts1", "scripts2")
js = ("extra1.js", "extra2.js")

media = Media(MyMedia)
script_tag = '<script type="text/javascript" src="%s"></script>' if django_version() < '3.1' else '<script src="%s"></script>'

if django_version() < "3.1":
script_tag = '<script type="text/javascript" src="%s"></script>'
else:
script_tag = '<script src="%s"></script>'

self.assertEqual(
MyMedia.js,
[
'extra1.js',
'extra2.js',
'/static/scripts1.min.js',
'/static/scripts2.min.js',
])
"extra1.js",
"extra2.js",
"/static/scripts1.min.js",
"/static/scripts2.min.js",
],
)
self.assertEqual(MyMedia.js, media._js)
self.assertEqual(
media.render_js(),
[
script_tag % path
for path in (
'/static/extra1.js',
'/static/extra2.js',
'/static/scripts1.min.js',
'/static/scripts2.min.js',
"/static/extra1.js",
"/static/extra2.js",
"/static/scripts1.min.js",
"/static/scripts2.min.js",
)
])
],
)

@pipeline_settings(PIPELINE_ENABLED=False)
def test_js_packages_with_pipeline_disabled(self):
"""Testing PipelineFormMedia.js_packages with PIPELINE_ENABLED=False"""

class MyMedia(PipelineFormMedia):
js_packages = ('scripts1', 'scripts2')
js = ('extra1.js', 'extra2.js')
js_packages = ("scripts1", "scripts2")
js = ("extra1.js", "extra2.js")

media = Media(MyMedia)
script_tag = '<script type="text/javascript" src="%s"></script>' if django_version() < '3.1' else '<script src="%s"></script>'

if django_version() < "3.1":
script_tag = '<script type="text/javascript" src="%s"></script>'
else:
script_tag = '<script src="%s"></script>'

self.assertEqual(
MyMedia.js,
[
'extra1.js',
'extra2.js',
'pipeline/js/first.js',
'pipeline/js/second.js',
'pipeline/js/application.js',
])
"extra1.js",
"extra2.js",
"pipeline/js/first.js",
"pipeline/js/second.js",
"pipeline/js/application.js",
],
)
self.assertEqual(MyMedia.js, media._js)
self.assertEqual(
media.render_js(),
[
script_tag % path
for path in (
'/static/extra1.js',
'/static/extra2.js',
'/static/pipeline/js/first.js',
'/static/pipeline/js/second.js',
'/static/pipeline/js/application.js',
"/static/extra1.js",
"/static/extra2.js",
"/static/pipeline/js/first.js",
"/static/pipeline/js/second.js",
"/static/pipeline/js/application.js",
)
])
],
)
67 changes: 34 additions & 33 deletions tests/tests/test_glob.py
Original file line number Diff line number Diff line change
@@ -28,15 +28,15 @@ def assertSequenceEqual(self, l1, l2):
self.assertEqual(set(l1), set(l2))

def setUp(self):
self.storage = FileSystemStorage(local_path('glob_dir'))
self.storage = FileSystemStorage(local_path("glob_dir"))
self.old_storage = glob.staticfiles_storage
glob.staticfiles_storage = self.storage
self.mktemp('a', 'D')
self.mktemp('aab', 'F')
self.mktemp('aaa', 'zzzF')
self.mktemp('ZZZ')
self.mktemp('a', 'bcd', 'EF')
self.mktemp('a', 'bcd', 'efg', 'ha')
self.mktemp("a", "D")
self.mktemp("aab", "F")
self.mktemp("aaa", "zzzF")
self.mktemp("ZZZ")
self.mktemp("a", "bcd", "EF")
self.mktemp("a", "bcd", "efg", "ha")

def glob(self, *parts):
if len(parts) == 1:
@@ -50,50 +50,51 @@ def tearDown(self):
glob.staticfiles_storage = self.old_storage

def test_glob_literal(self):
self.assertSequenceEqual(self.glob('a'), [self.normpath('a')])
self.assertSequenceEqual(self.glob('a', 'D'), [self.normpath('a', 'D')])
self.assertSequenceEqual(self.glob('aab'), [self.normpath('aab')])
self.assertSequenceEqual(self.glob("a"), [self.normpath("a")])
self.assertSequenceEqual(self.glob("a", "D"), [self.normpath("a", "D")])
self.assertSequenceEqual(self.glob("aab"), [self.normpath("aab")])

def test_glob_one_directory(self):
self.assertSequenceEqual(
self.glob('a*'), map(self.normpath, ['a', 'aab', 'aaa']))
self.glob("a*"), map(self.normpath, ["a", "aab", "aaa"])
)
self.assertSequenceEqual(self.glob("*a"), map(self.normpath, ["a", "aaa"]))
self.assertSequenceEqual(self.glob("aa?"), map(self.normpath, ["aaa", "aab"]))
self.assertSequenceEqual(
self.glob('*a'), map(self.normpath, ['a', 'aaa']))
self.assertSequenceEqual(
self.glob('aa?'), map(self.normpath, ['aaa', 'aab']))
self.assertSequenceEqual(
self.glob('aa[ab]'), map(self.normpath, ['aaa', 'aab']))
self.assertSequenceEqual(self.glob('*q'), [])
self.glob("aa[ab]"), map(self.normpath, ["aaa", "aab"])
)
self.assertSequenceEqual(self.glob("*q"), [])

def test_glob_nested_directory(self):
if os.path.normcase("abCD") == "abCD":
# case-sensitive filesystem
self.assertSequenceEqual(
self.glob('a', 'bcd', 'E*'), [self.normpath('a', 'bcd', 'EF')])
self.glob("a", "bcd", "E*"), [self.normpath("a", "bcd", "EF")]
)
else:
# case insensitive filesystem
self.assertSequenceEqual(self.glob('a', 'bcd', 'E*'), [
self.normpath('a', 'bcd', 'EF'),
self.normpath('a', 'bcd', 'efg')
])
self.assertSequenceEqual(
self.glob("a", "bcd", "E*"),
[self.normpath("a", "bcd", "EF"), self.normpath("a", "bcd", "efg")],
)
self.assertSequenceEqual(
self.glob('a', 'bcd', '*g'), [self.normpath('a', 'bcd', 'efg')])
self.glob("a", "bcd", "*g"), [self.normpath("a", "bcd", "efg")]
)

def test_glob_directory_names(self):
self.assertSequenceEqual(self.glob("*", "D"), [self.normpath("a", "D")])
self.assertSequenceEqual(self.glob("*", "*a"), [])
self.assertSequenceEqual(
self.glob('*', 'D'), [self.normpath('a', 'D')])
self.assertSequenceEqual(self.glob('*', '*a'), [])
self.assertSequenceEqual(
self.glob('a', '*', '*', '*a'),
[self.normpath('a', 'bcd', 'efg', 'ha')])
self.glob("a", "*", "*", "*a"), [self.normpath("a", "bcd", "efg", "ha")]
)
self.assertSequenceEqual(
self.glob('?a?', '*F'),
map(self.normpath, [os.path.join('aaa', 'zzzF'),
os.path.join('aab', 'F')]))
self.glob("?a?", "*F"),
map(self.normpath, [os.path.join("aaa", "zzzF"), os.path.join("aab", "F")]),
)

def test_glob_directory_with_trailing_slash(self):
# We are verifying that when there is wildcard pattern which
# ends with os.sep doesn't blow up.
paths = glob.glob('*' + os.sep)
paths = glob.glob("*" + os.sep)
self.assertEqual(len(paths), 4)
self.assertTrue(all([os.sep in path for path in paths]))
self.assertTrue(all(os.sep in path for path in paths))
35 changes: 20 additions & 15 deletions tests/tests/test_middleware.py
Original file line number Diff line number Diff line change
@@ -1,43 +1,48 @@

from unittest.mock import patch

from django.core.exceptions import MiddlewareNotUsed
from django.test import TestCase
from django.http import HttpRequest, HttpResponse
from django.test import TestCase

from pipeline.middleware import MinifyHTMLMiddleware


def dummy_get_response(request):
return None


class MiddlewareTest(TestCase):
whitespace = b' '
whitespace = b" "

def setUp(self):
self.req = HttpRequest()
self.req.META = {
'SERVER_NAME': 'testserver',
'SERVER_PORT': 80,
"SERVER_NAME": "testserver",
"SERVER_PORT": 80,
}
self.req.path = self.req.path_info = "/"
self.resp = HttpResponse()
self.resp.status_code = 200
self.resp.content = self.whitespace

def test_middleware_html(self):
self.resp['Content-Type'] = 'text/html; charset=UTF-8'
self.resp["Content-Type"] = "text/html; charset=UTF-8"

response = MinifyHTMLMiddleware().process_response(self.req, self.resp)
self.assertIn('text/html', response['Content-Type'])
response = MinifyHTMLMiddleware(dummy_get_response).process_response(
self.req, self.resp
)
self.assertIn("text/html", response["Content-Type"])
self.assertNotIn(self.whitespace, response.content)

def test_middleware_text(self):
self.resp['Content-Type'] = 'text/plain; charset=UTF-8'
self.resp["Content-Type"] = "text/plain; charset=UTF-8"

response = MinifyHTMLMiddleware().process_response(self.req, self.resp)
self.assertIn('text/plain', response['Content-Type'])
response = MinifyHTMLMiddleware(dummy_get_response).process_response(
self.req, self.resp
)
self.assertIn("text/plain", response["Content-Type"])
self.assertIn(self.whitespace, response.content)

@patch('pipeline.middleware.settings.PIPELINE_ENABLED', False)
@patch("pipeline.middleware.settings.PIPELINE_ENABLED", False)
def test_middleware_not_used(self):
self.resp['Content-Type'] = 'text/plain; charset=UTF-8'

self.assertRaises(MiddlewareNotUsed, MinifyHTMLMiddleware)
self.assertRaises(MiddlewareNotUsed, MinifyHTMLMiddleware, dummy_get_response)
40 changes: 21 additions & 19 deletions tests/tests/test_packager.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
from django.test import TestCase

from pipeline.collector import default_collector
from pipeline.packager import Packager, PackageNotFound

from pipeline.packager import PackageNotFound, Packager
from tests.utils import _


@@ -12,35 +11,38 @@ def setUp(self):

def test_package_for(self):
packager = Packager()
packager.packages['js'] = packager.create_packages({
'application': {
'source_filenames': (
_('pipeline/js/application.js'),
),
'output_filename': 'application.js'
packager.packages["js"] = packager.create_packages(
{
"application": {
"source_filenames": (_("pipeline/js/application.js"),),
"output_filename": "application.js",
}
}
})
)
try:
packager.package_for('js', 'application')
packager.package_for("js", "application")
except PackageNotFound:
self.fail()
try:
packager.package_for('js', 'broken')
packager.package_for("js", "broken")
self.fail()
except PackageNotFound:
pass

def test_templates(self):
packager = Packager()
packages = packager.create_packages({
'templates': {
'source_filenames': (
_('pipeline/templates/photo/list.jst'),
),
'output_filename': 'templates.js',
packages = packager.create_packages(
{
"templates": {
"source_filenames": (_("pipeline/templates/photo/list.jst"),),
"output_filename": "templates.js",
}
}
})
self.assertEqual(packages['templates'].templates, [_('pipeline/templates/photo/list.jst')])
)
self.assertEqual(
packages["templates"].templates,
[_("pipeline/templates/photo/list.jst")],
)

def tearDown(self):
default_collector.clear()
71 changes: 38 additions & 33 deletions tests/tests/test_storage.py
Original file line number Diff line number Diff line change
@@ -1,22 +1,20 @@
from io import StringIO

from django.contrib.staticfiles import finders
from django.contrib.staticfiles.storage import staticfiles_storage
from django.core.management import call_command
from django.test import TestCase
from django.test.utils import override_settings

from django.test.utils import modify_settings
from django.test.utils import modify_settings, override_settings

from pipeline.collector import default_collector
from pipeline.storage import PipelineStorage

from tests.tests.test_compiler import DummyCompiler
from tests.utils import pipeline_settings

from io import StringIO


class PipelineNoPathStorage(PipelineStorage):
"""Storage without an implemented path method"""

def path(self, *args):
raise NotImplementedError()

@@ -37,11 +35,12 @@ def listdir(self, *args):


class DummyCSSCompiler(DummyCompiler):
""" Handles css files """
output_extension = 'css'
"""Handles css files"""

output_extension = "css"

def match_file(self, path):
return path.endswith('.css')
return path.endswith(".css")


class StorageTest(TestCase):
@@ -54,50 +53,56 @@ def test_post_process_dry_run(self):
processed_files = PipelineStorage().post_process({}, True)
self.assertEqual(list(processed_files), [])

@pipeline_settings(JS_COMPRESSOR=None, CSS_COMPRESSOR=None, COMPILERS=['tests.tests.test_storage.DummyCSSCompiler'])
@pipeline_settings(
JS_COMPRESSOR=None,
CSS_COMPRESSOR=None,
COMPILERS=["tests.tests.test_storage.DummyCSSCompiler"],
)
def test_post_process(self):
default_collector.collect()
storage = PipelineStorage()
processed_files = storage.post_process({})
self.assertTrue(('screen.css', 'screen.css', True) in processed_files)
self.assertTrue(('scripts.js', 'scripts.js', True) in processed_files)

@override_settings(STATICFILES_STORAGE='tests.tests.test_storage.PipelineNoPathStorage')
@pipeline_settings(JS_COMPRESSOR=None, CSS_COMPRESSOR=None, COMPILERS=['tests.tests.test_storage.DummyCSSCompiler'])
self.assertTrue(("screen.css", "screen.css", True) in processed_files)
self.assertTrue(("scripts.js", "scripts.js", True) in processed_files)

@override_settings(
STATICFILES_STORAGE="tests.tests.test_storage.PipelineNoPathStorage",
)
@pipeline_settings(
JS_COMPRESSOR=None,
CSS_COMPRESSOR=None,
COMPILERS=["tests.tests.test_storage.DummyCSSCompiler"],
)
def test_post_process_no_path(self):
"""
Test post_process with a storage that doesn't implement the path method.
"""
staticfiles_storage._setup()
try:
call_command('collectstatic', verbosity=0, interactive=False)
call_command("collectstatic", verbosity=0, interactive=False)
except NotImplementedError:
self.fail('Received an error running collectstatic')
self.fail("Received an error running collectstatic")

@modify_settings(STATICFILES_FINDERS={
'append': 'pipeline.finders.PipelineFinder'
})
@modify_settings(STATICFILES_FINDERS={"append": "pipeline.finders.PipelineFinder"})
def test_nonexistent_file_pipeline_finder(self):
path = finders.find('nothing.css')
path = finders.find("nothing.css")
self.assertIsNone(path)

@modify_settings(STATICFILES_FINDERS={
'append': 'pipeline.finders.CachedFileFinder'
})
@modify_settings(
STATICFILES_FINDERS={"append": "pipeline.finders.CachedFileFinder"}
)
def test_nonexistent_file_cached_finder(self):
path = finders.find('nothing.css')
path = finders.find("nothing.css")
self.assertIsNone(path)

@modify_settings(STATICFILES_FINDERS={
'append': 'pipeline.finders.PipelineFinder'
})
@modify_settings(STATICFILES_FINDERS={"append": "pipeline.finders.PipelineFinder"})
def test_nonexistent_double_extension_file_pipeline_finder(self):
path = finders.find('app.css.map')
path = finders.find("app.css.map")
self.assertIsNone(path)

@modify_settings(STATICFILES_FINDERS={
'append': 'pipeline.finders.CachedFileFinder'
})
@modify_settings(
STATICFILES_FINDERS={"append": "pipeline.finders.CachedFileFinder"}
)
def test_nonexistent_double_extension_file_cached_finder(self):
path = finders.find('app.css.map')
path = finders.find("app.css.map")
self.assertIsNone(path)
133 changes: 94 additions & 39 deletions tests/tests/test_template.py
Original file line number Diff line number Diff line change
@@ -1,84 +1,139 @@
from jinja2 import Environment, PackageLoader

from django.template import Template, Context
from django.template import Context, Template
from django.test import TestCase
from jinja2 import Environment, PackageLoader

from pipeline.jinja2 import PipelineExtension

from tests.utils import pipeline_settings


class JinjaTest(TestCase):
def setUp(self):
self.env = Environment(extensions=[PipelineExtension],
loader=PackageLoader('pipeline', 'templates'))
self.env = Environment(
extensions=[PipelineExtension],
loader=PackageLoader("pipeline", "templates"),
)

def test_no_package(self):
template = self.env.from_string(u"""{% stylesheet "unknow" %}""")
self.assertEqual(u'', template.render())
template = self.env.from_string(u"""{% javascript "unknow" %}""")
self.assertEqual(u'', template.render())
template = self.env.from_string("""{% stylesheet "unknow" %}""")
self.assertEqual("", template.render())
template = self.env.from_string("""{% javascript "unknow" %}""")
self.assertEqual("", template.render())

def test_package_css(self):
template = self.env.from_string(u"""{% stylesheet "screen" %}""")
self.assertEqual(u'<link href="/static/screen.css" rel="stylesheet" type="text/css" />', template.render())
template = self.env.from_string("""{% stylesheet "screen" %}""")
self.assertEqual(
'<link href="/static/screen.css" rel="stylesheet" type="text/css" />',
template.render(),
)

@pipeline_settings(PIPELINE_ENABLED=False)
def test_package_css_disabled(self):
template = self.env.from_string(u"""{% stylesheet "screen" %}""")
self.assertEqual(u'''<link href="/static/pipeline/css/first.css" rel="stylesheet" type="text/css" />
template = self.env.from_string("""{% stylesheet "screen" %}""")
self.assertEqual(
"""<link href="/static/pipeline/css/first.css" rel="stylesheet" type="text/css" />
<link href="/static/pipeline/css/second.css" rel="stylesheet" type="text/css" />
<link href="/static/pipeline/css/urls.css" rel="stylesheet" type="text/css" />''', template.render())
<link href="/static/pipeline/css/urls.css" rel="stylesheet" type="text/css" />""", # noqa
template.render(),
)

def test_package_js(self):
template = self.env.from_string(u"""{% javascript "scripts" %}""")
self.assertEqual(u'<script type="text/javascript" src="/static/scripts.js" charset="utf-8"></script>', template.render())
template = self.env.from_string("""{% javascript "scripts" %}""")
self.assertEqual(
'<script type="text/javascript" src="/static/scripts.js" charset="utf-8"></script>', # noqa
template.render(),
)

def test_package_js_async(self):
template = self.env.from_string(u"""{% javascript "scripts_async" %}""")
self.assertEqual(u'<script async type="text/javascript" src="/static/scripts_async.js" charset="utf-8"></script>', template.render())
template = self.env.from_string("""{% javascript "scripts_async" %}""")
self.assertEqual(
'<script async type="text/javascript" src="/static/scripts_async.js" charset="utf-8"></script>', # noqa
template.render(),
)

def test_package_js_defer(self):
template = self.env.from_string(u"""{% javascript "scripts_defer" %}""")
self.assertEqual(u'<script defer type="text/javascript" src="/static/scripts_defer.js" charset="utf-8"></script>', template.render())
template = self.env.from_string("""{% javascript "scripts_defer" %}""")
self.assertEqual(
'<script defer type="text/javascript" src="/static/scripts_defer.js" charset="utf-8"></script>', # noqa
template.render(),
)

def test_package_js_async_defer(self):
template = self.env.from_string(u"""{% javascript "scripts_async_defer" %}""")
self.assertEqual(u'<script async defer type="text/javascript" src="/static/scripts_async_defer.js" charset="utf-8"></script>', template.render())
template = self.env.from_string("""{% javascript "scripts_async_defer" %}""")
self.assertEqual(
'<script async defer type="text/javascript" src="/static/scripts_async_defer.js" charset="utf-8"></script>', # noqa
template.render(),
)


class DjangoTest(TestCase):
def render_template(self, template):
return Template(template).render(Context())

def test_compressed_empty(self):
rendered = self.render_template(u"""{% load pipeline %}{% stylesheet "unknow" %}""")
self.assertEqual(u'', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% stylesheet "unknow" %}""",
)
self.assertEqual("", rendered)

def test_compressed_css(self):
rendered = self.render_template(u"""{% load pipeline %}{% stylesheet "screen" %}""")
self.assertEqual(u'<link href="/static/screen.css" rel="stylesheet" type="text/css" media="all" />', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% stylesheet "screen" %}""",
)
self.assertEqual(
'<link href="/static/screen.css" rel="stylesheet" type="text/css" media="all" />', # noqa
rendered,
)

def test_compressed_css_media(self):
rendered = self.render_template(u"""{% load pipeline %}{% stylesheet "screen_media" %}""")
self.assertEqual(u'<link href="/static/screen_media.css" rel="stylesheet" type="text/css" media="screen and (min-width:500px)" />', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% stylesheet "screen_media" %}""",
)
self.assertEqual(
'<link href="/static/screen_media.css" rel="stylesheet" type="text/css" media="screen and (min-width:500px)" />', # noqa
rendered,
)

def test_compressed_css_title(self):
rendered = self.render_template(u"""{% load pipeline %}{% stylesheet "screen_title" %}""")
self.assertEqual(u'<link href="/static/screen_title.css" rel="stylesheet" type="text/css" media="all" title="Default Style" />', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% stylesheet "screen_title" %}""",
)
self.assertEqual(
'<link href="/static/screen_title.css" rel="stylesheet" type="text/css" media="all" title="Default Style" />', # noqa
rendered,
)

def test_compressed_js(self):
rendered = self.render_template(u"""{% load pipeline %}{% javascript "scripts" %}""")
self.assertEqual(u'<script type="text/javascript" src="/static/scripts.js" charset="utf-8"></script>', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% javascript "scripts" %}""",
)
self.assertEqual(
'<script type="text/javascript" src="/static/scripts.js" charset="utf-8"></script>', # noqa
rendered,
)

def test_compressed_js_async(self):
rendered = self.render_template(u"""{% load pipeline %}{% javascript "scripts_async" %}""")
self.assertEqual(u'<script async type="text/javascript" src="/static/scripts_async.js" charset="utf-8"></script>', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% javascript "scripts_async" %}""",
)
self.assertEqual(
'<script async type="text/javascript" src="/static/scripts_async.js" charset="utf-8"></script>', # noqa
rendered,
)

def test_compressed_js_defer(self):
rendered = self.render_template(u"""{% load pipeline %}{% javascript "scripts_defer" %}""")
self.assertEqual(u'<script defer type="text/javascript" src="/static/scripts_defer.js" charset="utf-8"></script>', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% javascript "scripts_defer" %}""",
)
self.assertEqual(
'<script defer type="text/javascript" src="/static/scripts_defer.js" charset="utf-8"></script>', # noqa
rendered,
)

def test_compressed_js_async_defer(self):
rendered = self.render_template(u"""{% load pipeline %}{% javascript "scripts_async_defer" %}""")
self.assertEqual(u'<script async defer type="text/javascript" src="/static/scripts_async_defer.js" charset="utf-8"></script>', rendered)
rendered = self.render_template(
"""{% load pipeline %}{% javascript "scripts_async_defer" %}""",
)
self.assertEqual(
'<script async defer type="text/javascript" src="/static/scripts_async_defer.js" charset="utf-8"></script>', # noqa
rendered,
)
8 changes: 3 additions & 5 deletions tests/tests/test_utils.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# -*- coding: utf-8 -*-

import mimetypes

from django.test import TestCase
@@ -9,9 +7,9 @@

class UtilTest(TestCase):
def test_guess_type(self):
self.assertEqual('text/css', guess_type('stylesheet.css'))
self.assertEqual('text/coffeescript', guess_type('application.coffee'))
self.assertEqual('text/less', guess_type('stylesheet.less'))
self.assertEqual("text/css", guess_type("stylesheet.css"))
self.assertEqual("text/coffeescript", guess_type("application.coffee"))
self.assertEqual("text/less", guess_type("stylesheet.less"))

def test_mimetypes_are_str(self):
for ext, mtype in mimetypes.types_map.items():
18 changes: 9 additions & 9 deletions tests/tests/test_views.py
Original file line number Diff line number Diff line change
@@ -13,16 +13,16 @@
@pipeline_settings(PIPELINE_COLLECTOR_ENABLED=True, PIPELINE_ENABLED=False)
class ServeStaticViewsTest(TestCase):
def setUp(self):
super(ServeStaticViewsTest, self).setUp()
super().setUp()

self.filename = 'pipeline/js/first.js'
self.filename = "pipeline/js/first.js"
self.storage = staticfiles_storage
self.request = RequestFactory().get('/static/%s' % self.filename)
self.request = RequestFactory().get(f"/static/{self.filename}")

default_collector.clear()

def tearDown(self):
super(ServeStaticViewsTest, self).tearDown()
super().tearDown()

default_collector.clear()
staticfiles_storage._setup()
@@ -31,7 +31,7 @@ def test_found(self):
self._test_found()

def test_not_found(self):
self._test_not_found('missing-file')
self._test_not_found("missing-file")

@override_settings(DEBUG=False)
def test_debug_false(self):
@@ -62,9 +62,9 @@ def test_collector_disabled_and_found(self):
def test_collector_disabled_and_not_found(self):
self._test_not_found(self.filename)

def _write_content(self, content='abc123'):
def _write_content(self, content="abc123"):
"""Write sample content to the test static file."""
with self.storage.open(self.filename, 'w') as f:
with self.storage.open(self.filename, "w") as f:
f.write(content)

def _test_found(self, **kwargs):
@@ -73,8 +73,8 @@ def _test_found(self, **kwargs):
self.assertEqual(response.status_code, 200)
self.assertTrue(self.storage.exists(self.filename))

if hasattr(response, 'streaming_content'):
content = b''.join(response.streaming_content)
if hasattr(response, "streaming_content"):
content = b"".join(response.streaming_content)
else:
content = response.content

9 changes: 4 additions & 5 deletions tests/urls.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
from django.conf.urls import url
from django.contrib import admin
from django.urls import path
from django.views.generic import TemplateView


urlpatterns = [
url(r'^$', TemplateView.as_view(template_name="index.html"), name="index"),
url(r'^empty/$', TemplateView.as_view(template_name="empty.html"), name="empty"),
url(r'^admin/', admin.site.urls),
path("", TemplateView.as_view(template_name="index.html"), name="index"),
path("empty/", TemplateView.as_view(template_name="empty.html"), name="empty"),
path("admin/", admin.site.urls),
]
7 changes: 3 additions & 4 deletions tests/utils.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
import os

import django

from django.test import override_settings


def _(path):
# Make sure the path contains only the correct separator
return path.replace('/', os.sep).replace('\\', os.sep)
return path.replace("/", os.sep).replace("\\", os.sep)


class pipeline_settings(override_settings):
@@ -16,5 +15,5 @@ def __init__(self, **kwargs):
# Django 1.10's override_settings inherits from TestContextDecorator
# and its __init__ method calls its superclass' __init__ method too,
# so we must do the same.
super(pipeline_settings, self).__init__()
self.options = {'PIPELINE': kwargs}
super().__init__()
self.options = {"PIPELINE": kwargs}
42 changes: 23 additions & 19 deletions tox.ini
Original file line number Diff line number Diff line change
@@ -1,45 +1,48 @@
[tox]
envlist =
pypy3-dj{22,30,31}
py{36,37,38,39}-dj{22,30,31,32}
py{38,39}-djmain
pypy3-dj{41,42}
py{39,310,311}-dj41
py{39,310,311,312}-dj42
py{310,311,312}-dj50
py{310,311,312}-dj51
py{310,311,312}-djmain
docs

[gh-actions]
python =
3.6: py36
3.7: py37
3.8: py38, docs
3.9: py39
3.9: py39, docs
3.10: py310
3.11: py311
3.12: py312
pypy3: pypy3

[gh-actions:env]
DJANGO =
2.2: dj22
3.0: dj30
3.1: dj31
3.2: dj32
4.1: dj41
4.2: dj42
5.0: dj50
5.1: dj51
main: djmain

[testenv]
basepython =
pypy3: pypy3
py36: python3.6
py37: python3.7
py38: python3.8
py39: python3.9
py310: python3.10
py311: python3.11
py312: python3.12
deps =
pypy3: mock
dj22: Django>=2.2.1,<2.3
dj30: Django>=3.0,<3.1
dj31: Django>=3.1,<3.2
dj32: Django>=3.2,<3.3
dj41: Django>=4.1,<4.2
dj42: Django>=4.2,<4.3
dj50: Django>=5.0,<5.1
dj51: Django>=5.1,<5.2
djmain: https://github.com/django/django/archive/main.tar.gz
jinja2
coverage
jsmin==3.0.0
ply==3.4
slimit==0.8.1
css-html-js-minify==2.5.5
setenv =
DJANGO_SETTINGS_MODULE = tests.settings
@@ -54,9 +57,10 @@ ignore_outcome =
djmain: True
ignore_errors =
djmain: True
allowlist_externals=npm

[testenv:docs]
basepython = python3.8
basepython = python3.9
changedir = docs
deps = sphinx
commands =