Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix compilation #28

Merged
merged 6 commits into from
Jan 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ tag = True
tag_name = v{new_version}
message = v{new_version}

[bumpversion:file:sparkmonitor/_version.py]

[bumpversion:file:package.json]
search = "version": "{current_version}"
replace = "version": "{new_version}"
31 changes: 0 additions & 31 deletions .eslintrc.js

This file was deleted.

40 changes: 40 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Tests the build of the extension

on:
# Triggers the workflow on push or pull request events but only for the master branch
pull_request:
branches: [ master ]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

jobs:
test-build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Set environnment package version from tag
run: echo "PACKAGE_VERSION=${GITHUB_REF#refs/*/v}" >> $GITHUB_ENV # extract "1.2.3" from refs/tags/v1.2.3

- name: Install Python
uses: actions/setup-python@v1
with:
python-version: '3.x'

- name: Install node
uses: actions/setup-node@v4
with:
node-version: '20.x'

- name: Set up JDK 1.8
uses: actions/setup-java@v1
with:
java-version: 1.8

- name: Install dependencies
run: pip install build twine

- name: Build wheel
run: python -m build
2 changes: 1 addition & 1 deletion .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ on:
workflow_dispatch:

jobs:
build:
check-code:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
Expand Down
131 changes: 118 additions & 13 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,21 +1,126 @@
.vscode/settings.json
*.pyc
*.bundle.*
lib/
node_modules/
*.log
**/.ipynb_checkpoints/
.eslintcache
.stylelintcache
*.egg-info/
.ipynb_checkpoints
*.tsbuildinfo
sparkmonitor/labextension
sparkmonitor/nbextension
scalalistener/project/
scalalistener/target/
sparkmonitor/*.jar
*.egg-info/
dist/
# Version file is handled by hatchling
sparkmonitor/_version.py

# Created by https://www.gitignore.io/api/python
# Edit at https://www.gitignore.io/?templates=python

### Python ###
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
node_modules/
sparkmonitor/static/
docs/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
venv
.tox
.metals
*.tsbuildinfo
scalalistener/.bsp
.yarn
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage/
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# Mr Developer
.mr.developer.cfg
.project
.pydevproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# End of https://www.gitignore.io/api/python

# OSX files
.DS_Store

# Yarn cache
.yarn/
3 changes: 1 addition & 2 deletions .yarnrc.yml
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
enableImmutableInstalls: false
nodeLinker: node-modules
nodeLinker: node-modules
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,15 @@ SparkMonitor is an extension for Jupyter Notebook & Lab that enables the live mo

## Requirements

- Jupyter Lab 4 OR Jupyter Notebook 4.4.0 or higher
- pyspark 2 or 3
- Jupyter Lab 4 OR Jupyter Notebook 4.4.0 or higher
- pyspark 2 or 3

## Features

- Automatically displays a live monitoring tool below cells that run Spark jobs in a Jupyter notebook
- A table of jobs and stages with progressbars
- A timeline which shows jobs, stages, and tasks
- A graph showing number of active tasks & executor cores vs time
- Automatically displays a live monitoring tool below cells that run Spark jobs in a Jupyter notebook
- A table of jobs and stages with progressbars
- A timeline which shows jobs, stages, and tasks
- A graph showing number of active tasks & executor cores vs time

<table>
<tr>
Expand Down Expand Up @@ -100,18 +100,18 @@ sbt +package

## History

- This project was originally written by krishnan-r as a [Google Summer of Code project](https://github.com/krishnan-r/sparkmonitor) for Jupyter Notebook with the [SWAN](https://swan.web.cern.ch/swan/) Notebook Service team at [CERN](http://home.cern/).
- This project was originally written by krishnan-r as a [Google Summer of Code project](https://github.com/krishnan-r/sparkmonitor) for Jupyter Notebook with the [SWAN](https://swan.web.cern.ch/swan/) Notebook Service team at [CERN](http://home.cern/).

- Further fixes and improvements were made by the team at CERN and members of the community maintained at [swan-cern/jupyter-extensions/tree/master/SparkMonitor](https://github.com/swan-cern/jupyter-extensions/tree/master/SparkMonitor)
- Further fixes and improvements were made by the team at CERN and members of the community maintained at [swan-cern/jupyter-extensions/tree/master/SparkMonitor](https://github.com/swan-cern/jupyter-extensions/tree/master/SparkMonitor)

- [Jafer Haider](https://github.com/itsjafer) created the fork [jupyterlab-sparkmonitor](https://github.com/itsjafer/jupyterlab-sparkmonitor) to update the extension to be compatible with JupyterLab as part of his internship at Yelp.

- This repository merges all the work done above and provides support for Lab & Notebook from a single package.
- [Jafer Haider](https://github.com/itsjafer) created the fork [jupyterlab-sparkmonitor](https://github.com/itsjafer/jupyterlab-sparkmonitor) to update the extension to be compatible with JupyterLab as part of his internship at Yelp.

- This repository merges all the work done above and provides support for Lab & Notebook from a single package.

## Changelog

This repository is published to pypi as [sparkmonitor](https://pypi.org/project/sparkmonitor/)

- 2.x see the [github releases page](https://github.com/swan-cern/sparkmonitor/releases) of this repository

- 1.x and below were published from [swan-cern/jupyter-extensions](https://github.com/swan-cern/jupyter-extensions) and some initial versions from [krishnan-r/sparkmonitor](https://github.com/krishnan-r/sparkmonitor)
- 1.x and below were published from [swan-cern/jupyter-extensions](https://github.com/swan-cern/jupyter-extensions) and some initial versions from [krishnan-r/sparkmonitor](https://github.com/krishnan-r/sparkmonitor)
6 changes: 3 additions & 3 deletions install.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"packageManager": "python",
"packageName": "sparkmonitor",
"uninstallInstructions": "Use your Python package manager (pip, conda, etc.) to uninstall the package sparkmonitor"
"packageManager": "python",
"packageName": "sparkmonitor",
"uninstallInstructions": "Use your Python package manager (pip, conda, etc.) to uninstall the package sparkmonitor"
}
Loading
Loading