💹 justETF Scraping

Scrape the justETF.

🛠️ Installation

To use justETF scraping package in your project, install the actual version from GitHub:

pip install git+https://github.com/druzsan/justetf-scraping.git

🚀 Usage

📋 Scrape the justETF Screener

Load overviews for all available (over 3200 at the moment) ETFs (requires a request for all ETF type: long-only, active, short & leveraged):

import justetf_scraping

df = justetf_scraping.load_overview()
df

	wkn	ticker	valor	name	index	inception_date	age_in_days	age_in_years	strategy	domicile_country	currency	hedged	securities_lending	dividends	ter	replication	size	is_sustainable	number_of_holdings	yesterday	last_week	last_month	last_three_months	last_six_months	last_year	last_three_years	last_five_years	2023	2022	2021	2020	last_dividends	last_year_dividends	last_year_volatility	last_three_years_volatility	last_five_years_volatility	last_year_return_per_risk	last_three_years_return_per_risk	last_five_years_return_per_risk	max_drawdown	last_year_max_drawdown	last_three_years_max_drawdown	last_five_years_max_drawdown	asset	instrument	region	exchange
isin
GB00BM9JYH62	A3GZKD	AVMX	<NA>	Global X Aave ETP	Aave	2023-03-13	311	0.852055	Long-only	Jersey	USD	False	False	Accumulating	0.99	Physically backed	0	False	<NA>	-9.48	1.09	-6.73	56.94	38.15	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	-40.18	NaN	NaN	NaN	Cryptocurrencies	ETN		gettex, XETRA
IE000GGQK173	A3D4VW	R8T	125589092	abrdn Global Real Estate Active Thematics UCITS ETF USD Accumulating ETF	abrdn Global Real Estate Active Thematics	2023-02-22	330	0.904110	Long-only, Active	Ireland	USD	False	False	Accumulating	0.40	Full replication	10	True	<NA>	-2.14	-1.93	-1.82	8.67	4.21	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	-15.23	NaN	NaN	NaN	Real Estate	ETF	World	gettex, XETRA
IE00B0M62Y33	A0HGWF	IUSJ	2308837	iShares AEX UCITS ETF	AEX®	2005-11-18	6635	18.178082	Long-only	Ireland	EUR	False	True	Distributing	0.30	Full replication	573	False	26	-2.49	-0.98	-3.28	4.46	0.15	4.79	25.08	70.81	16.77	-11.78	29.89	5.14	2.18	2.23	12.06	16.27	18.5	0.4	0.48	0.61	-62.93	-9.52	-22.39	-35.73	Equity	ETF		gettex, London, SIX Swiss Exchange, Euronext Amsterdam

Further enrich the data with additional information (asset class, region, exchanges and instrument, it requires further requests):

df = justetf_scraping.load_overview(enrich=True)

Load long-only ETFs (only requires one single request):

df = justetf_scraping.load_overview(strategy="epg-longOnly")

Load MSCI World ETFs:

df = justetf_scraping.load_overview(strategy="epg-longOnly", index="MSCI World")

📈 Scrape ETF Chart Data from justETF (e.g.)

Load the whole history of a chosen ETF by its ISIN:

df = justetf_scraping.load_chart("IE00B0M62Q58")
df

	quote	relative	dividends	cumulative_dividends	quote_with_dividends	relative_with_dividends	reinvested_dividends	quote_with_reinvested_dividends	relative_with_reinvested_dividends
date
2005-10-28	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-29	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-30	20.60	0.000000	0.0	0.00	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-31	20.99	1.893204	0.0	0.00	20.99	1.893204	0.000000	20.990000	1.893204
2005-11-01	21.03	2.087379	0.0	0.00	21.03	2.087379	0.000000	21.030000	2.087379
...	...	...	...	...	...	...	...	...	...
2024-01-13	61.60	199.029126	0.0	9.43	71.03	244.805825	19.169742	80.769742	292.086128
2024-01-14	61.60	199.029126	0.0	9.43	71.03	244.805825	19.169742	80.769742	292.086128
2024-01-15	61.48	198.446602	0.0	9.43	70.91	244.223301	19.132399	80.612399	291.322324
2024-01-16	61.51	198.592233	0.0	9.43	70.94	244.368932	19.141735	80.651735	291.513275
2024-01-17	61.19	197.038835	0.0	9.43	70.62	242.815534	19.042151	80.232151	289.476464

6656 rows × 9 columns

Compare

df = justetf_scraping.compare_charts(
    {
        "IE00B0M62Q58": justetf_scraping.load_chart("IE00B0M62Q58"),
        "IE00B0M63177": justetf_scraping.load_chart("IE00B0M63177"),
    },
    input_value="quote_with_dividends"
)
df

	IE00B0M62Q58	IE00B0M63177
date
2005-11-18	0.000000	0.000000
2005-11-19	0.000000	0.000000
2005-11-20	0.000000	0.000000
2005-11-21	-0.539811	-0.887436
2005-11-22	0.629780	-0.934143
...	...	...
2024-01-13	219.523167	114.806165
2024-01-14	219.523167	114.806165
2024-01-15	218.983356	114.572630
2024-01-16	219.118309	112.937879
2024-01-17	217.678812	108.967772

6635 rows × 2 columns

For further exploration examples, see Jupyter Notebooks

⚒️ Development Setup

To setup locally cloned project, first install Poetry:

pip install poetry

In the local project folder, install all dependencies and extras:

poetry install --all-extras

Activate local Poetry environment:

poetry shell

Setup pre-commit hooks:

pre-commit install

Similar Projects

• https://github.com/AshNL/justETF-overview-scraper
• https://github.com/SimonMandlik/etf_filter

Thanks

This project was inspired by this Stack Overflow question.