pycausalmatch is a Python library for causal inference integrated with the process of selecting suitable control groups.
(I hope to develop this as a causal inference project for Big Data with the intent of eventually deploying pipelines)
The functionality that has been implemented so far is essentially a Python translation of the some of the features of the R library: MarketMatching (v.1.1.7 - as of Dec 2020), and combines 2 packages: tfcausalimpact and dtw-python
(I had originally used the causal inference library - dafiti-causalimpact, which has since been removed)
The DTW package is used for selection of most suitable control groups.
For an overview of the use of this library, please refer to the README of the R library.
This is still an alpha release - This package has only been tested for a single test market - I'm in the process of adding more features, incl using it with multiple test markets and fixing all the bugs soon!
Use the package manager pip to install pycausalmatch.
pip install pycausalmatchNotebooks (available in the notebooks/examples folder):
-
starter_example: The causal impact from this Python library matches the impact for the test market ('CPH') in the example in the R library, as shown in the plots in this notebook. -
prop_99_example: I've added an example on the causal impact of Prop 99 in California in the notebook under the notebooks/examples folder. I will keep updating this example as I develop the library further. -
r_lib_examples: The rest of this section mirrors the Examples section of the README of the R library, all of which are available in this notebook
##-----------------------------------------------------------------------
## Find the best matches (default is 5) for each airport time series
##-----------------------------------------------------------------------
TODO
ip_weather_data_path = '../notebooks/examples/datasets/weather.csv'
weather_df = rmm.read_data(ip_weather_data_path)
##-----------------------------------------------------------------------
## Or just search for 5 control markets for CPH and SFO
##-----------------------------------------------------------------------
TODO:: Add SFO
mm_only_cph = rmm.best_matches(
data=weather_df,
id_variable="Area",
date_variable="Date",
markets_to_be_matched=["CPH"],
matching_variable="Mean_TemperatureF",
parallel=False,
warping_limit=1,
dtw_emphasis=1,
matches=5,
start_match_period="2014-01-01",
end_match_period="2014-10-01",
)
##-----------------------------------------------------------------------
## Analyze causal impact of a made-up weather intervention in Copenhagen
## Since this is weather data it is a not a very meaningful example.
## This is merely to demonstrate the functionality.
##-----------------------------------------------------------------------
results_only_cph = rmm.inference(
matched_markets=mm_only_cph, test_market="CPH", end_post_period="2015-10-01"
)
##-------------------------------------------------------------------------
## You can also pass specific bsts model arguments (see bsts documentation)
##-------------------------------------------------------------------------
TODOA view of the best matches dataframe generated by the best_matches() function
mm_only_cph["best_matches"]
Plot actual observations for test market (CPH) versus the expectation,
Plot the cumulative impact, and
Plot point wise effects:
results_only_cph['all_output_plots'](figsize=(15, 14))
-
add remaining functionality of the R package
-
Add more examples
-
add tests, logging ...
-
add statistical inference
-
Integrate into an MLOps workflow
-
Add parallel execution
-
Add Streamlit and Dash app
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.
├── LICENSE
├── Makefile <- Makefile with commands like `make data` or `make train`
├── README.md <- The top-level README for developers using this project.
├── data
│ ├── external <- Data from third party sources.
│ ├── interim <- Intermediate data that has been transformed.
│ ├── processed <- The final, canonical data sets for modeling.
│ └── raw <- The original, immutable data dump.
│
├── docs <- A default Sphinx project; see sphinx-doc.org for details
│
├── models <- Trained and serialized models, model predictions, or model summaries
│
├── notebooks <- Jupyter notebooks. Naming convention is a number (for ordering),
│ the creator's initials, and a short `-` delimited description, e.g.
│ `1.0-jqp-initial-data-exploration`.
│
├── references <- Data dictionaries, manuals, and all other explanatory materials.
│
├── reports <- Generated analysis as HTML, PDF, LaTeX, etc.
│ └── figures <- Generated graphics and figures to be used in reporting
│
├── requirements.txt <- The requirements file for reproducing the analysis environment, e.g.
│ generated with `pip freeze > requirements.txt`
│
├── setup.py <- makes project pip installable (pip install -e .) so src can be imported
├── src <- Source code for use in this project.
│ ├── __init__.py <- Makes src a Python module
│ │
│ ├── data <- Scripts to download or generate data
│ │ └── make_dataset.py
│ │
│ ├── features <- Scripts to turn raw data into features for modeling
│ │ └── build_features.py
│ │
│ ├── models <- Scripts to train models and then use trained models to make
│ │ │ predictions
│ │ ├── predict_model.py
│ │ └── train_model.py
│ │
│ └── visualization <- Scripts to create exploratory and results oriented visualizations
│ └── visualize.py
│
└── tox.ini <- tox file with settings for running tox; see tox.readthedocs.io
Project based on the cookiecutter data science project template. #cookiecutterdatascience