Skip to content

Commit

Permalink
No macos specific python dependencies (#241)
Browse files Browse the repository at this point in the history
TensorFlow itself is now available for Mac arm64 chips. No need to have separate dependencies and commands for Mac.
  • Loading branch information
Theodlz authored Sep 14, 2023
1 parent c5e7d05 commit 4c0043d
Show file tree
Hide file tree
Showing 4 changed files with 2 additions and 84 deletions.
16 changes: 0 additions & 16 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,6 @@ python_dependencies_api:
python_dependencies_ingester:
$(PYTHON) kowalski/tools/pip_install_requirements.py requirements/requirements_ingester.txt

python_dependencies_ingester_macos:
$(PYTHON) kowalski/tools/pip_install_requirements.py requirements/requirements_ingester_macos.txt

# SUPERVISORD
generate_supervisord_conf:
$(PYTHON) kowalski/tools/generate_supervisord_conf.py all
Expand All @@ -56,12 +53,8 @@ setup_api: system_dependencies python_dependencies_api generate_supervisord_api_

setup_ingester: system_dependencies python_dependencies_ingester generate_supervisord_ingester_conf

setup_ingester_macos: system_dependencies python_dependencies_ingester_macos generate_supervisord_ingester_conf

setup_all: setup python_dependencies_api python_dependencies_ingester generate_supervisord_conf

setup_all_macos: setup python_dependencies_api python_dependencies_ingester_macos generate_supervisord_conf

init_models:
$(PYTHON) kowalski/tools/init_models.py

Expand All @@ -71,18 +64,12 @@ init_kafka:
run: setup_all init_models init_kafka
$(SUPERVISORD)

run_macos : setup_all_macos init_models init_kafka
$(SUPERVISORD)

run_api: setup_api
$(SUPERVISORD_API)

run_ingester: setup_ingester init_models init_kafka
$(SUPERVISORD_INGESTER)

run_ingester_macos: setup_ingester_macos init_models init_kafka
$(SUPERVISORD_INGESTER)

monitor:
$(SUPERVISORCTL) -i

Expand All @@ -95,9 +82,6 @@ monitor_ingester:
test: setup_all init_models init_kafka
$(PYTHON) kowalski/tools/tests.py

test_macos: setup_all_macos init_models init_kafka
$(PYTHON) kowalski/tools/tests.py

docker_setup: setup
$(PYTHON) kowalski/tools/docker.py setup $(FLAGS)

Expand Down
31 changes: 1 addition & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,6 @@ source env/bin/activate

to create your virtual environment. If you are told that pip is not found, try using pip3 instead. For the following steps however (in the virtualenv), pip should work.

*We do not suggest using conda, as we experienced trouble installing arm64 specific packages like tensorflow-macos and tensor flow-metal.*

The python dependencies will be install automatically when you start the app. The same will happen for Kafka and the ML models.

Expand Down Expand Up @@ -127,8 +126,6 @@ source env/bin/activate

to create your virtual environment. If you are told that pip is not found, try using pip3 instead. For the following steps however (in the virtualenv), pip should work.

*We do not suggest using conda, as we experienced trouble installing arm64 specific packages like tensorflow-macos and tensor flow-metal.*

The python dependencies will be install automatically when you start the app. The same will happen for Kafka and the ML models.

## Spin up your own `kowalski` **without Docker**
Expand Down Expand Up @@ -169,7 +166,7 @@ exit
If you are using a mongodb atlas cluster, kowalski won't be able to create admin users, so you will need to do so manually on the cluster's web interface. You will need to create 2 users: admin user and user, based on what usernames and passwords you've set in the config file in the `database` section. *Don't forget to also allow access from your IP address, or simply allow access from anywhere.*
### Start and test the app **on Linux amd64**
### Start and test the app
#### Starting the app
Expand All @@ -195,32 +192,6 @@ If you want to run a specific test, you can do so by running:
PYTHONPATH=. pytest -s kowalski/tests/<test_file.py>
```
### Start and test the app **on MacOS arm64 (M1 or M2)**
#### Starting the app
To start the app, run:
```bash
make run_macos
```
This will start the API, the dask clusters and alert brokers.
#### Run the tests
To run the tests, run:
```bash
make test_macos
```
If you want to run a specific test, you can do so by running:
```bash
PYTHONPATH=. pytest -s kowalski/tests/<test_file.py>
```
### Ingester (Pushing alerts to a local kafka topic)
Once the broker is running, you might want to create a local kafka stream of alerts to test it. To do so, you can run the ingester with
Expand Down
2 changes: 1 addition & 1 deletion requirements/requirements_ingester.txt
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,6 @@ requests==2.31.0
slack_sdk==3.20.0
supervisor==4.2.5
tables==3.8.0
tensorflow==2.12.0
tensorflow==2.13.0
tqdm==4.64.1
uvloop==0.17.0
37 changes: 0 additions & 37 deletions requirements/requirements_ingester_macos.txt

This file was deleted.

0 comments on commit 4c0043d

Please sign in to comment.