Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ahoy containers #15

Merged
merged 15 commits into from
Mar 24, 2021
Merged

Ahoy containers #15

merged 15 commits into from
Mar 24, 2021

Conversation

ruiconti
Copy link
Contributor

@ruiconti ruiconti commented Mar 23, 2021

This PR (partly) resolves #8.

What this adds

Mainly, it implements a few things:

  1. Local-development service execution can now be fully abstracted w/ docker-compose.
  2. Local-testing can now be done without the need to deal with python environments
  3. Makefile has been updated to help tooling.

Memory footprint

Using debian's busters distro, a few checks that might be useful when we're benchmarking this app.

  1. Idle-mode, meaning handling 0 requests, 8 workers:

image

  1. Idle-mode, meaning handling 0 requests, single-worker:

image

Which is pretty straightforward, ~44MiB per worker of memory footprint. IMO it's definitely something work investigating i.e. what is taking so much space since this is supposed to be a lean framework.

What is currently missing

For this PR, it is missing a few things that are being added as work is progressed:

  • Proper volume mapping
  • Proper logging
  • Usage of poetry for dep mgmt

What needs to be done

This work enables the service to be fully usable in local and development environments. So what is "missing" from this work in order to be used in production environments:

  1. Security concerns (TLS everywhere)
  2. Real infrastructure concerns (reversed proxies, load balancers)
  3. CI/CD pipelines
  4. Integration to cloud providers & image repositories
  5. Proper & standardized logging

But those concerns would be addressed after we went public, IMO. But are kept in record so we have a clear vision of next steps.

@rafamelos @andreyrcdias care to review & complement?

@ruiconti ruiconti added the enhancement New feature or request label Mar 23, 2021
@ruiconti ruiconti added this to the Going Public milestone Mar 23, 2021
@ruiconti ruiconti self-assigned this Mar 23, 2021
Copy link
Contributor

@andreyrcdias andreyrcdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented on some suggestions about the names of the services.

About the volumes, I made some suggestions so that we can at that first moment, persist the data somewhere.

In the case of high memory usage, correct me if I'm wrong, is this related in any way to all the dev dependencies that are installed?

About the proper logging and poetry dependency management, I need to do a study and see the best practices

docker-compose.yml Outdated Show resolved Hide resolved
docker-compose.yml Show resolved Hide resolved
docker-compose.yml Show resolved Hide resolved
Copy link

@rafamelos rafamelos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have much knowledge about using Docker (I definitely need some remembering and new studies on it), with that said I think I can't really give a appropriated opinion, but this seems good to me.
About the memory footprint, do you have some refs our thoughts on what level of memory use we should have? I think a good approach to investigate this would be to strip the project removing everything and running only with starlette, and again only with starlette and ariadne, in a way that we can learn what is the minimum consumption we can achieve with our current choices.

@ruiconti ruiconti changed the title WIP: Ahoy containers Ahoy containers Mar 24, 2021
@ruiconti
Copy link
Contributor Author

ruiconti commented Mar 24, 2021

I commented on some suggestions about the names of the services.
About the volumes, I made some suggestions so that we can at that first moment, persist the data somewhere.

Thanks! Those were included.

In the case of high memory usage, correct me if I'm wrong, is this related in any way to all the dev dependencies that are installed?

No way to tell without investigating. Created #16 to track that.

About the proper logging and poetry dependency management, I need to do a study and see the best practices

Gonna leave logging for future improvements as they aren't critical.

As for poetry. This open issue made me re-think using it for installing. It'd be best to use it only to manage inclusions & removals of dependencies.

I don't have much knowledge about using Docker (I definitely need some remembering and new studies on it), with that said I think I can't really give a appropriated opinion, but this seems good to me.

Alright! I'm merging then.

About the memory footprint, do you have some refs our thoughts on what level of memory use we should have? I think a good approach to investigate this would be to strip the project removing everything and running only with starlette, and again only with starlette and ariadne, in a way that we can learn what is the minimum consumption we can achieve with our current choices.

It varies proportionately to how many dependencies there are. I remember reading that vanilla django footprint was ~20MB. However, I did a search and couldn't find proper references and benchmarks. It's something worth doing to in #16.

@ruiconti ruiconti merged commit 4230d93 into main Mar 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Setting-up containerization
3 participants