Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implemented user registration, sending email notifications, and api documentation #9

Merged
merged 15 commits into from
Jun 2, 2021
Merged

Conversation

vickywane
Copy link
Contributor

This resolves this issue

This PR contains a YAML exported API Definition for the Ambianic-functions-API using the created API collection on Postman.

@commit-lint
Copy link

commit-lint bot commented Mar 8, 2021

Features

Update

  • added code coverage report (f05572e)
  • added code coverage report (487258b)
  • updated function to use local netlify server (f31188b)
  • updated function to use local netlify server (6b43349)

Bug Fixes

  • moved stripe key to secrets && improved scripts (a82f2ba)
  • fixed broken github pages dir (50037de)

Contributors

vickywane

Commit-Lint commands

You can trigger Commit-Lint actions by commenting on this PR:

  • @Commit-Lint merge patch will merge dependabot PR on "patch" versions (X.X.Y - Y change)
  • @Commit-Lint merge minor will merge dependabot PR on "minor" versions (X.Y.Y - Y change)
  • @Commit-Lint merge major will merge dependabot PR on "major" versions (Y.Y.Y - Y change)
  • @Commit-Lint merge disable will desactivate merge dependabot PR
  • @Commit-Lint review will approve dependabot PR
  • @Commit-Lint stop review will stop approve dependabot PR

@vickywane vickywane requested a review from ivelin March 8, 2021 11:14
@vickywane vickywane self-assigned this Mar 8, 2021
@ivelin
Copy link
Contributor

ivelin commented Mar 8, 2021

@vickywane This is a static document. We need a CI workflow that automatically re-generates and publishes to github pages API docs based on any PR included code changes.

@ivelin
Copy link
Contributor

ivelin commented Mar 8, 2021

@vickywane upon further research, I noticed that the latest best practice is as follows:

  • We can use the static yaml doc you generated via Swagger Inspector as the initial Open API spec file.
  • Going forward we should use the Open API spec file as the single source of truth to define API contracts and then auto generate server stubs, client libraries, docs and testing mockups with OpenAPI Generator (cli & npm).
  • Use CI workflow to (example 1, example 2):
    • re-generate client SDKs and server stubs, and run tests when a new API spec of server implementation code changes are pushed via PR.
    • use semantic release (as we already do so for ambianic-ui and ambianic-edge) to determine and if necessary create amd push a new release of:
      • API docs (option 1, option 2, option 3 )
      • server stubs and implementation
      • JavaScript client SDK implementation and npm package (see how we do this for ambianic-ui)
    • deploy new release of netlify functions
    • trigger new ambianic-ui release if there is an API breaking change introduced with this functions API release

Notice that there are some details that need to be carefully worked out. Netlify deployments guarantee that all production deployments remain at their permanent versioned URLs. This is critically important to avoid maintenance windows with service downtime. If a serverless API has a breaking change, the currently deployed PWA will keep running against the previous API version URL. Deploying a new API version and consequently a new PWA version should lead to a smooth user transition to the new versions without any downtime. This cycle has to be carefully thought out and tested.

@vickywane
Copy link
Contributor Author

vickywane commented Mar 9, 2021

@ivelin i have the OpenAPI generation ( client, stubs & docs ) setup.

However, I need more clarification on how they would be integrated into this repo. After each generation, it creates an entirely new project with several unneeded files.

I feel we should clean up some of the generated files ( using a script) and the rest should be stored in docs directory.

@vickywane
Copy link
Contributor Author

We need a CI workflow that automatically re-generates and publishes to GitHub pages API docs based on any PR included code changes.

Do we still need to publish the API to Github pages?
It should be possible by generating a markdown file from the Swagger YAML and publish to Github pages.

@pull-request-size pull-request-size bot added size/XL and removed size/M labels Mar 9, 2021
@vickywane vickywane added the documentation Improvements or additions to documentation label Mar 9, 2021
@ivelin
Copy link
Contributor

ivelin commented Mar 9, 2021

We need a CI workflow that automatically re-generates and publishes to GitHub pages API docs based on any PR included code changes.

Do we still need to publish the API to Github pages?
It should be possible by generating a markdown file from the Swagger YAML and publish to Github pages.

The end results should be a freshly updated public API docs that is pushed in sync with code release.

@ivelin
Copy link
Contributor

ivelin commented Mar 9, 2021

@ivelin i have the OpenAPI generation ( client, stubs & docs ) setup.

However, I need more clarification on how they would be integrated into this repo. After each generation, it creates an entirely new project with several unneeded files.

I feel we should clean up some of the generated files ( using a script) and the rest should be stored in docs directory.

Makes sense. We only need to commit in github the source files required to re-produce a binary release of a client SDK, server implementation and API docs. Interim local build artifacts can be ignored as usually.

@ivelin ivelin marked this pull request as draft March 10, 2021 19:33
Copy link
Contributor

@ivelin ivelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice progress:

TODO:

  • run tests against a locally deployed version of the API in the current PR

  • run tests against a netlify preview deployment of the API

  • Add a code coverage report after the test run. See here.

  • semantic release

    • tag and label a new github release
    • publish client sdk to npm
  • follow up: PWA import of client sdk for user subscription management

docs/test/api/DefaultApi.spec.js Outdated Show resolved Hide resolved
docs/test/api/DefaultApi.spec.js Outdated Show resolved Hide resolved
docs/src/ApiClient.js Outdated Show resolved Hide resolved

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@codecov
Copy link

codecov bot commented Mar 17, 2021

Codecov Report

❗ No coverage uploaded for pull request base (main@6098713). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head fdf975a differs from pull request most recent head 50037de. Consider uploading reports for the commit 50037de to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##             main       #9   +/-   ##
=======================================
  Coverage        ?   22.01%           
=======================================
  Files           ?        8           
  Lines           ?      536           
  Branches        ?      157           
=======================================
  Hits            ?      118           
  Misses          ?      418           
  Partials        ?        0           

@ivelin
Copy link
Contributor

ivelin commented Mar 31, 2021

@ivelin i have the OpenAPI generation ( client, stubs & docs ) setup.

However, I need more clarification on how they would be integrated into this repo. After each generation, it creates an entirely new project with several unneeded files.

I feel we should clean up some of the generated files ( using a script) and the rest should be stored in docs directory.

Yes, agreed. Only keep in github files that are required for the build to succeed. Interim files can be cleaned up. Deployment artifacts should be automatically tested and published. Docs should be updated and published with each new release.

See how we do that for the Edge Python API.

Once we have an URL for the premium subscription API, we will included it in the official docs.

@ivelin
Copy link
Contributor

ivelin commented Mar 31, 2021

@ivelin for some strange reason changes within the docs/package.json file are not being applied in the CI. It has been changed to generate a code coverage report using instanbul which is further being used by CodeCov.

I am not sure what you are asking me here. Please clarify what the issue is and what you want me to help with.

@vickywane
Copy link
Contributor Author

@vickywane

Here is what the premium subscriber function could look like:

name: isAmbianicPremiumSubscriptionActive(user-id)

parameter: user-id - unique user identified for the Auth0 service.

This has been implemented partly in the subscription-data endpoint. It returns a user_metadata containing a duration field with a start_date and end_date for the subscription. From the PWA, a check is done to see if the day is within the subscription duration.

@ivelin
Copy link
Contributor

ivelin commented May 10, 2021

@vickywane
Please add code coverage report to the CI and test runs against a local install and a cloud deployment preview.

@vickywane vickywane changed the title feat: added generated swaggger docs feat: implemented user registration and sending notification using Mailgun May 14, 2021
@vickywane vickywane changed the title feat: implemented user registration and sending notification using Mailgun feat: implemented user registration, sending email notifications, and api documentation May 14, 2021
@vickywane
Copy link
Contributor Author

@ivelin I had to make adjustments here for code coverage.

Previously we decided to stick with using tests from Postman ( run with Newman), but this doesn't fulfill the code coverage requirement. ( see this discussion ).

So I had to bring back using Mocha tests. Using Istanbul, we can generate code coverage.

I think we later have to decide which to stick with or evaluate if we really need code coverage here.

Copy link
Contributor

@ivelin ivelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vickywane looks like the main Open API definition file disappeared. Only docs and js files are left in this PR. Other comments inline.

.github/workflows/ci.yml Show resolved Hide resolved
env:
CI: true
STRIPE_TEST_KEY: sk_test_7LEesZrYrgDtyxsc6Hy5i0lS00GVFEbmqb
ENDPOINT: https://deploy-preview-${{github.event.number}}--ambianic-serverless.netlify.app/.netlify/functions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vickywane comments on my question from April 6?

vickywane-ambianic-stripe-swagger-docs-0.1-resolved.yaml Outdated Show resolved Hide resolved
openapi-docs/docs/DefaultApi.md Outdated Show resolved Hide resolved
openapi-docs/docs/DefaultApi.md Outdated Show resolved Hide resolved
openapi-docs/docs/DefaultApi.md Outdated Show resolved Hide resolved
openapi-docs/docs/MiscApi.md Outdated Show resolved Hide resolved
@vickywane
Copy link
Contributor Author

vickywane commented May 18, 2021

@ivelin I have fixed the naming comments on this pull request.

I am however being held back by an issue with the wait-for-netlify-action being used in the CI pipeline. I have reported the issue in the wait-for-netlify-action repository here and also in the Netlify community forum here.

In the interim, can I substitute the wait-for-netlify-action with my custom script written in the scripts.sh file ( wait-for-url function). It takes the preview number and waits for the deployed URL to return a 200 status before proceeding.

This is the reason why the current CI check fails and also should address your two comments on the CI file above here.

Copy link
Contributor

@ivelin ivelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments. Please point me to your own fork's URL where several CI runs completed successfully with test runs, API docs published and semantic releases published.

.idea/modules.xml Outdated Show resolved Hide resolved
.github/workflows/ci.yml Show resolved Hide resolved
.github/workflows/ci.yml Outdated Show resolved Hide resolved

## Documentation for Authorization

All endpoints do not require authorization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vickywane How are we going to protect these public APIs from malicious attacks that result into random notifications sent to users?

openapi-docs/docs/MiscApi.md Outdated Show resolved Hide resolved
@vickywane
Copy link
Contributor Author

vickywane commented May 20, 2021

How are we going to protect these public APIs from malicious attacks that result into random notifications sent to users?

I have created this new issue within this repository to track work done to address and provide a solution for this potential problem.

It would involve the PWA and edge device authenticating with this Cloud API each time a request is made.

vickywane and others added 3 commits May 21, 2021 03:43
* feat: project CI

* feat: corrected docs and added architecture diagram to readme

* feat: project CI

* update: added code coverage report

* update: added code coverage report

* update: updated function to use local netlify server

* update: updated function to use local netlify server

* fix: moved stripe key to secrets && improved scripts

* fix: update

* fix: update

Co-authored-by: Ivelin Ivanov <[email protected]>
@vickywane
Copy link
Contributor Author

vickywane commented May 24, 2021

@ivelin As requested, below are the relevant links generated from my own fork of this pull request here;

  • API documentation published to GitHub Pages ( Here )
  • API packaged published to NPM ( Here )
  • Github Action run containing with a successful semantic release run ( Here )

Below is a list of follow-up issues generated from this feature to worked upon after this pull request is merged;

Copy link
Contributor

@ivelin ivelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple unresolved comments.

.github/workflows/ci.yml Show resolved Hide resolved

## Documentation for Authorization

All endpoints do not require authorization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vickywane this comment is still unresolved. What protection do we have against malicious use of these public APIs?

[**deleteSubscription**](DefaultApi.md#deleteSubscription) | **DELETE** /subscription | Delete an Ambianic&#39;s user subscription
[**getSubscriptionData**](DefaultApi.md#getSubscriptionData) | **GET** /subscription | Get a user&#39;s subscription data
[**sendNotification**](DefaultApi.md#sendNotification) | **POST** /notification | Send an event detection notification
[**subscribeUser**](DefaultApi.md#subscribeUser) | **POST** /subscribe | Subscribe Ambianic user to premium notification
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vickywane for naming and semantic consistency, we should consider using

POST /subscription
as an endpoint to add a new subscription for a user.

instead of a separate endpoint name

POST /subscribe

That way we end up with two clean end points

/subscription that handles all CRUD operations (created, read, update, delete) for user subscriptions.

and

/notification for triggering premium notifications.

Copy link
Contributor Author

@vickywane vickywane May 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vickywane this comment is still unresolved. What protection do we have against the malicious use of these public APIs?

By default, Netlify doesn't provide any means of protection for the deployed endpoints. However, I have created a follow-up issue ( here ) to implement protection of the endpoints by ensuring that each request received is passed through a middleware that checks the origin of the request using a similar approach from Middlyjs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay.

Something to note is that each time a change is made in the route, it also has to be updated in the Ambianic PWA.

@vickywane
Copy link
Contributor Author

@ivelin As requested, below is a list of objectives for this pull request.

Each respective item below contains a link to where they were implemented in the continuous integration workflow and executed on my fork;

  • Tests against Netlify function pre-deploy ( Implemented here )
  • Tests validating that the latest implementation of API functions matches the latest open API definitions. ( Implemented here )
  • Semantic versioning, release, and deployment of the Netlify functions implementing the latest open API spec. ( Implemented here )
  • semantic release that packages and published the client SDK to npm. ( Implemented here )
  • semantic release of versioned docs, generated and published on GitHub pages ( Implemented Here )

Sorry, something went wrong.

@ivelin
Copy link
Contributor

ivelin commented May 26, 2021 via email

@vickywane
Copy link
Contributor Author

@ivelin I just implemented a change to use a local mock server created using @stoplight/prism-cli for mocking the responses made in each of the API tests. Previously we used mock servers managed by postman.

With this new change, you would find some inlineResponse files autogenerated from the openapi-spec.yaml file. These files contain the data for the mock response served by Prism

@ivelin
Copy link
Contributor

ivelin commented May 27, 2021 via email

@vickywane vickywane requested a review from ivelin May 28, 2021 21:15
Copy link
Contributor

@ivelin ivelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See inline comments.

README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
docs/index.html Outdated Show resolved Hide resolved
docs/index.html Outdated Show resolved Hide resolved
openapi-docs/README.md Outdated Show resolved Hide resolved
@vickywane vickywane requested a review from ivelin May 31, 2021 14:18
@vickywane
Copy link
Contributor Author

@ivelin

If adding all generated docs for the Cloud API is what is left from this pull request, can we proceed with a merge so I can add the live links and push it to the docs pull request?

README.md Outdated Show resolved Hide resolved
Copy link
Contributor

@ivelin ivelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Time to merge.

@ivelin ivelin merged commit 7dcc527 into ambianic:main Jun 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation size/XXL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants