Skip to content

A serverless webscrapper to extract data from Google maps reviews using async architeture (with queues between lambdas) and persist the data on a relational Database.

Notifications You must be signed in to change notification settings

Mesheo/Google-Reviews-Analyzer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

45 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Useful Commands

if you clone this repository by personal interest the following commands will for sure make your life easier throughout the usage and adaptation of the code

Serverless related

  • Deploy a new version of the code to AWS: sls deploy
  • Watch the logs: sls logs -f function_name -s stage
  • Run the code locally: serverless invoke local -function function_name

Database related

  • Create a migration: npx sequelize-cli migration:generate --name migration_name
  • Run migration: npx sequelize-cli db:migrate
  • Undo the most recent migration: npx sequelize-cli db:migrate:undo

Serverless Framework Node HTTP API on AWS

This template demonstrates how to make a simple HTTP API with Node.js running on AWS Lambda and API Gateway using the Serverless Framework.

This template does not include any kind of persistence (database). For more advanced examples, check out the serverless/examples repository which includes Typescript, Mongo, DynamoDB and other examples.

Usage

Deployment

$ serverless deploy

After deploying, you should see output similar to:

Deploying aws-node-http-api-project to stage dev (us-east-1)

✔ Service deployed to stack aws-node-http-api-project-dev (152s)

endpoint: GET - https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/
functions:
  hello: aws-node-http-api-project-dev-hello (1.9 kB)

Note: In current form, after deployment, your API is public and can be invoked by anyone. For production deployments, you might want to configure an authorizer. For details on how to do that, refer to http event docs.

Invocation

After successful deployment, you can call the created application via HTTP:

curl https://xxxxxxx.execute-api.us-east-1.amazonaws.com/

Which should result in response similar to the following (removed input content for brevity):

{
  "message": "Go Serverless v2.0! Your function executed successfully!",
  "input": {
    ...
  }
}

Local development

You can invoke your function locally by using the following command:

serverless invoke local --function hello

Which should result in response similar to the following:

{
  "statusCode": 200,
  "body": "{\n  \"message\": \"Go Serverless v3.0! Your function executed successfully!\",\n  \"input\": \"\"\n}"
}

Alternatively, it is also possible to emulate API Gateway and Lambda locally by using serverless-offline plugin. In order to do that, execute the following command:

serverless plugin install -n serverless-offline

It will add the serverless-offline plugin to devDependencies in package.json file as well as will add it to plugins in serverless.yml.

After installation, you can start local emulation with:

serverless offline

To learn more about the capabilities of serverless-offline, please refer to its GitHub repository.

delete this latter

About

A serverless webscrapper to extract data from Google maps reviews using async architeture (with queues between lambdas) and persist the data on a relational Database.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published