get_people

web scrapping of LinkedIn

Table of contents

• General info
• Technologies
• Workflow
• Setup
• Status
• Contact
• Sources

General info

WIP

Technologies

• Puppeteer
• Google Cloud function
• Google Cloud functions-framework to run locally
• Some nice Google Sheet API wrapper (npm | doc)
• also used Google Cloud Storage to debug

Workflow

just puppeteer

Setup

WIP

Status

WIP

Contact

Created by @Fraberg - feel free to contact me

Sources

• https://medium.com/benextcompany/how-to-web-scrape-with-puppeteer-in-google-cloud-functions-59ad86d3abdc
• https://rominirani.com/using-puppeteer-in-google-cloud-functions-809a14856e14
• https://cloud.google.com/solutions/serverless-web-performance-monitoring-using-cloud-functions
• https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#class-cdpsession
• https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#targetcreatecdpsession
• https://jsoverson.medium.com/using-chrome-devtools-protocol-with-puppeteer-737a1300bac0
• https://jsoverson.medium.com/how-to-deploy-node-js-functions-to-google-cloud-8bba05e9c10a
• https://stackoverflow.com/questions/58048447/keeping-same-puppeteer-page-open-nodejs
• https://www.youtube.com/watch?v=i8THvr03FaY
• https://cloud.google.com/functions/docs/functions-framework
• https://stackoverflow.com/questions/52497252/puppeteer-wait-until-page-is-completely-loaded
• https://stackoverflow.com/questions/6460604/how-to-describe-object-arguments-in-jsdoc
• https://cloud.google.com/functions/docs/concepts/exec#:~:text=a%20storage%20option.-,Auto%2Dscaling%20and%20Concurrency,to%20instances%20of%20your%20function.&text=This%20automatic%20scaling%20behavior%20allows,different%20instance%20of%20your%20function.