Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update redwood / Add Docker / Fix Scraper #13

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
.DS_Store
.netlify
dev.db*
dist
dist-babel
node_modules
yarn-error.log

# types files keep changing
.redwood

.vscode
docker-compose.yml
Dockerfile
api/prisma/dev.db*
.git*
2 changes: 2 additions & 0 deletions .env.defaults
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@
# into version control.

DATABASE_URL=file:./dev.db
REDWOOD_ENV_DEFAULT_COUNTRY=usa
REDWOOD_ENV_DEFAULT_COUNTRY_NAME=United States
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
.DS_Store
.env
.netlify
dev.db
dev.db*
dist
dist-babel
node_modules
yarn-error.log

# types files keep changing
.redwood
25 changes: 25 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
FROM node:14.16-alpine3.13

WORKDIR /app

# needed by api to scrape
RUN apk add --no-cache curl

# copy package files
COPY web/package.json ./web/
COPY api/package.json ./api/
COPY package.json \
yarn.lock \
./

# install all dependencies in all workspaces
RUN yarn --no-progress --non-interactive --frozen-lockfile

# copy everything
COPY . .

# overwrite redwood.toml
RUN cp docker-redwood.toml redwood.toml

# change db to sqlite
RUN sed -i -e 's/mysql/sqlite/g' /app/api/prisma/schema.prisma
46 changes: 28 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,40 +27,44 @@ We use Yarn as our package manager. To get the dependencies installed, just do t
yarn
```

### Fire it up
### Initialize Database

```terminal
yarn redwood dev
```
We’re using [Prisma2](https://github.com/prisma/prisma2), a modern DB toolkit to query, migrate and model your database.

Your browser should open automatically to `http://localhost:8910` to see the web app. Lambda functions run on
`http://localhost:8911` and are also proxied to `http://localhost:8910/api/functions/*`.
Prisma2 is [ready for production](https://isprisma2ready.com).

But we don’t have any data, or even a database!
To create a development database:

### Database
1. Change `./api/prisma/schema.prisma` to use `"sqlite"`:

We’re using [Prisma2](https://github.com/prisma/prisma2), a modern DB toolkit to query, migrate and model your database.
```prisma
datasource DS {
provider = "sqlite"
url = env("DATABASE_URL")
}
```

Prisma2 is [not ready for production](https://isprisma2ready.com) at the moment.
2. Then run `yarn redwood prisma migrate dev`

To create a development database:
This will read the schema definition in `api/prisma/schema.prisma` and generate a SQLite database in `api/prisma/dev.db`

### Fire it up

```terminal
yarn redwood db up
yarn redwood dev
```

This will read the schema definition in `api/prisma/schema.prisma` and generate a SQLite database in `api/prisma/dev.db`
Your browser should open automatically to `http://localhost:8910` to see the web app. Lambda functions run on
`http://localhost:8911` and are also proxied to `http://localhost:8910/api/functions/*`.

(If you’ve made changes to the schema run `yarn redwood db save` to generate a migration, and `yarn redwood db up`
to apply the migration/generate a new ORM client.)
But we don’t have any data!

### Downloading data

First, seed the database:

```terminal
yarn redwood db seed
yarn redwood prisma db seed
```

Now, run the scraper. In one terminal, start the server (`yarn rw dev`), and in another, make this request:
Expand All @@ -73,8 +77,14 @@ curl http://localhost:8911/scrape

You should be good to go now! Open [localhost:8910](http://localhost:8910) & enjoy development.

***
### Docker

You should be able to start the app in docker with `docker-compose up --build`. If you want to run it in the background: `docker-compose up -d --build`. If you want to run a shell in a container: `docker-compose run --rm web sh` or `docker-compose run --rm api sh`

Database should persist, so you can run `docker-compose down` and restart whenever you'd like. If you want to delete the database/volume, run `docker-compose down -v`.

---

*Thanks to [dDara](https://thenounproject.com/dDara/) for [the icon](https://thenounproject.com/dDara/collection/coronavirus/).*
_Thanks to [dDara](https://thenounproject.com/dDara/) for [the icon](https://thenounproject.com/dDara/collection/coronavirus/)._

MIT License
2 changes: 1 addition & 1 deletion api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"version": "0.0.0",
"private": true,
"dependencies": {
"@redwoodjs/api": "^0.2.4",
"@redwoodjs/api": "^0.31.2",
"apollo-server-plugin-response-cache": "^0.4.0",
"cheerio": "^1.0.0-rc.3",
"isomorphic-unfetch": "^3.0.0"
Expand Down
43 changes: 17 additions & 26 deletions api/prisma/seeds.js → api/prisma/countries.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,3 @@
/* eslint-disable no-console */
const { PrismaClient } = require('@prisma/client')
const dotenv = require('dotenv')

dotenv.config()
const db = new PrismaClient()

const countries = [
{
iso: 'itl',
Expand Down Expand Up @@ -50,24 +43,22 @@ const countries = [
iso: 'gbr',
worldometersSlug: 'uk',
name: 'United Kingdom',
}
},
{
iso: 'cad',
worldometersSlug: 'canada',
name: 'Canada',
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closes #6

},
{
iso: 'ind',
worldometersSlug: 'india',
name: 'India',
},
{
iso: 'bra',
worldometersSlug: 'brazil',
name: 'Brazil',
},
]

async function asyncForEach(array, callback) {
for (let index = 0; index < array.length; index++) {
await callback(array[index], index, array)
}
}

async function main() {
await asyncForEach(countries, async (country) => {
await db.country.create({ data: country })
})
}


main()
.catch((e) => console.error(e))
.finally(async () => {
await db.disconnect()
})
module.exports = countries

This file was deleted.

This file was deleted.

Loading