diff --git a/SUMMARY.md b/SUMMARY.md index e83ea34e..2fb2c1eb 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -115,6 +115,14 @@ - [Session Plan](courses/backend/databases/week2/session-plan.md) - [Assignment](courses/backend/databases/week2/assignment.md) - [Node](courses/backend/node/README.md) + - [Week 1](courses/backend/node/week1/README.md) + - [Preparation](courses/backend/node/week1/preparation.md) + - [Session Plan](courses/backend/node/week1/session-plan.md) + - [Assignment](courses/backend/node/week1/assignment.md) + - [Week 2](courses/backend/node/week2/README.md) + - [Preparation](courses/backend/node/week2/preparation.md) + - [Session Plan](courses/backend/node/week2/session-plan.md) + - [Assignment](courses/backend/node/week2/assignment.md) - [Final Backend Project](courses/backend/final-project/README.md) - [Common Modules](shared-modules/README.md) diff --git a/courses/backend/node/README.md b/courses/backend/node/README.md index ddd5ab78..3473db15 100644 --- a/courses/backend/node/README.md +++ b/courses/backend/node/README.md @@ -4,10 +4,10 @@ This module is part of the Backend specialism and focuses on using Node.js to bu ## Contents -| Week | Topic | Preparation | Lesson Plan | Assignment | +| Week | Topic | Preparation | Session Plan | Assignment | | ---- | ------------------------ | ----------------------------------- | ----------------------------------- | ------------------------------------- | -| 1. | Express | [Preparation](week1/preparation.md) | [Assignment](./week1/assignment.md) | [Session plan](week1/session-plan.md) | -| 2. | Database connection; API | [Preparation](week2/preparation.md) | [Assignment](./week1/assignment.md) | [Session plan](week2/session-plan.md) | +| 1. | Express | [Preparation](./week1/preparation.md) | [Assignment](./week1/assignment.md) | [Session plan](./week1/session-plan.md) | +| 2. | Database connection; API | [Preparation](./week2/preparation.md) | [Assignment](./week1/assignment.md) | [Session plan](./week2/session-plan.md) | ## Module Learning Goals @@ -16,6 +16,6 @@ By the end of this module, you will be able to: - [ ] Build web servers with Express.js - [ ] Design and implement APIs using HTTP methods following REST principles - [ ] Use middlewares for authentication, logging, and validation -- [ ] Test APIs using Postman - [ ] Use logging and debugging tools to monitor and troubleshoot applications - [ ] Connect to databases and implement CRUD operations +- [ ] Test APIs using Postman diff --git a/courses/backend/node/week1/README.md b/courses/backend/node/week1/README.md index 457c425d..b1966286 100644 --- a/courses/backend/node/week1/README.md +++ b/courses/backend/node/week1/README.md @@ -16,4 +16,3 @@ By the end of this session, you will be able to: - [ ] Implement routing in Express to handle different HTTP requests and endpoints. - [ ] Use logging and debugging tools to monitor and troubleshoot Node.js applications. - [ ] Apply middleware functions in Express to process requests and responses. -- [ ] Use Postman to test and debug APIs you have built. diff --git a/courses/backend/node/week1/session-materials/06-auth.md b/courses/backend/node/week1/session-materials/06-auth.md index 57ff25e8..cb0dc187 100644 --- a/courses/backend/node/week1/session-materials/06-auth.md +++ b/courses/backend/node/week1/session-materials/06-auth.md @@ -116,4 +116,4 @@ It is left as an optional exercise to add the following routes: - `PUT /api/snippets/:id` to update a snippet - `DELETE /api/snippets/:id` to delete a snippet -Also, it could be a good idea to deny the request if the user making the request is not confirmed +Also, it could be a good idea to deny the request if the user making the request is not confirmed. diff --git a/courses/backend/node/week1/session-plan.md b/courses/backend/node/week1/session-plan.md index 6b2f1946..756d0c28 100644 --- a/courses/backend/node/week1/session-plan.md +++ b/courses/backend/node/week1/session-plan.md @@ -1,6 +1,6 @@ # Session plan -## Session Outline +## Session outline - Express - What is Express (10 mins) diff --git a/courses/backend/node/week2/README.md b/courses/backend/node/week2/README.md index e69de29b..4a69996c 100644 --- a/courses/backend/node/week2/README.md +++ b/courses/backend/node/week2/README.md @@ -0,0 +1,50 @@ +# Node (Week 2) + +In this session we will focus on connecting to a database, building an API, and using Postman to test our API endpoints. We will also cover how to structure our code for better maintainability and scalability. + +## Contents + +- [Preparation](./preparation.md) +- [Session Plan](./session-plan.md) (for mentors) +- [Assignment](./assignment.md) + +## Session Learning goals + +By the end of this session, you will be able to: +TODO - Format as `verb` for API section + +- [ ] Learn how to manage advanced database interactions in your service + - [ ] Set up a connection to your mysql database using Knex + - [ ] Configure environment variables + - [ ] Execute `select`, `create`, `delete` and `update` queries using Knex +- [ ] API + - [ ] REST + - [ ] CRUD + - [ ] Router verb `GET`, `POST`, `DELETE`, `PUT` + - [ ] POST mention express.json middleware + - [ ] Configure Postman for advanced backend development + - [ ] Creating collections and saving requests + - [ ] Set up multiple environments + - [ ] Managing secrets + - [ ] Create basic test suites + +TODO - Move this content somewhere else + +### 1. What is Representational State Transfer (REST)? + +Building software is like building houses: architecture is everything. The design of each part is just as important as the utility of it. REST is a specific architectural style for web applications. It serves to organise code in **predictable** ways. + +The most important features of REST are: + +- An application has a `frontend` (client) and a `backend` (server). This is called [separation of concerns](https://medium.com/machine-words/separation-of-concerns-1d735b703a60): each section has its specific job to do. The frontend deals with presenting data in a user friendly way, the backend deals with all the logic and data manipulation +- The server is `stateless`, which means that it doesn't store any data about a client session. Whenever a client sends a request to the server, each request from the client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. This makes it possible to handle requests from millions of users. +- Server responses can be temporarily stored on the client (a browser) using a process called `caching`: storing files like images or webpages in the browser to load the next time you enter a website (instead of getting them from the server, which generally takes longer to do). +- Client-server communication is done through `Hypertext Transfer Protocol (HTTP)` (more on that later), which serves as the style (the how) of communication. + +It's important to know about REST because it teaches us how web applications are designed and holds us to a standard that makes development and usage predictable. However, don't worry if you don't know what any of this means just yet. It's good to be exposed to it, and understanding will come with experience. + +For more research, check the following resource: + +- [What is REST: a simple explanation for beginners](https://medium.com/extend/what-is-rest-a-simple-explanation-for-beginners-part-1-introduction-b4a072f8740f) + +- [@NoerGitKat (lots of web app clones/examples to learn from)](https://github.com/NoerGitKat) diff --git a/courses/backend/node/week2/assignment.md b/courses/backend/node/week2/assignment.md index e69de29b..d3188c42 100644 --- a/courses/backend/node/week2/assignment.md +++ b/courses/backend/node/week2/assignment.md @@ -0,0 +1,203 @@ +# Assignment + +Once again, you will deliver 2 pull requests: + +- A pull request for the **Warmup** - in your regular hyf-homework repository +- A pull request for the additional **meal sharing endpoints** - in the meal-sharing repository + +In both repositories, create a `nodejs-week2` branch from `main` to work on the homework (`git checkout -b nodejs-week2` ). + +TODO - We should add a task to practice postman too. Maybe add all their final endpoints to their collection. + +## Warmup + +For the warmup you will be handed a Contacts API with a single endpoint: + +- `GET /api/contacts` + +This endpoint accepts a query parameter `sort`. Here's how you can use it: + +- `GET /api/contacts?sort=first_name%20ASC` + - Sorts contacts by first name, ascending +- `GET /api/contacts?sort=last_name%20DESC` + - Sorts contacts by last name, descending + +But this `sort` query parameter has been introduced with a SQL injection vulnerability and the goal is to demonstrate the issue and then fix and remove the vulnerability. + +### Setup + +TODO - Review assignment to work with sqlite. + +Go to `nodejs/week2` in your `hyf-homework` repo: + +```shell +npm init -y +npm i express mysql2 knex +npm i --save-dev nodemon +npm set-script dev "nodemon app.js" +``` + +Make sure you have `"type": "module"` in your `package.json`. + +You should also ensure that the `node_modules/` folder is ignored by Git: + +```shell +echo node_modules/ >> .gitignore +``` + +Create a database/schema called `hyf_node_week2_warmup` with a `contacts` table: + +```sql +CREATE TABLE `contacts` ( + `id` int unsigned NOT NULL AUTO_INCREMENT, + `first_name` varchar(255) NOT NULL, + `last_name` varchar(255) NOT NULL, + `email` varchar(255) DEFAULT NULL, + `phone` varchar(255) DEFAULT NULL, + PRIMARY KEY (`id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; + +-- Sample data +insert into contacts (id, first_name, last_name, email, phone) values (1, 'Selig', 'Matussov', 'smatussov0@pinterest.com', '176-630-4577'); +insert into contacts (id, first_name, last_name, email, phone) values (2, 'Kenny', 'Yerrington', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (3, 'Emilie', 'Gaitskell', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (4, 'Jordon', 'Tokell', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (5, 'Sallyann', 'Persse', 'spersse4@webnode.com', '219-157-2368'); +insert into contacts (id, first_name, last_name, email, phone) values (6, 'Berri', 'Bulter', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (7, 'Lanni', 'Ivanilov', 'livanilov6@fda.gov', null); +insert into contacts (id, first_name, last_name, email, phone) values (8, 'Dagny', 'Milnthorpe', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (9, 'Annadiane', 'Bansal', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (10, 'Tawsha', 'Hackley', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (11, 'Rubetta', 'Ozelton', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (12, 'Charles', 'Boughey', 'cbougheyb@senate.gov', '605-358-5664'); +insert into contacts (id, first_name, last_name, email, phone) values (13, 'Shantee', 'Robbe', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (14, 'Gleda', 'Peat', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (15, 'Arlinda', 'Ethersey', 'aetherseye@biglobe.ne.jp', '916-139-1300'); +insert into contacts (id, first_name, last_name, email, phone) values (16, 'Armando', 'Meachem', 'ameachemf@oaic.gov.au', '631-442-5339'); +insert into contacts (id, first_name, last_name, email, phone) values (17, 'Codi', 'Redhouse', null, '401-953-6897'); +insert into contacts (id, first_name, last_name, email, phone) values (18, 'Ann', 'Buncombe', 'abuncombeh@ow.ly', '210-338-0748'); +insert into contacts (id, first_name, last_name, email, phone) values (19, 'Louis', 'Matzkaitis', 'lmatzkaitisi@ebay.com', '583-996-6979'); +insert into contacts (id, first_name, last_name, email, phone) values (20, 'Jessey', 'Pala', null, null); +insert into contacts (id, first_name, last_name, email, phone) values (21, 'Archy', 'Scipsey', 'ascipseyk@ask.com', '420-983-2426'); +insert into contacts (id, first_name, last_name, email, phone) values (22, 'Benoit', 'Mould', 'bmouldl@bing.com', '271-217-9218'); +insert into contacts (id, first_name, last_name, email, phone) values (23, 'Sherm', 'Girardey', 'sgirardeym@guardian.co.uk', '916-999-2957'); +insert into contacts (id, first_name, last_name, email, phone) values (24, 'Raquel', 'Mudge', 'rmudgen@slate.com', '789-830-7473'); +insert into contacts (id, first_name, last_name, email, phone) values (25, 'Tabor', 'Reavey', null, null); +``` + +Create `app.js`: + +```js +import knex from "knex"; +const knexInstance = knex({ + client: "mysql2", + connection: { + host: process.env.DB_HOST || "127.0.0.1", + port: process.env.DB_PORT || 3306, + user: process.env.DB_USER || "root", + password: process.env.DB_PASSWORD || "my-secret-pw", + database: process.env.DB_NAME || "hyf_node_week2_warmup", + multipleStatements: true, + }, +}); + +import express from "express"; +const app = express(); +const port = process.env.PORT || 3000; + +app.use(express.json()); + +const apiRouter = express.Router(); +app.use("/api", apiRouter); + +const contactsAPIRouter = express.Router(); +apiRouter.use("/contacts", contactsAPIRouter); + +contactsAPIRouter.get("/", async (req, res) => { + let query = knexInstance.select("*").from("contacts"); + + if ("sort" in req.query) { + const orderBy = req.query.sort.toString(); + if (orderBy.length > 0) { + query = query.orderByRaw(orderBy); + } + } + + console.log("SQL", query.toSQL().sql); + + try { + const data = await query; + res.json({ data }); + } catch (e) { + console.error(e); + res.status(500).json({ error: "Internal server error" }); + } +}); + +app.listen(port, () => { + console.log(`Listening on port ${port}`); +}); +``` + +As mentioned above, the `sort` query parameter has been introduced with a SQL injection vulnerability. + +First, you should demonstrate the SQL injection and that it for instance is possible to drop/delete the `contacts` table with the `sort` query parameter. +You can for instance demonstrate this with a screen recording and include it in the PR description. + +After having demonstrated the SQL injection vulnerability, the goal is then to fix the issue by updating `app.js`. + +**Hint:** the `multipleStatements: true` part in the configuration indicates how you can use the vulnerability. The configuration should not be changed though, the SQL injection should be fixed by making changes in the `/api/contacts` route. + +## Meal sharing endpoints + +You will continue working in the meal-sharing repository for this task. + +You should have the basic [CRUD](https://www.freecodecamp.org/news/crud-operations-explained/) endpoints for **meals** and **reservations** as the result of last week's homework. This week, you will add **query parameters**, that will allow you to **sort** and **filter** the information retrieved from the database. + +### Routes + +#### Meals + +Work with your `GET api/meals` route to add the query parameters. +Make sure that the query parameters can be combined, f.x. `?limit=4&maxPrice=90`. + +| Parameter | Data type | Description | Example | +| ----------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- | +| `maxPrice` | Number | Returns all meals that are cheaper than `maxPrice`. | `api/meals?maxPrice=90` | +| `availableReservations` | Boolean | Returns all meals that still have available spots left, if `true`. If `false`, return meals that have no available spots left.[^1] | `api/meals?availableReservations=true` | +| `title` | String | Returns all meals that partially match the given title. `Rød grød` will match the meal with the title `Rød grød med fløde`. | `api/meals?title=Indian%20platter` | +| `dateAfter` | Date | Returns all meals where the date for `when` is after the given date. | `api/meals?dateAfter=2022-10-01` | +| `dateBefore` | Date | Returns all meals where the date for `when` is before the given date. | `api/meals?dateBefore=2022-08-08` | +| `limit` | Number | Returns the given number of meals. | `api/meals?limit=7` | +| `sortKey`[^2] | String | Returns all meals sorted by the given key. Allows `when`, `max_reservations` and `price` as keys. Default sorting order is asc(ending). | `api/meals?sortKey=price` | +| `sortDir`[^3] | String | Returns all meals sorted in the given direction. Only works combined with the `sortKey` and allows `asc` or `desc`. | `api/meals?sortKey=price&sortDir=desc` | + +[^1]: `availableReservations` requires you to work with several database tables at once. Try practicing the right query in MySQL Workbench first (you might have it from Database week2 homework) and once you have it working, build it with `knex`. + +[^2]: This used to be `sort_key` in a previous version of the homework text. + +[^3]: This used to be `sort_dir` in a previous version of the homework text. + +#### Reviews + +By now, you have the basic set of endpoints for **meals** and **reservations** and even a collection of query parameters for **meals**. To practice a bit more and finalize the basic backend functionality, create the set of routes for **reviews**: + +| Route | HTTP method | Description | +| ----------------------------- | ----------- | ---------------------------------------- | +| `/api/reviews` | GET | Returns all reviews. | +| `/api/meals/:meal_id/reviews` | GET | Returns all reviews for a specific meal. | +| `/api/reviews` | POST | Adds a new review to the database. | +| `/api/reviews/:id` | GET | Returns a review by `id`. | +| `/api/reviews/:id` | PUT | Updates the review by `id`. | +| `/api/reviews/:id` | DELETE | Deletes the review by `id`. | + +#### Knex + +You should try to avoid using `knex.raw` and instead use the different `knex` functions, for example: + +- `.select`, `.from`, `.where`, `join`, `leftJoin` +- `.insert` +- `.update` +- `.del` (for deletion) + +Check out the [Knex cheatsheet](https://devhints.io/knex)! diff --git a/courses/backend/node/week2/preparation.md b/courses/backend/node/week2/preparation.md index e69de29b..0a2c6f55 100644 --- a/courses/backend/node/week2/preparation.md +++ b/courses/backend/node/week2/preparation.md @@ -0,0 +1,11 @@ +# Preparation + +- [NodeJS Web API with KNEX and Express](https://www.youtube.com/watch?v=QNw9q4YXR4E) (15 min) +- up until the `The Visual Studio Code REST client` section (15 min) +- - Free API for testing and prototyping. (5 min) + +## Flipped classroom videos + +- [Connecting nodejs to a database using Knex part 1 - Class 3](https://youtu.be/W5xFbiAl4bo) +- [Connecting nodejs to a database using Knex part 2 - Class 3](https://youtu.be/cacTSGU7Hrc) +- [Creating an api using nodejs and express - Class 3](https://youtu.be/i-BUdUMz6Zk) diff --git a/courses/backend/node/week2/session-plan.md b/courses/backend/node/week2/session-plan.md index e69de29b..2cc239a1 100644 --- a/courses/backend/node/week2/session-plan.md +++ b/courses/backend/node/week2/session-plan.md @@ -0,0 +1,93 @@ +# Session plan + +## Session outline + +- Database interaction + - Connecting to mysql using knex + - Executing queries + - `select`, `create`. You could let the students figure out how `delete` and `update` works + - [Code inspiration](#phonebook-database) especially focus on the promise and query part +- API + - REST + - CRUD + - Router verb `GET`, `POST`, `DELETE`, `PUT` + - Especially focus on post with `app.use(express.urlencoded({ extended: true }));` and `app.use(express.json());` + - [Code inspiration](#phonebook-api) +- Postman + - `POST`, `DELETE`, `PUT` requests +- Exercise finish concerts api + +## Code inspiration + +### Phonebook database + +- Go to the `teacher-live-coding` [repo](https://github.com/HackYourFuture-CPH/teacher-live-coding), to the relevant folder +- Copy the `.env.example` and rename the copied file to `.env` +- Run `npm install` +- Start the application by running `nodemon ./src/backend/phonebook-database-queries.js` + +Try and implement this functionality from the bottom while explaining. + +## Postman + +Postman can be used for quickly testing your APIs, but can also be configured in more advanced ways to support your development workflow. Here, you'll learn about four ways to level up your Postman game. + +### 1. Creating collections and saving requests + +Collections let you group related requests into a reusable library. This makes it easy to organize, run, and share sets of API calls. Read more on the [Official docs](https://learning.postman.com/docs/collections/collections-overview/). + +#### Exercise +Create a collection for your Snippets API. Add an unauthenticated `GET /api/snippets` request, and save it to the collection. + +1. In Postman, click **New → Collection**, and give it a meaningful name and description. +2. On the collection, click the **+** icon and create the GET request. Give it a meaningful name and **Save** it to the collection, once it's working. + +### 2. Set Up Multiple Environments + +Environments in Postman let you define sets of variables (e.g. base URLs, tokens) for different contexts. That could be your local environment, staging, and production. Switching environments changes the variable values used in your requests. +Read more on the [Official docs](https://learning.postman.com/docs/sending-requests/variables/managing-environments/). + +#### Exercise +1. In Postman, go to **Environments** (top-right environment selector) and click **Add**. +2. Name your environment your first environment `Local` +3. Add a variable called `base_url` and set it to `http://localhost:3000` (or wherver you're running your local server). +4. Update your requests to use them in the URL, like: `{{base_url}}/api/snippets`. +5. Select the environment in the dropdown to apply its variables to all requests you run. + +All details you place in variables are local by default. For additional security, mark them as sensitive, if you keep secrets or passwords in here. + +In the future, when you come to deploy your app to the web, you can create a new environment in the same way called `Production` and recreate the same variables with updated values for your deployed app. Then you can easily switch between them in Postman to test both local and production versions of the APIs. + +### 3. Managing Secrets + +You'll often need to use sensitive data in your requests, namely secrets (API keys, passwords, tokens). These should not be hard-coded in your requests for security reasons! Postman provides a **Vault** and sensitive variable settings to securely store and reuse secrets. Read more on the [Official docs](https://learning.postman.com/docs/sending-requests/postman-vault/postman-vault-secrets/). + +#### Exercise +Let's add the authenticated request to `GET /api/snippets`, so we can test returning private snippets. + +1. Open the **Postman Vault** (Vault icon or `Ctrl+Shift+V`). +2. When prompted, Postman generates a vault key - save it securely. +3. Add a new secret called `users_token` with a value from the `users.token` column in your database. +4. Now you're ready to use it in a request! Create a new GET request as you did in the first exercise, but this time add an authorization header. Where you need to reference your token, use `{{vault:users_token}}`. + +Now you can safely and securely test APIs using secrets. Test to make sure the request is working correctly, and save it to your collection. + +### 4. Create Basic Test Suites + +Postman allows you to write test scripts (in JavaScript) that validate your API responses — checking status codes, payloads, and performance. These tests can be grouped into collections and run automatically. They are a handy way to validate that your API is working correctly, and continues to work correctly as you make changes. Read more on the [Official docs](https://learning.postman.com/docs/tests-and-scripts/write-scripts/test-scripts/). + +#### Exercise +1. In a request, open the **Tests** tab. +2. Write assertions using `pm.test()` and the `pm.response` object. Here are two examples you can paste in: +```javascript +pm.test("Status code is 200", function () { + pm.response.to.have.status(200); +}); + +pm.test("Snippet has a numeric 'id' field", function () { + const json = pm.response.json(); + pm.expect(json.id).to.be.a("number"); +}); +``` +3. Write at least one additional test case for both of your requests. +4. These tests will run automatically each time you send the request. You can also run them all together, like a test suite. Click on your collection, and find the "Run" button. Make sure all the requests are checked that you wish to test. Click run, and a report will display all of test results.