diff --git a/.markdownlint.yaml b/.markdownlint.yaml index 336eceb4..e3103e29 100644 --- a/.markdownlint.yaml +++ b/.markdownlint.yaml @@ -6,6 +6,8 @@ MD026: MD033: allowed_elements: - nobr + - summary + - details MD046: style: fenced "search-replace": diff --git a/SUMMARY.md b/SUMMARY.md index 0265e09e..e6012144 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -106,6 +106,14 @@ - [Backend](courses/backend/README.md) - [Advanced JavaScript](courses/backend/advanced-javascript/README.md) - [Databases](courses/backend/databases/README.md) + - [Week 1](courses/backend/databases/week1/README.md) + - [Preparation](courses/backend/databases/week1/preparation.md) + - [Session Plan](courses/backend/databases/week1/session-plan.md) + - [Assignment](courses/backend/databases/week1/assignment.md) + - [Week 2](courses/backend/databases/week2/README.md) + - [Preparation](courses/backend/databases/week2/preparation.md) + - [Session Plan](courses/backend/databases/week2/session-plan.md) + - [Assignment](courses/backend/databases/week2/assignment.md) - [Node](courses/backend/node/README.md) - [Final Backend Project](courses/backend/final-project/README.md) diff --git a/courses/backend/databases/README.md b/courses/backend/databases/README.md index 25168515..9a5b3332 100644 --- a/courses/backend/databases/README.md +++ b/courses/backend/databases/README.md @@ -1,3 +1,26 @@ # Databases -Coming soon +This module is part of the Backend specialism course and goes deep on databases, including SQL topics like CRUD, relationships and other advanced concepts like transactions, security, and deployment. It also covers different types of databases and their use cases. + +| Week | Topic | Preparation | Lesson Plan | Assignment | +| ---- | -------------------------------------------------------------- | ------------------------------------- | ----------------------------------------------------- | ----------------------------------- | +| 1. | [Data Modeling and Relational Fundamentals](./week1/README.md) | [Preparation](./week1/preparation.md) | [Session Plan](./week1/session-plan.md) (for mentors) | [Assignment](./week1/assignment.md) | +| 2. | [Database Security and Transactions](./week2/README.md) | [Preparation](./week2/preparation.md) | [Session Plan](./week2/session-plan.md) (for mentors) | [Assignment](./week2/assignment.md) | + +## Learning Goals + +By the end of this module, you will be able to: + +- [ ] Explain the purpose of databases in web applications +- [ ] Perform basic CRUD operations using SQL +- [ ] Understand and implement relationships between tables in SQL +- [ ] Write more complex SQL queries involving joins and aggregations +- [ ] Understand the use cases for different types of databases (SQL, NoSQL, Key-Value, Graph) +- [ ] Communicate the importance of database security +- [ ] Understand the most common security threats and how to mitigate them with best practices + +## Prerequisites + +You should have completed or understand the learning goals in the following modules before proceeding: + +- [ ] [Foundation - Databases](../../foundation/databases/README.md) diff --git a/courses/backend/databases/week1/README.md b/courses/backend/databases/week1/README.md new file mode 100644 index 00000000..9f85ddf1 --- /dev/null +++ b/courses/backend/databases/week1/README.md @@ -0,0 +1,30 @@ +# Data Modeling and Relational Fundamentals + +This session is about understanding the entity relationship model, being able to use that knowledge and implement it building a database schema. With the database schema created, you should be able to create a database and test it using SQL statements. + +## Contents + +- [Preparation](./preparation.md) +- [Session Plan](./session-plan.md) (for mentors) +- [Assignment](./assignment.md) + +## Learning Goals + +By the end of this session, you will be able to: + +- [ ] Create an entity relationship diagram from a description of data requirements +- [ ] Translate a simple entity relationship diagram into a database schema +- [ ] Use SQL CRUD operations for more complex scenarios (INSERT with Foreign keys, DELETE with filters) +- [ ] Use foreign keys to establish relationships between tables +- [ ] Write SQL queries to retrieve related data using JOIN operations + +## Main Topics + +- Entity-Relationship Diagrams (ERDs): + - Introduction to ERDs + - Components: Entities, Attributes, Relationships (1:1, 1:M, M:M) +- Translating ERDs to Database Schema: + - Mapping rules for tables, columns, primary keys (PKs), and foreign keys (FKs) +- Working with Related Data: + - Foreign Keys and Constraints (ON DELETE, ON UPDATE actions) + - SQL JOIN Operations (INNER JOIN, LEFT JOIN) diff --git a/courses/backend/databases/week1/assignment.md b/courses/backend/databases/week1/assignment.md new file mode 100644 index 00000000..061096f9 --- /dev/null +++ b/courses/backend/databases/week1/assignment.md @@ -0,0 +1,121 @@ +# Assignment + +In this assignment, you'll practice working with the task management database we worked in the session. +The idea is for you to create a database from scratch, insert and update data writing queries, practice querying relationships, and finally modify the database schema to add new functionality. + +## Getting Started + +> [!TIP] +> If problems arise, remember you can ask on Slack for help. +> We encourage you to create the DB from scratch, but you can also use the provided database, [tasks.sqlite3](./session-materials/tasks.sqlite3) which has the same content as the one created in the session. + +[tasks.sql](./session-materials/tasks.sql) contains SQL statements to create a database. + +To create a database called `tasks.sqlite3` executing the SQL statements in `tasks.sql`, run the following command in your terminal: + +```shell +sqlite3 tasks.sqlite3 < session-materials/tasks.sql +``` + +> [!NOTE] +> Remember you can delete the `tasks.sqlite3` file and run the command again to recreate it from scratch. + +The script will also insert some sample data for you to work with, including users, tasks, and statuses. +And the database schema will look like this: + +```mermaid +erDiagram + USER { + int id PK + string name + string email + string phone + } + STATUS { + int id PK + string name + } + TASK { + int id PK + string title + string description + datetime created + datetime updated + date due_date + int status_id FK + } + USER_TASK { + int user_id FK + int task_id FK + } + + USER ||--o{ USER_TASK : assigns + TASK ||--o{ USER_TASK : is_assigned + STATUS ||--o{ TASK : has +``` + +## Part 1: Basic CRUD Operations + +Write SQL queries to perform the following operations: + +1. Insert a new user with your own name and email +2. Insert a new task assigned to yourself with the following attributes: + - Title: "Learn SQL" + - Description: "Practice database queries" + - Status: "In Progress" + - Due date: One week from today +3. Update the title of the task you just created to "Master SQL Basics" +4. Change the due date of your task to two weeks from today +5. Change the status of your task to "Done" +6. Delete one of the tasks in the database (choose any task) + +For each operation, save your SQL query in a text file. + +## Part 2: Working with Relationships + +Write SQL queries to answer the following questions: + +1. List all users who don't have any tasks assigned +1. Find all tasks with a status of "Done" +1. Find all overdue tasks (due_date is earlier than today) + +## Part 3: Modifying the Database Schema + +Now let's modify our database structure to add more functionality: + +1. Add a new column called `priority` to the `task` table with possible values: 'Low', 'Medium', 'High'. 💡 Remember to provide default values. +2. Update some existing tasks to have different priority values +3. Create a new table called `category` with columns: + - id (PRIMARY KEY) + - name (e.g., "Work", "Personal", "Study") + - color (e.g., "red", "blue", "green") +4. Create a linking table called `task_category` to establish a many-to-many relationship between tasks and categories: + - task_id (FOREIGN KEY to task.id) + - category_id (FOREIGN KEY to category.id) +5. Insert at least 3 categories +6. Assign categories to at least 5 different tasks + +## Part 4: Advanced Queries + +Now that you've enhanced the database, write queries to: + +1. Find all tasks in a specific category (e.g., "Work") +2. List tasks ordered by priority (High to Low) and by due date (earliest first) +3. Find which category has the most tasks +4. Get all high priority tasks that are either "In Progress" or "To Do" +5. Find users who have tasks in more than one category + +## Submission + +Submit your assignment as a single .sql file containing all your queries, clearly labeled with comments indicating which part and question each query addresses. + +Example: + +```sql +-- Part 1, Question 1: Insert a new user +INSERT INTO user (name, email, phone) VALUES ('My Name', 'my_email@example.com', '123-456-7890'); + +-- Part 1, Question 2: Insert a new task +INSERT INTO task (title, description, created, updated, due_date, status_id) +VALUES ('Learn SQL', 'Practice database queries', datetime('now'), datetime('now'), date('now', '+7 days'), 2); +``` diff --git a/courses/backend/databases/week1/preparation.md b/courses/backend/databases/week1/preparation.md new file mode 100644 index 00000000..24bc543c --- /dev/null +++ b/courses/backend/databases/week1/preparation.md @@ -0,0 +1,18 @@ +# Preparation + +## Tools Setup + +If you haven't already, install the following tools: + +- [DBeaver](https://dbeaver.io/download/), [SQlite Viewer - VSCode Extension](https://marketplace.visualstudio.com/items?itemName=qwtel.sqlite-viewer) or any other SQL client of your choice. +- (Optional) [SQLite](https://www.sqlite.org/download.html) if you want to practice using the command line. + +## Reading List + +Read/watch through this list of content before you come to the session: + +- [Database Design - Phase 1 - Analysis](https://mariadb.com/docs/general-resources/database-theory/database-design/database-design-example-phase-1-analysis) +- [Database Design - Phase 2 - Design](https://mariadb.com/docs/general-resources/database-theory/database-design/database-design-example-phase-2-design) +- [Database Design - Phase 3 - Implementation](https://mariadb.com/docs/general-resources/database-theory/database-design/database-design-example-phase-3-implementation) - Don't implement the database, just read through the example. +- [What is a foreign key?](https://mariadb.com/docs/general-resources/database-theory/relational-databases-foreign-keys) +- [Databases resources from Foundation](../../../foundation/databases/week1/preparation.md) diff --git a/courses/backend/databases/week1/session-materials/articles_example.json b/courses/backend/databases/week1/session-materials/articles_example.json new file mode 100644 index 00000000..31672a30 --- /dev/null +++ b/courses/backend/databases/week1/session-materials/articles_example.json @@ -0,0 +1,50 @@ +[ + { + "id": 1, + "authors": [ + { + "id": 1, + "name": "James Smith" + }, + { + "id": 2, + "name": "Jane Jones" + } + ], + "title": "BREAKING NEWS: Water is wet!", + "content": "Scientists have discovered that water is wet, it's amazing what.... ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.", + "tags": ["science", "breaking"] + }, + { + "id": 2, + "authors": [ + { + "id": 3, + "name": "Aliya Awad" + }, + { + "id": 4, + "name": "Igor Vladimir" + } + ], + "title": "Heavy Snowfall Expected this Weekend", + "content": "Lots of snow is expected... Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.", + "tags": ["weather", "winter"] + }, + { + "id": 3, + "authors": [ + { + "id": 2, + "name": "Jane Jones" + }, + { + "id": 5, + "name": "Kim Jensen" + } + ], + "title": "BREAKING NEWS: These 10 Clickbait Titles Are Bad for Your Health, Number 7 Will SHOCK You!", + "content": "Haha, you clicked! Minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat ", + "tags": ["clickbait", "breaking"] + } +] diff --git a/courses/backend/databases/week1/session-materials/sql-joins.png b/courses/backend/databases/week1/session-materials/sql-joins.png new file mode 100644 index 00000000..c785b0bc Binary files /dev/null and b/courses/backend/databases/week1/session-materials/sql-joins.png differ diff --git a/courses/backend/databases/week1/session-materials/tasks.sql b/courses/backend/databases/week1/session-materials/tasks.sql new file mode 100644 index 00000000..c3a20fe8 --- /dev/null +++ b/courses/backend/databases/week1/session-materials/tasks.sql @@ -0,0 +1,129 @@ +-- SQLite schema + +CREATE TABLE user ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT NOT NULL, + email TEXT NOT NULL, + phone TEXT +); + +CREATE TABLE status ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT NOT NULL +); + +CREATE TABLE task ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + title TEXT NOT NULL, + description TEXT, + created DATETIME NOT NULL, + updated DATETIME NOT NULL, + due_date DATETIME, + status_id INTEGER NOT NULL, + FOREIGN KEY (status_id) REFERENCES status(id) ON DELETE CASCADE ON UPDATE CASCADE +); + +CREATE TABLE user_task ( + user_id INTEGER NOT NULL, + task_id INTEGER NOT NULL, + PRIMARY KEY(user_id, task_id), + FOREIGN KEY (user_id) REFERENCES user(id) ON DELETE CASCADE ON UPDATE CASCADE, + FOREIGN KEY (task_id) REFERENCES task(id) ON DELETE CASCADE ON UPDATE CASCADE +); + +-- Users +INSERT INTO user (name, email, phone) VALUES ('Aarika Ellingworth', 'aellingworth0@harvard.edu', '483-396-8795'); +INSERT INTO user (name, email, phone) VALUES ('Pren Goldsworthy', 'pgoldsworthy1@spotify.com', '635-572-8467'); +INSERT INTO user (name, email, phone) VALUES ('Pablo Kisbee', 'pkisbee2@lulu.com', '790-962-8683'); +INSERT INTO user (name, email, phone) VALUES ('Rodie Duncan', 'rduncan3@quantcast.com', '646-743-6191'); +INSERT INTO user (name, email, phone) VALUES ('Aubry Polak', 'apolak4@indiatimes.com', '302-678-7931'); +INSERT INTO user (name, email, phone) VALUES ('Maryrose Meadows', 'mmeadows5@comcast.net', '251-524-6594'); +INSERT INTO user (name, email, phone) VALUES ('Pavel Brushneen', 'pbrushneen6@techcrunch.com', '316-170-3640'); +INSERT INTO user (name, email, phone) VALUES ('Hedy Gerault', 'hgerault7@nymag.com', '176-177-5579'); +INSERT INTO user (name, email, phone) VALUES ('王秀英', 'wang.xiuying@weebly.com', '891-952-6749'); +INSERT INTO user (name, email, phone) VALUES ('إلياس', 'elias@github.com', '202-517-6983'); +INSERT INTO user (name, email, phone) VALUES ('Donald Duck', 'donald@duck.com', NULL); + + +-- Tasks +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Wash clothes', 'Title says it all.', '2017-10-25 06:54:16', '2017-10-15 13:05:09', null, 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Become a billionaire', 'This should not take long, just invent a time machine, travel back to 2010 and buy bitcoin', '2017-09-26 03:06:46', '2017-10-08 06:14:31', '2017-12-22 20:58:03', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Plan meeting with London office', 'We will probably use skype', '2017-10-04 18:07:37', '2017-10-14 16:01:31', '2017-12-05 19:42:15', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Order groceries online', 'The fridge is almost empty, we need eggs and milk', '2017-09-20 19:34:43', '2017-10-15 23:35:45', '2017-12-24 16:00:46', 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Empty the mailbox', NULL, '2017-09-27 15:17:08', '2017-10-08 17:31:16', null, 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Fix the flat tire on the bike', 'Tools are in the garage', '2017-09-13 23:16:30', '2017-10-06 04:03:52', '2017-12-07 11:51:11', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Wash the car', NULL, '2017-10-06 19:39:16', '2017-10-03 04:49:05', '2017-12-04 17:43:16', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Walk the dog', NULL, '2017-09-03 02:47:17', '2017-10-12 18:40:08', null, 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Write a book', 'Maybe something about dragons?', '2017-10-11 06:14:01', '2017-10-17 12:19:08', '2017-12-21 20:18:05', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Do HackYourFuture assignment', NULL, '2017-10-04 13:55:16', '2017-10-10 00:18:05', '2017-12-19 17:01:10', 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Iron shirts', NULL, '2017-09-23 03:59:58', '2017-10-19 08:30:48', '2017-12-08 11:00:35', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Water the potted plants', 'Maybe they need fertilizer as well', '2017-09-29 23:38:42', '2017-10-08 04:24:53', null, 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Buy wine for the birthday party', 'Both red and white wine', '2017-10-10 14:57:22', '2017-10-14 14:03:30', '2017-12-10 23:43:56', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Buy gift for Paul', 'He could use a shirt or a tie and some socks', '2017-09-09 05:22:08', '2017-10-17 15:58:05', '2017-12-04 20:45:18', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Change lightbulb in hallway', 'Should be an LED bulb', '2017-10-01 19:07:35', '2017-10-03 10:02:27', '2017-12-08 17:09:03', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Wash windows', NULL, '2017-10-02 22:15:17', '2017-10-07 22:31:35', '2017-12-06 03:36:09', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Setup salary databases for accounting', 'Use MySQL', '2017-10-25 05:35:33', '2017-10-10 23:22:33', '2017-12-05 00:19:08', 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Learn how databases work', NULL, '2017-09-06 03:16:47', '2017-10-10 16:56:58', '2017-12-18 05:08:05', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Make the databases perform better', 'It should be possible to optimize the indexes', '2017-10-03 09:27:20', '2017-10-01 16:27:46', '2017-12-01 13:28:35', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Buy beer for the company party', '2 or 3 cases should be enough', '2017-10-08 01:39:02', '2017-10-13 23:07:41', null, 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Knit sweater', NULL, '2017-09-22 17:14:55', '2017-10-08 09:01:35', '2017-12-15 20:33:57', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Charge electric bicycle', 'It sucks to ride it without a battery!', '2017-10-10 12:25:07', '2017-10-07 21:45:01', '2017-12-10 19:02:17', 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Buy new phone', 'The battery in the current one only lasts 5 hours 😞', '2017-09-17 00:25:34', '2017-10-09 11:48:12', null, 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Ride bike around Sjælland', 'Remember rainclothes and tire repair kit!', '2017-10-20 19:21:13', '2017-10-07 01:38:06', '2017-12-19 15:08:18', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Look at apartments in Ørestad', '2 or 3 rooms', '2017-10-30 09:47:00', '2017-10-19 06:11:26', null, 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Empty Mr Fluffy''s litterbox', NULL, '2017-09-28 03:09:06', '2017-10-13 10:38:34', '2017-12-20 23:37:18', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Buy new dining room table and chairs', 'Ikea has some on sale', '2017-09-21 12:02:34', '2017-10-02 02:05:11', '2017-12-06 00:14:30', 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Renew buscard', '3 zones', '2017-10-07 22:47:51', '2017-10-09 15:50:03', '2017-12-01 14:25:40', 2); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Sign up for linkedin', 'Make the CV awesome! 😄', '2017-09-04 00:57:47', '2017-10-18 18:07:48', '2017-12-07 23:04:38', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Remove facebook from phone', 'To avoid interruptions when working', '2017-10-26 17:15:07', '2017-10-13 03:36:47', '2017-12-19 11:10:02', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Backup databases to external disk', 'Remember to store the disk in another physical location', '2017-09-09 17:32:33', '2017-10-01 21:18:59', '2017-12-23 14:21:01', 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Put up the new lamp in the hallway', NULL, '2017-10-15 05:45:54', '2017-10-16 14:05:35', '2017-12-29 02:29:26', 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Hang up paintings in living room', NULL, '2017-09-10 05:36:11', '2017-10-09 17:40:42', null, 3); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Buy plane ticket to Auckland', 'Check prices online first!', '2017-09-05 09:07:22', '2017-10-15 09:36:06', '2017-12-07 11:10:05', 1); +INSERT INTO task (title, description, created, updated, due_date, status_id) VALUES ('Learn about NoSQL databases', 'MongoDB, CouchDB, etc.', '2017-10-20 01:41:53', '2017-10-04 07:19:56', '2017-12-23 10:13:42', 2); + +-- Users-tasks +INSERT INTO user_task (user_id, task_id) VALUES(1, 5); +INSERT INTO user_task (user_id, task_id) VALUES(1, 35); +INSERT INTO user_task (user_id, task_id) VALUES(1, 11); +INSERT INTO user_task (user_id, task_id) VALUES(2, 4); +INSERT INTO user_task (user_id, task_id) VALUES(2, 26); +INSERT INTO user_task (user_id, task_id) VALUES(2, 29); +INSERT INTO user_task (user_id, task_id) VALUES(3, 22); +INSERT INTO user_task (user_id, task_id) VALUES(3, 13); +INSERT INTO user_task (user_id, task_id) VALUES(3, 19); +INSERT INTO user_task (user_id, task_id) VALUES(4, 24); +INSERT INTO user_task (user_id, task_id) VALUES(4, 20); +INSERT INTO user_task (user_id, task_id) VALUES(5, 20); +INSERT INTO user_task (user_id, task_id) VALUES(5, 18); +INSERT INTO user_task (user_id, task_id) VALUES(5, 15); +INSERT INTO user_task (user_id, task_id) VALUES(6, 10); +INSERT INTO user_task (user_id, task_id) VALUES(6, 7); +INSERT INTO user_task (user_id, task_id) VALUES(6, 27); +INSERT INTO user_task (user_id, task_id) VALUES(7, 33); +INSERT INTO user_task (user_id, task_id) VALUES(7, 18); +INSERT INTO user_task (user_id, task_id) VALUES(7, 23); +INSERT INTO user_task (user_id, task_id) VALUES(8, 26); +INSERT INTO user_task (user_id, task_id) VALUES(8, 30); +INSERT INTO user_task (user_id, task_id) VALUES(8, 11); +INSERT INTO user_task (user_id, task_id) VALUES(9, 34); +INSERT INTO user_task (user_id, task_id) VALUES(9, 15); +INSERT INTO user_task (user_id, task_id) VALUES(9, 1); +INSERT INTO user_task (user_id, task_id) VALUES(10, 29); +INSERT INTO user_task (user_id, task_id) VALUES(10, 16); +INSERT INTO user_task (user_id, task_id) VALUES(10, 1); +INSERT INTO user_task (user_id, task_id) VALUES(11, 26); +INSERT INTO user_task (user_id, task_id) VALUES(11, 27); +INSERT INTO user_task (user_id, task_id) VALUES(11, 17); +INSERT INTO user_task (user_id, task_id) VALUES(11, 2); +INSERT INTO user_task (user_id, task_id) VALUES(1, 3); +INSERT INTO user_task (user_id, task_id) VALUES(2, 6); +INSERT INTO user_task (user_id, task_id) VALUES(3, 8); +INSERT INTO user_task (user_id, task_id) VALUES(4, 9); +INSERT INTO user_task (user_id, task_id) VALUES(5, 12); +INSERT INTO user_task (user_id, task_id) VALUES(6, 14); +INSERT INTO user_task (user_id, task_id) VALUES(7, 21); +INSERT INTO user_task (user_id, task_id) VALUES(8, 25); +INSERT INTO user_task (user_id, task_id) VALUES(9, 28); +INSERT INTO user_task (user_id, task_id) VALUES(10, 31); +INSERT INTO user_task (user_id, task_id) VALUES(11, 32); diff --git a/courses/backend/databases/week1/session-materials/tasks.sqlite3 b/courses/backend/databases/week1/session-materials/tasks.sqlite3 new file mode 100644 index 00000000..e9cce3f6 Binary files /dev/null and b/courses/backend/databases/week1/session-materials/tasks.sqlite3 differ diff --git a/courses/backend/databases/week1/session-plan.md b/courses/backend/databases/week1/session-plan.md new file mode 100644 index 00000000..d2a9569e --- /dev/null +++ b/courses/backend/databases/week1/session-plan.md @@ -0,0 +1,307 @@ +# Session Plan + +- Focus on practical modeling exercises and SQL practice +- Ensure trainees understand how to translate business requirements into database structures +- Emphasize database design principles and when to use different relationships +- Build toward the assignment step by step + +## Table of Contents + +1. The main idea is to start with a requirement +2. Build the Entity-Relationship Diagram (ERD) the trainees +3. Translate the ERD into SQL statements + - See the limitations of the initial design +4. Introduce foreign keys (status) + - Add many-to-many relationships (user-task) +5. Practice querying relationships +6. Let the trainees practice designing a database based on [articles_example.json](session-materials/articles_example.json) + +> [!NOTE] +> Teaching Format: +> +> - **DEMO** = Mentor shows on screen, trainees observe +> - **EXERCISE** = Trainees work on their own machines + +## Entity-Relationship Modeling + +### What is an ERD and why do we use it? + +- Entities, attributes, relationships +- Types of relationships (1:1, 1:M, M:M) +- Primary keys and foreign keys + +### DEMO: ERD Task Management System + +- Mentor leads ERD creation with trainees input using Excalidraw/Draw.io +- Ask trainees to suggest entities, attributes, and relationships + +**Scenario**: Design a simple task management system with these requirements: + +- Users can have multiple tasks +- Tasks have a title, description, and due date +- Tasks can have one of three statuses: "Not started", "In progress", "Done" +- Each task is assigned to exactly one user + +## Translating ERDs to Database Schema + +### DEMO: Moving from conceptual to physical model + +- How do you actually store this in a database? +- Tables, columns, constraints +- Data types in SQL +- Normalization basics (focus up to 3rd normal form) + - Example: move `status` to a separate table to avoid redundancy + +### EXERCISE 1: Database Creation (10/15 minutes) + +Using the ERD from the previous exercise, trainees need to write CREATE TABLE statements for storing users and tasks with statuses. + +```sql +CREATE TABLE user ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT NOT NULL, + -- Your columns here +); + +CREATE TABLE task ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + -- Your columns here +); +``` + +**Hints:** + +- Use INTEGER PRIMARY KEY for IDs +- Don't forget NOT NULL constraints +- created DATETIME NOT NULL, + +- Use appropriate data types (TEXT, INTEGER, DATETIME) + +### DEMO: Database creation solution + +
+Mentor shows the complete solution + +```shell +# Create a new SQLite database file or use a GUI tool +touch test.sqlite3 + +# Create users table +CREATE TABLE user ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT NOT NULL, + email TEXT NOT NULL, + phone TEXT +); + +CREATE TABLE task ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + title TEXT NOT NULL, + description TEXT, + created DATETIME NOT NULL, + updated DATETIME NOT NULL, + due_date DATETIME, + status TEXT NOT NULL +); +``` + +
+ +#### Add some sample data + +```sql +INSERT INTO user (name, email, phone) VALUES + ('John Doe', '', '+4512345678'), + ('Jane Smith', 'jane@gmail.com', '+4512345679'); + +INSERT INTO task (title, description, created, updated, due_date, status) VALUES + ('Study SQL Queries', 'Practice writing SQL queries for data retrieval', datetime('now'), datetime('now'), '2025-08-02', 'Done'), + ('Learn Database Design', 'Study ER modeling and normalization', datetime('now'), datetime('now'), '2025-08-10', 'Not started'), + ('Write Unit Tests', 'Add test coverage for user authentication', datetime('now'), datetime('now'), '2025-08-05', 'Not started'), + ('Deploy Application', 'Set up production environment', datetime('now'), datetime('now'), '2025-08-20', 'Not started'); +``` + +At this moment, the database works but... is not very useful. 😅 It still has some limitations. + +1. The status is repeated in every task +2. The users are not linked to tasks. + +> ![NOTE] +> Ask trainees for possible improvements. + +## Defining Relationships + +### DEMO: Understanding Foreign Keys and Relationships + +**Key concepts:** + +- Foreign keys maintain data integrity +- Many-to-many relationships require linking tables +- JOINs connect related data across tables + +The steps we will take: + +#### 1. Move `status` to a separate table to avoid redundancy + +```sql +CREATE TABLE status ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT NOT NULL UNIQUE +); + +-- Insert initial statuses +INSERT INTO status (name) VALUES ('Not started'), ('In progress'), ('Done'); + +-- Modify task table to use status_id +ALTER TABLE task ADD COLUMN status_id INTEGER REFERENCES status(id) DEFAULT 1; + +-- Update existing tasks to use status_id +UPDATE task SET status_id = 1 WHERE status = 'Not started'; +UPDATE task SET status_id = 2 WHERE status = 'In progress'; +UPDATE task SET status_id = 3 WHERE status = 'Done'; + +-- Finally, remove old status column after migration +ALTER TABLE task DROP COLUMN status; +``` + +> [!IMPORTANT] +> What we have done: +> +> - Created a `status` table to store task statuses. The benefit? Avoids redundancy and allows easy updates +> - Modified the `task` table to reference `status_id` from the `status` table + +Normally this is known as [database migrations](https://en.wikipedia.org/wiki/Schema_migration). We'll not cover this in detail, but it's a common practice in real-world applications. + +#### 2. Create a linking table for users and tasks to handle many-to-many relationships + +```sql +-- Many-to-many relationship between users and tasks +CREATE TABLE user_task ( + user_id INTEGER NOT NULL, + task_id INTEGER NOT NULL, + PRIMARY KEY (user_id, task_id), + FOREIGN KEY (user_id) REFERENCES user(id) ON DELETE CASCADE, + FOREIGN KEY (task_id) REFERENCES task(id) ON DELETE CASCADE +); + +-- Link a user to a task +INSERT INTO user_task (user_id, task_id) VALUES (1, 1); +``` + +> [!IMPORTANT] > **What we have done:** +> +> - Created an intermediary table called `user_task` to connect users and tasks. +> - This allows a user to have multiple tasks and a task to be assigned to multiple users. + +This is usually called a **linking table** or **junction table**. It allows us to represent many-to-many relationships in a relational database. [Wikipedia]() + +## Querying Relationships + +Now that we have our tables set up, let's practice querying related data. + +### DEMO: Querying related data + +- Introduction to JOINs (INNER, LEFT) +- Filtering with WHERE clauses +- Using foreign keys effectively + +#### Anatomy of a JOIN query + +```sql +-- Example: Get all tasks with their status ids +SELECT title, status_id FROM task t; + +-- Introducing JOIN to show the status name +SELECT title, status.name FROM task +JOIN status ON task.status_id = status.id; + +-- Joining even further to get more information +SELECT task.title, status.name AS status_name, user.name AS user_name +FROM task +JOIN status ON task.status_id = status.id +JOIN user_task ON task.id = user_task.task_id +JOIN user ON user_task.user_id = user.id +WHERE user.phone LIKE '+45%'; -- Filter by phone number starting with +45 +``` + +> ![IMPORTANT] > **What we have done:** +> Used JOINs to connect related tables: +> +> - `JOIN` connects the tables +> - `ON` specifies how they are related (foreign keys) +> - `SELECT` retrieves specific columns from both tables +> - `AS` allows renaming columns for clarity + +Why does the query return only 1 row? +The default `JOIN` is an **INNER JOIN**, which only **returns rows where there is a match in both tables**. + +It's good to know there are several types of JOINs: ![SQL Joins](./session-materials/sql-joins.png) + +### EXERCISE 3: Practice more advanced queries (15 minutes) + +> ![TIP] +> Provide the trainees with the sample inserts in [tasks.sql](session-materials/tasks.sql) to have more data in their database. + +**Trainees practice writing these queries using the sample data:** + +1. Get all tasks assigned to a specific user name. +2. Find all users working on the task 'Deploy to production' +3. Find how many tasks each user is responsible for. Hint: `COUNT(), GROUP BY` +4. Find how many completed tasks each user has. Order them in descending order. + +### DEMO: Show the solution + +
Queries solution + +```sql +-- 1. Get all tasks assigned to 'John Doe' +SELECT t.title, t.description, s.name AS status +FROM task t +JOIN user_task ut ON t.id = ut.task_id +JOIN user u ON ut.user_id = u.id +JOIN status s ON t.status_id = s.id +WHERE u.name = 'John Doe'; + +-- 2. Find all users working on 'Deploy Application' +-- #TODO - Add more users working on this task +SELECT u.name +FROM user u +JOIN user_task ut ON u.id = ut.user_id +JOIN task t ON ut.task_id = t.id +WHERE t.title = 'Deploy Application'; + +-- 3. Find how many tasks each user is responsible for +SELECT u.name, COUNT(ut.task_id) AS task_count +FROM user u +LEFT JOIN user_task ut ON u.id = ut.user_id +GROUP BY u.name; + +-- 4. Find how many completed tasks each user has +SELECT u.name, COUNT(t.id) AS completed_tasks +FROM user u +LEFT JOIN user_task ut ON u.id = ut.user_id +LEFT JOIN task t ON ut.task_id = t.id AND t.status_id = 3 +GROUP BY u.name -- Show what happens if we comment this line +ORDER BY completed_tasks DESC; +``` + +
+ +## EXERCISE 4: Design and implement a database for existing data + +Design an ER model and implement the respective database for the data in [this file](session-materials/articles_example.json). + +Remember: + +- Don't worry if you can't do every step perfectly. +- The important thing is to understand the main ideas. +- Take your time and ask questions if you're confused. + +### Steps + +1. Analyze the JSON structure +2. Identify entities and relationships +3. Create an ERD +4. Translate to CREATE TABLE statements +5. Insert sample data +6. Write queries to retrieve information diff --git a/courses/backend/databases/week2/README.md b/courses/backend/databases/week2/README.md new file mode 100644 index 00000000..0db975d6 --- /dev/null +++ b/courses/backend/databases/week2/README.md @@ -0,0 +1,55 @@ +# Advanced Database Features + +This session is about understanding more advanced database features through practical examples. We'll use aggregate functions to create business reports, explore database security through hands-on SQL injection demonstrations, and learn about transactions to ensure data integrity. We'll also cover different database types and deployment strategies. + +## Contents + +- [Preparation](./preparation.md) +- [Session Plan](./session-plan.md) (for mentors) +- [Assignment](./assignment.md) + +## Learning Goals + +By the end of this session, you will be able to: + +- [ ] Use aggregate functions (COUNT, AVG, SUM, MIN, MAX) with GROUP BY for business reporting +- [ ] Identify and prevent SQL injection vulnerabilities in web applications +- [ ] Understand and apply database transactions to ensure data integrity using ACID properties +- [ ] Recognize when to use different types of databases (relational vs NoSQL) +- [ ] Deploy a database application to render.com with proper environment configuration + +## Main Topics + +### Aggregate Functions for Business Reporting + +- Using COUNT, AVG, SUM, MIN, MAX for dashboard statistics +- Comparing inefficient application-level calculations vs efficient SQL aggregations +- Practical examples: tasks per user, task status distribution + +### Database Security and SQL Injection + +- Hands-on demonstration of SQL injection vulnerabilities +- Data extraction and manipulation through injection attacks +- Security best practices: parameterized queries, input validation, security audits + +### Transactions and Data Integrity + +- Real-world scenario: transferring task ownership between users +- Demonstrating data corruption without transactions +- ACID properties (Atomicity, Consistency, Isolation, Durability) +- SQL transaction example: `BEGIN TRANSACTION, COMMIT, ROLLBACK` + +### Database Types Overview + +- When NOT to use relational databases +- NoSQL alternatives and their use cases: + - **Key-Value Stores (Redis)** + - **Document Stores (MongoDB)** + - **Graph Databases (Neo4j)** + - **Time-Series Databases** + +### Database Deployment + +- Migrating from local SQLite to PostgreSQL on render.com +- Environment variables for database connections +- Production vs development database considerations diff --git a/courses/backend/databases/week2/assignment.md b/courses/backend/databases/week2/assignment.md new file mode 100644 index 00000000..3cf80361 --- /dev/null +++ b/courses/backend/databases/week2/assignment.md @@ -0,0 +1,3 @@ +# Assignment + +## TODO diff --git a/courses/backend/databases/week2/preparation.md b/courses/backend/databases/week2/preparation.md new file mode 100644 index 00000000..0eae3629 --- /dev/null +++ b/courses/backend/databases/week2/preparation.md @@ -0,0 +1,6 @@ +# Preparation + +- [Hackers & CyberAttacks - SQL Injection](https://youtu.be/_GzE99AmAQU?t=376) (6 min) +- [ACID & Concurrency Control with Transactions](https://mariadb.com/docs/general-resources/database-theory/acid-concurrency-control-with-transactions) (Feel free to explore some other resources from this page, they're pretty valuable!) +- [SQL Aggregate Functions](https://www.sqltutorial.org/sql-aggregate-functions/) +- [Concurrency - Transactions and Race Conditions](https://cs50.harvard.edu/sql/notes/5/#concurrency) diff --git a/courses/backend/databases/week2/session-materials/local-vs-prod-environments.png b/courses/backend/databases/week2/session-materials/local-vs-prod-environments.png new file mode 100644 index 00000000..96a264b8 Binary files /dev/null and b/courses/backend/databases/week2/session-materials/local-vs-prod-environments.png differ diff --git a/courses/backend/databases/week2/session-plan.md b/courses/backend/databases/week2/session-plan.md new file mode 100644 index 00000000..5e920d5c --- /dev/null +++ b/courses/backend/databases/week2/session-plan.md @@ -0,0 +1,319 @@ +# Session Plan - Week 2 + +- The idea is to build on the previous week's knowledge, focusing on advanced database features and practical applications. +- Use the same task database throughout all examples for consistency +- Use problem-based approach: show issues, guide trainees to solutions + +## Requirements: Setup DB and start the example API + +### Mentor Instructions + +You need to have the example API running to demonstrate concepts and exercises. +Share your screen and setup the session materials: + +```shell +git clone https://github.com/HackYourFuture-CPH/hyf-assignment-template.git +cd courses/backend/databases/ +``` + +### Trainee Instructions + +- They should have their `tasks.sqlite3` database from Week 1 ready to use with a GUI tool. [Tools setup - Week 1](../week1/preparation.md#tools-setup) +- They should follow the same instructions but on their own fork of the assignment repo. +- The materials for this session are located in `courses/backend/databases/` + +## Aggregate Functions + +### Reporting + +- A common business need: "We need reports from our task management system" +- Show inefficient approach: calculating stats in application code + - + - Show code in `/tasks-per-user-unoptimized` endpoint +- Introduce SQL aggregates as a more performant solution + - Show code in `/tasks-per-user` and `/status-distribution` endpoints + +### Key Aggregate Functions + +Other examples of questions we can answer with aggregates: + +- **COUNT**: How many tasks were created? +- **SUM**: Total estimated hours across tasks +- **AVG**: Average completion time +- **MIN/MAX**: Earliest/latest due dates +- **GROUP BY**: Essential for aggregation + +### Exercise: Write Your Own Aggregate Queries + +#### Trainees work with their tasks.sqlite3 from Week 1 + +Write SQL queries to answer these questions: + +1. How many tasks are overdue? (due_date < today) +2. What's the average number of tasks per user? +3. Which status has the most tasks? +4. Find the user with the most completed tasks. + +#### Solutions discussion + +
+Click to see the solutions + +This can be executed directly in the SQLite command line or any SQLite client. + +```sql +-- Count overdue tasks +SELECT COUNT(*) AS overdue_count +FROM task +WHERE due_date < DATE('now'); + +-- Average tasks per user +SELECT AVG(task_count) AS average_tasks +FROM ( + SELECT user_id, COUNT(*) AS task_count + FROM user_task + GROUP BY user_id +); + +-- Status with most tasks +SELECT s.name, COUNT(*) AS task_count +FROM task t +JOIN status s ON t.status_id = s.id +GROUP BY s.id, s.name +ORDER BY task_count DESC +LIMIT 1; + +-- User with most completed tasks (status_id = 3 for 'Done') +SELECT u.name, COUNT(*) AS completed_tasks +FROM user u +JOIN user_task ut ON u.id = ut.user_id +JOIN task t ON ut.task_id = t.id +WHERE t.status_id = 3 +GROUP BY u.id, u.name +ORDER BY completed_tasks DESC +LIMIT 1; +``` + +
+ +--- + +## Database Security + +### SQL Injection Demo + +- The idea is to show the vulnerable search endpoint and how it can be exploited + +Normal search: + +Leak user data: + +### Exercise: SQL Injection Attack Practice + +#### Trainees try to exploit the vulnerable endpoint + +Using the running API, try these attacks: + +1. Extract all user emails +2. Try to delete data (see what happens) +3. Attempt to find hidden information + +#### Attack strings to try + +```sql +' OR '1'='1 +'; DROP TABLE tasks; -- +' UNION SELECT * FROM user -- +``` + +### Security Best Practices + +- Show how it's fixed in the `search/secure` endpoint +- Always use parameterized queries or ORMs which make things easier for developers +- Mention the importance of validation, both in client and server + +## Database Types Overview + +### When NOT to use relational databases? + +- **Key-Value Stores (Redis)**: Caching, real-time features +- **Document Stores (MongoDB)**: Flexible schemas, JSON-like data +- **Graph Databases (Neo4j)**: Social networks, recommendation engines +- **Time-Series**: IoT sensor data, financial metrics + +## Deployment Overview + +### Development vs Production: Understanding the Journey + +![Local vs Prod environment](./session-materials/local-vs-prod-environments.png) + +> [!NOTE] +> The diagram illustrates the difference between local development and production environments. + +#### Why do we care about different environments? + +- We use them to test and develop our applications in a safe space without affecting real users or data. +- They help us identify and fix issues before deployment. + +#### Refreshing what we refer when we say "the cloud" + +- A remote server (computer) running 24/7 somewhere else +- Multiple users can connect simultaneously +- Databases run on these servers +- Your data needs to be accessible from anywhere + +#### Why do we need to migrate to another database? + +| Aspect | SQLite (Dev) | PostgreSQL (Production) | +| ------------------- | ----------------------- | ----------------------------- | +| Location | Local file | Remote server | +| Use case | Development, small apps | Production, real applications | +| Type of information | Testing, prototyping | Real user data, critical info | +| Scaling | Limited | Excellent | + +### Demo time! + +> [!NOTE] +> Guide the trainees through the steps, explaining each part. +> You can choose to show it entirely and have them replicate later or to give them some time to follow along. + +We're going to recreate our local SQLite database to a remote PostgreSQL database on Render.com + +#### Step 1: Create Postgres DB on Render + +Follow the steps from the hyf-project-template: + +#### Step 2: Recreate Database using DBeaver + +We'll use DBeaver to connect to the remote PostgreSQL database and run SQL commands to recreate our schema and data. + +1. Connect to your PostgreSQL database on Render: + - Open DBeaver + - New Connection (Ctrl + Shift + N) → PostgreSQL + - Fill in connection details from Render's "Connection Info" page + - Test connection + +2. Create a new SQL script: + - Right-click on your PostgreSQL connection → SQL Editor → New SQL Script + - Copy the provided PostgreSQL-compatible SQL (see `tasks-postgres.sql`) +3. Execute the script: + - Paste the entire SQL content + - Click **Execute SQL Script** (Not "Execute SQL Statement") + - Should see an output saying the number of queries executed and the updated rows + +4. Verify: + - Disconnect and reconnect to refresh + - Expand Tables to see: `user`, `task`, `status`, `user_task` + - Right-click any table → View Data to verify records + +#### Step 3: Environment Variables + +Environment variables allow us to configure our application differently based on where it's running (development vs production) without changing code. + +Create a `.env` file in the `example-api` folder + +```bash +# .env file for local development +NODE_ENV=development + +# For production testing (get this from Render) +DATABASE_URL=postgresql://user:password@host:5432/database +``` + +> [!IMPORTANT] +> Never commit `.env` files to git! They contain sensitive credentials. Always add `.env` to your `.gitignore` file. + +##### Database Configuration Code + +Show the trainees how the code switches between databases: + +```javascript +// example-api/index.js + +// Development: local SQLite +const developmentConfig = { + client: "sqlite3", + //... +}; + +// Production: remote PostgreSQL +const productionConfig = { + client: "pg", + //... +}; +``` + +#### Step 4: Test Both Environments + +Now we'll test our API with both databases to see the differences. + +##### Testing development (SQLite) + +```bash +npm run dev +``` + +Open your browser: + +- + +What's happening + +- Reading from local `tasks.sqlite3` file +- Perfect for development and testing + +##### Testing production (PostgreSQL) + +```bash +# Update .env to use production +# DATABASE_URL=your-render-database-url + +npm run prod +``` + +Open your browser to the same endpoints: + +- + +What's happening: + +- Reading from remote PostgreSQL on Render +- Slightly slower (network latency) +- Same data, different database engine + +> [!TIP] +> You can show the diagram again to illustrate how you are connecting to different databases based on environment. + +### Key Concepts to Emphasize + +#### Environment Variables + +- Never hardcode database credentials +- Different configs for dev/staging/production +- `.env` file locally, Render UI for production +- Always add `.env` to `.gitignore` + +#### Migration Best Practices + +- Always backup before migrating +- Monitor after deployment +- Keep development and production schemas in sync + +#### Alternative: Knex Migrations (Optional) + +> [!TIP] +> For production projects, you'll likely use migration tools like Knex.js to version-control your schema changes. This is beyond today's scope but worth exploring for your projects. + +#### Questions to Explore + +- What happens if the remote database goes down? +- How do we update the database after the first setup? +- Can multiple developers work with the same production database? + +## Summary & Q&A + +### Key takeaways + +- SQL aggregates are your friends for calculations: reporting, dashboard stats +- Security: Never trust user input, always validate. +- Choose the right database for your use case