Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #19

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open

Dev #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
bafcb5e
Added: files repository
ccornali611 Mar 14, 2024
9d37ae7
Updated: seed data, added username
ccornali611 Mar 14, 2024
f094761
Updated: seed script to use FileRepository methods
ccornali611 Mar 14, 2024
e4fa074
Added: file upload helper
ccornali611 Mar 14, 2024
be5ca6d
Added: middleware to send username in request
ccornali611 Mar 14, 2024
42c0a47
Updated: get username endpoint to get username from request
ccornali611 Mar 14, 2024
e3d075c
Updated: impleted using FileRepository on endpoints
ccornali611 Mar 14, 2024
9891d5d
Updated: put file endpoint, use FileUploadHelper to generate duplicat…
ccornali611 Mar 14, 2024
a992bcd
Added: test data generation helper function
ccornali611 Mar 15, 2024
fae6d52
Added: test validation function
ccornali611 Mar 15, 2024
2956f0d
Updated: tests to use helper functions
ccornali611 Mar 15, 2024
452bbfb
Added: test for getting different user files
ccornali611 Mar 15, 2024
2ce6c7f
Added: tests for dup upload
ccornali611 Mar 15, 2024
16c225d
Added: tests for dup upload to fill gaps
ccornali611 Mar 15, 2024
b82d970
Added: add'l dup tests
ccornali611 Mar 15, 2024
a6512fa
Added: test for filenaming edgecases
ccornali611 Mar 15, 2024
abcfcd3
Updated: fileRepository and server index, display paginated results
ccornali611 Mar 15, 2024
31b618c
Added: pagination tests
ccornali611 Mar 15, 2024
2928b88
Updated: db test file
ccornali611 Mar 15, 2024
4a0f9dd
Updated: endpoint get files, display paginated results
ccornali611 Mar 15, 2024
044eba6
Updated: seed.db file
ccornali611 Mar 15, 2024
0e94853
Added: todo for fix
ccornali611 Mar 15, 2024
7f230cf
Added: notes to README
ccornali611 Mar 15, 2024
98772ed
small fixes
ccornali611 Mar 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,14 @@ You can add new files!
<img width="636" alt="Screen Shot 2020-05-07 at 12 30 03 PM" src="https://user-images.githubusercontent.com/69169/81336645-8cb1a780-905e-11ea-86ce-f882a2d5fd46.png">

That's all it does without your help!


#### Notes
I configured the application to only display files uploaded by the user
If there was functionality on the front end to look at files for a different user, that is possible
Some of changes I would make if this was truly for production:
1) Use pagination to display the files, have buttons for next, back and clicking on the individual pages
2) Allow user to sort by title and date uploaded (asc or desc)
- add updated_at and created_at columns to "files" table to facilitate
3) Have table for user and add `user_id` colmumn in "files" table
4) Implement authentication middleware and not get username from os
32 changes: 7 additions & 25 deletions scripts/create-seed-db.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ const fs = require('fs')
const path = require('path')
const Database = require('better-sqlite3')
const seedData = require('./seed.json')
const FileRepository = require('../src/db/repositories/fileRepository')

const SEED_PATH = path.join(__dirname, '..', 'src', 'db', 'seed.db')

Expand All @@ -12,34 +13,15 @@ function main () {
}

const db = new Database(SEED_PATH)
createTables(db)
addSeedData(db)
const fileRepository = new FileRepository(db)
fileRepository.createTable()
addSeedData(fileRepository)
}

function createTables (db) {
console.log('Creating files table')
db.prepare(`
CREATE TABLE files (
id INTEGER PRIMARY KEY AUTOINCREMENT,
description TEXT not null,
filename TEXT not null,
mimetype TEXT not null,
src TEXT not null
)
`).run()
}

function addSeedData (db) {
console.log('Inserting seed data')
const insertFiles = db.prepare(`
INSERT INTO files
(description, filename, mimetype, src)
VALUES
(@description, @filename, @mimetype, @src)
`)

function addSeedData (fileRepository) {
console.log('Inserting seed data');
for (const file of seedData.files) {
insertFiles.run(file)
fileRepository.insertFile(file.description, file.filename, file.mimetype, file.src, file.username);
}
}

Expand Down
13 changes: 9 additions & 4 deletions scripts/seed.json

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions src/db/index.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
const fs = require('fs')
const path = require('path')
const Database = require('better-sqlite3')
const FileRepository = require('db/repositories/fileRepository')

const SEED_DB = path.join(__dirname, 'seed.db')
const DEV_DB = path.join(__dirname, 'dev.db')
Expand All @@ -27,5 +28,7 @@ module.exports = {
resetToSeed () {
copySeedDB({ force: true })
db = new Database(localDBPath)
const fileRepository = new FileRepository(db)
fileRepository.createTable()
},
}
222 changes: 222 additions & 0 deletions src/db/repositories/fileRepository.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
/**
* Represents a repository for managing files in the database
*/
class FileRepository {
constructor (database) {
this.db = database
this.createTable()
}

/**
* Creates the "files" table in the database if it does not exist
*/
createTable () {
this.db.prepare(`
CREATE TABLE IF NOT EXISTS files (
id INTEGER PRIMARY KEY AUTOINCREMENT,
description TEXT NOT NULL,
filename TEXT NOT NULL,
mimetype TEXT NOT NULL,
src TEXT NOT NULL,
username TEXT NOT NULL
);
`).run()
const schema = this.db.prepare("PRAGMA table_info('files')").all()
const usernameColumnExists = schema.some((column) => column.name === 'username')

if (!usernameColumnExists) {
this.db.prepare(`
ALTER TABLE files ADD COLUMN username TEXT;
`).run()
}
// add column for duplicate count and orig_hashed_filename
this.db.prepare(`
CREATE INDEX IF NOT EXISTS idx_filename ON files (filename);
`).run()

this.db.prepare(`
CREATE INDEX IF NOT EXISTS idx_filename ON files (filename);
`).run()

this.db.prepare(`
CREATE INDEX IF NOT EXISTS idx_username ON files (username);
`).run()

this.db.prepare(`
CREATE INDEX IF NOT EXISTS idx_src ON files (src);
`).run()
}

/**
* Inserts a single file record into the database
* @param {Object} file - An object containing description, filename, mimetype, src, and username
* @returns {Object} The inserted file record
*/
insertFile ({ description, filename, mimetype, src, username }) {
const result = this.db
.prepare(`
INSERT INTO files (description, filename, mimetype, src, username)
VALUES (@description, @filename, @mimetype, @src, @username)
RETURNING *
`)
.run({ description, filename, mimetype, src, username })
const dbFile = this.getFileById(result.lastInsertRowid)
return dbFile
}

/**
* Inserts multiple file records into the database in a single transaction.
* @param {Array} data - An array of objects (filename, descriptionm mimetype, src, and username) to be inserted into the database.
*/
bulkInsertFiles (data) {
const insert = this.db.prepare(`
INSERT INTO files
(description, filename, mimetype, src, username)
VALUES
(@description, @filename, @mimetype, @src, @username)
`)

const insertMany = this.db.transaction((cats) => {
for (const cat of cats) {
insert.run({
description: cat.description,
filename: cat.filename,
mimetype: cat.mimetype,
src: cat.src,
username: cat.username,
})
}
})
insertMany(data)
}

/**
* Retrieves a file record from the database by its ID.
* @param {number} id - The ID of the file record.
* @returns {Object} The file record matching the provided ID.
*/
getFileById (id) {
return this.db.prepare(`
SELECT * FROM files
WHERE id = :id
`).get({ id })
}

/**
* Retrieves the count of files associated with a specific user.
* @param {string} username - The username of the user.
* @returns {number} The count of files uploaded by the user.
*/
getUserFilesCount (username) {
return this.db.prepare(`
SELECT COUNT(*) AS count FROM files
WHERE username = :username
`).get({ username }).count
}

/**
* Retrieves paginated files associated with a specific user.
* @param {string} username - The username of the user.
* @param {number} size - The maximum number of files to retrieve.
* @param {number} page - The number of files to skip before starting to return records.
* @returns {object} An object of file records uploaded by the user and total.
*/
getUserFiles ({ username, page, size }) {
page = page || 1
size = size || 30
const offset = (page -1) * size
return this.db.prepare(`
SELECT * FROM files
WHERE username = :username
LIMIT :limit OFFSET :offset
`).all({ username, limit: size, offset })
}

/**
* Retrieves the count of all files.
* @returns {number} The count of all files.
*/
getTotal () {
return this.db.prepare(`
SELECT COUNT(*) AS count FROM files
`).get().count
}

/**
* Retrieves all files stored in the database.
* @param {number} size - The maximum number of files to retrieve.
* @param {number} page - The number of files to skip before starting to return records.s
* @returns {Array} An array of all file records.
*/
getAllFiles ({ page=1, size=30 }) {
page = page || 1
size = size || 30
const offset = (page - 1) * size
return this.db.prepare(`SELECT * FROM files LIMIT :limit OFFSET :offset`).all({ limit: size, offset })
}

/**
* Deletes a file record from the database by its ID.
* @param {number} id - The ID of the file record to be deleted.
* @returns {boolean} A boolean, return true if deletion was successful.
*/
deleteFileById (id) {
const deleteStmt = this.db.prepare(`
DELETE FROM files
WHERE id = :id
`)
return deleteStmt.run({ id }).changes > 0
}

/**
* Retrieves the original file record matching the provided criteria.
* @param {Object} criteria - An object match the containing (username, filename, and base64) for finding the original file.
* @returns {Object|null} The original file record matching the provided criteria or null if not found.
*/
getOriginalFile ({ username, filename, base64 }) {
return this.db.prepare(`
SELECT id, filename, src
FROM files
WHERE
filename == :filename
AND src == :src
AND username == :username
`)
.get({
filename,
username,
src: base64,
})
}

/**
* Retrieves all files similar to the provided criteria. Used to get the list of duplicate document
* @param {Object} criteria - An object containing the criteria (username, filename, fileExt, and base64) for finding similar files.
* @returns {Array} An object with records similar to the provided criteria and total matching in DB.
*/
findAllLike ({ username, filename, fileExt, base64, size=30, page=1 }) {
page = page || 1
size = size || 30
const offset = (page - 1) * size
return this.db.prepare(`
SELECT id, filename, src
FROM files
WHERE
filename == :filename OR filename LIKE :similarFilename
AND src == :src
AND username == :username
ORDER BY filename ASC
LIMIT :limit OFFSET :offset
`)
.all({
filename: `${filename}.${fileExt}`,
similarFilename: `${filename}(%.${fileExt}`,
src: base64,
username,
offset,
limit: size,
})
}
}

module.exports = FileRepository
Binary file modified src/db/seed.db
Binary file not shown.
7 changes: 7 additions & 0 deletions src/server/index.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
const path = require('path')
const os = require('os')
const express = require('express')
const appModulePath = require('app-module-path')

Expand All @@ -7,6 +8,12 @@ appModulePath.addPath(path.join(__dirname, '..', '..', 'src'))
const routes = require('./routes')
const app = express()

// Middleware to attach username to request object
app.use((req, res, next) => {
req.username = os.userInfo().username
next()
})

app.use(express.static('dist'))
app.use(express.json({
inflate: true,
Expand Down
Loading