Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

production release #288

Merged
merged 140 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
1f276e5
[pre-commit.ci] pre-commit autoupdate
pre-commit-ci[bot] Sep 30, 2024
62ce7ce
feat: added updated at field in task events table
Pradip-p Oct 1, 2024
79e85ce
Merge pull request #252 from hotosm/pre-commit-ci-update-config
nrjadkry Oct 1, 2024
fb5c465
fix: handle exception if project not found in db
Pradip-p Oct 1, 2024
b9a78af
feat(user-profile): pass watch on form props
Oct 1, 2024
d259bce
fix(user-profile)-#209: able to submit if the password and confirm pa…
Oct 1, 2024
9b6f57c
fix(user-profile)-#208: validation issue
Oct 1, 2024
e77a20f
feat: added updated_at field when user request, lock, unlock task
Pradip-p Oct 1, 2024
002ee20
feat: added updated_at in task details endpoint
Pradip-p Oct 1, 2024
9d0766f
fix: get superuser from user table instead of access token
Pradip-p Oct 1, 2024
24167c1
feat: update file and folder structure and rename `UserProfile` to `C…
Oct 1, 2024
74b794d
refac: remove super user from AuthUser Schemas
Pradip-p Oct 1, 2024
ba0752d
Merge pull request #253 from hotosm/feat/updated-datetime
nrjadkry Oct 1, 2024
23dd289
feat: add Breadcrumb component
Oct 1, 2024
aa3dc33
feat: replace user-profile layout component and and add new route cal…
Oct 1, 2024
52a418d
feat: added two new enums such as IMAGE_UPLOADED & IMAGE_PROCESSED
Pradip-p Oct 1, 2024
4045d43
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 1, 2024
d43b2d5
Merge pull request #254 from hotosm/feat/updated-datetime
nrjadkry Oct 1, 2024
e3a423c
fix(task-description): Remove BLOB middleware for processing result d…
Oct 1, 2024
735df24
Merge pull request #256 from hotosm/fix/remove-blob
nrjadkry Oct 1, 2024
4297042
fix: issues reslove on updated date
Pradip-p Oct 2, 2024
e85338e
feat: added change password based on old and new password
Pradip-p Oct 2, 2024
13d8ae8
feat(individual-project): post date with other data on task lock/unlock
Oct 2, 2024
e133895
feat(task-description): show task locked date
Oct 2, 2024
e23749e
Merge pull request #257 from hotosm/feat/change-password
nrjadkry Oct 2, 2024
008276b
Merge pull request #258 from hotosm/fix/remove-blob
nrjadkry Oct 2, 2024
f364c43
feat(update-profile): add BasicDetails component
Oct 2, 2024
39dd3cc
feat(update-profile): add Header component
Oct 2, 2024
ae7fba7
feat(update-profile): add OrganizationDetail component
Oct 2, 2024
6c555ed
feat(update-profile): add Password component
Oct 2, 2024
fb5cc92
feat(update-profile): add OtherDetails component
Oct 2, 2024
99da3b2
feat: add common css class main-content that gives the full screen he…
Oct 2, 2024
31fcf4d
feat(user-profile): add UpdateUserProfile component
Oct 2, 2024
e880d59
feat: remove the route `user-profile` add `complete-profile` from the…
Oct 2, 2024
4d1adf5
feat(user-profile): update basic details
Oct 2, 2024
c692906
refactor: remove user data fetching on projects page
Oct 2, 2024
1eedd82
refactor(user-profile): revalidate user details data on basic details…
Oct 2, 2024
10195fb
feat: store user details on local storage on fetch success
Oct 2, 2024
33ca871
feat(user-profile): create user profile patch service
Oct 2, 2024
22ad197
feat(user-profile): call user details api on userProfile of nav section
Oct 2, 2024
cee0890
feat(user-profile): update organization details
Oct 2, 2024
cd34d0b
feat(user-profile): update other details
Oct 2, 2024
2be552d
feat(user-profile): update password
Oct 2, 2024
86c6a7f
fix: typo
Oct 2, 2024
f686c84
fix: minor style
Oct 2, 2024
e800299
Merge branch 'develop' of github.com:hotosm/drone-tm into feat/user-p…
Oct 2, 2024
d87f2d6
fix: typo
Oct 2, 2024
b41631b
fix(ser-profile): make patch request on profile data change
Oct 2, 2024
cec7555
feat(user-profile): show error message
Oct 2, 2024
a10d813
hot-fix(user-profile): check old & new password before verify the pas…
Pradip-p Oct 2, 2024
66469eb
fix: added message if user do not sent old and new password
Pradip-p Oct 2, 2024
6c0db7f
refac: exception raise on password check
Pradip-p Oct 2, 2024
8c5f546
Merge pull request #259 from hotosm/feat/change-password
Pradip-p Oct 2, 2024
6621b75
[Merge branch 'develop' of github.com:hotosm/drone-tm into feat/user-…
Oct 2, 2024
01e5e50
feat(user-profile): redirect to complete profile page if the user not…
Oct 2, 2024
4f30c97
feat(user-profile): redirect to complete profile on google auth success
Oct 2, 2024
b33ed4d
fix(user-profile): minor UI
Oct 2, 2024
7bd0bc4
fix(user-profile): navigate to complete profile only after the profil…
Oct 2, 2024
9203dd6
fix: typo
Oct 2, 2024
c8dca57
used presigned url for map screenshot in projects endpoint
nrjadkry Oct 4, 2024
4109a1d
Merge pull request #261 from hotosm/optimize_dashboard_api
nrjadkry Oct 4, 2024
5cea65a
feat: added search features in projects read endpoit
Pradip-p Oct 4, 2024
17012de
Merge pull request #263 from hotosm/feat/search-project
nrjadkry Oct 4, 2024
a8b2aba
Merge pull request #260 from hotosm/feat/user-profile
nrjadkry Oct 4, 2024
e43edaa
Merge pull request #266 from hotosm/feat/download-orthophoto
nrjadkry Oct 4, 2024
3ba1c73
Merge pull request #265 from hotosm/feat/orthophoto-download
nrjadkry Oct 4, 2024
81d86de
fix(update-userProfile): update background color
Oct 8, 2024
a589483
feat(dashboard): show user role
Oct 8, 2024
befca95
feat: updated the task as completed after process from s3
Pradip-p Oct 8, 2024
9c7f1e8
Feat: updated pagination on list projects (#264)
Pradip-p Oct 8, 2024
475527e
feat: update task state to reflect the start of image processing
Pradip-p Oct 8, 2024
94679cb
fix: remove comment from the task update function
Pradip-p Oct 8, 2024
d633c45
fix: adjust the state based on processing...
Pradip-p Oct 8, 2024
1f7bb9e
fix: refine the state of task processing
Pradip-p Oct 8, 2024
46e3770
Feat: Updated the task as completed after process from S3 (#270)
Pradip-p Oct 8, 2024
0a59358
feat(projects): Show status on project card
Oct 8, 2024
3740f2f
feat(dashboard): update UI and pass other project details on project …
Oct 8, 2024
83ddfe3
feat: update the status of each projects on projects list endpoint
Pradip-p Oct 8, 2024
647998c
feat(dashboard): show slug on project card
Oct 8, 2024
e1cf543
fix: merge conflict
Oct 8, 2024
28ce0a6
feat: refine the status of tasks
Pradip-p Oct 8, 2024
fc9d2dd
feat(project-details): add new status on legend
Oct 8, 2024
79751fa
feat(project-details): show skeleton on project data fetching
Oct 8, 2024
62937bb
fix: reslove merge conflict project and task schema
Pradip-p Oct 8, 2024
de438ad
feat(project-details): fill different color for completed task and im…
Oct 8, 2024
7849afa
Merge pull request #271 from hotosm/feat/add-project-status
Pradip-p Oct 8, 2024
d316101
feat(project-details): show loader on table
Oct 8, 2024
3ac8981
Merge branch 'develop' of github.com:hotosm/drone-tm into feat/show-s…
Oct 8, 2024
8f55c8b
feat(dashboard): make logs table responsive
Oct 8, 2024
42bb7ef
feat: add Status chip component
Oct 8, 2024
68ab945
feat(project-dashboard): show complete status on project card
Oct 8, 2024
9cce047
feat: fill circle color code from properties on VectorLayerWithCluste…
Oct 8, 2024
bd40cc9
feat(project-description): add different color code as per project st…
Oct 8, 2024
6899671
hotfix: Update project/id to project_id in S3 image processing logic
Pradip-p Oct 9, 2024
31c52fe
Merge pull request #273 from hotosm/feat/update-completed-task
Pradip-p Oct 9, 2024
6f3d1d7
feat: reduce table height
Oct 9, 2024
75f99b8
feat(project-details): image processing status text update
Oct 9, 2024
f1596a8
fix(project-details): available tasks not showing on table
Oct 9, 2024
eaff9ac
fix: added asyncio event loop to call async update_task_state in proc…
Pradip-p Oct 9, 2024
f361b6c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 9, 2024
a26a517
Merge pull request #274 from hotosm/feat/update-completed-task
Pradip-p Oct 9, 2024
20ab8a5
Feat: Show project status and task status on map and project cards (#…
suzit-10 Oct 9, 2024
52e75fc
fix: resolve RuntimeError for missing event loop in AnyIO worker thread
Pradip-p Oct 9, 2024
1d685d9
hot-fix: minor UI issue on chip
Oct 9, 2024
775a2c9
fix: merge conflict
Oct 9, 2024
4db4150
feat: added asgiref package to convert the async to sync function
Pradip-p Oct 9, 2024
30f158e
Merge pull request #275 from hotosm/feat/update-completed-task
Pradip-p Oct 9, 2024
d2a9a6f
feat(project-description): increase opacity of task completed tasks f…
Oct 9, 2024
dfb32d7
Merge pull request #276 from hotosm/feat/show-status
suzit-10 Oct 9, 2024
dc612ee
fix: completed task status count
Pradip-p Oct 9, 2024
b421771
Merge pull request #278 from hotosm/feat/update-completed-task
Pradip-p Oct 9, 2024
f57bdc9
fix: update ongoing_task_count to count tasks in specific states of p…
Pradip-p Oct 9, 2024
8a7ce11
fix: optimize project status calculation logic & update ongoing statu…
Pradip-p Oct 9, 2024
c9f772e
fix(task-description): set height to update takeoff point button section
Oct 14, 2024
aadc06b
feat: added new assets download endpoint
Pradip-p Oct 14, 2024
839f164
fix: remove await from assets download endpoint
Pradip-p Oct 14, 2024
19c22b0
fix(task-description): set height to update takeoff point button sect…
suzit-10 Oct 14, 2024
c7df31d
feat: added users authentication
Pradip-p Oct 14, 2024
9d11b53
feat(project-dashboard): udpdate row per page options
Oct 14, 2024
464dcc3
feat(project-dashboard): update map and project card style
Oct 14, 2024
8738424
Merge branch 'feat/assets-download-url' of github.com:hotosm/drone-tm…
Oct 14, 2024
553f41b
feat(task-description): update assest information api endpoint
Oct 14, 2024
14940ad
feat(individual-project): get assets list
Oct 14, 2024
b064de0
feat(individual-project):fetch all assets and modify project details …
Oct 14, 2024
6a42ca2
Style: update rows per page and set project card column to 3 (#283)
suzit-10 Oct 14, 2024
b6bb97c
feat(individual-project):add loader
Oct 14, 2024
c5dc954
Feat: Updated endpoints(download-assets) to fetch task assets status.…
Pradip-p Oct 14, 2024
f0bbe76
fix(project-details): add `/` on api endpoint
Oct 14, 2024
bdc17fe
fix: merge conflict
Oct 14, 2024
18a114a
Merge pull request #286 from hotosm/feat/update-assets-download-url
suzit-10 Oct 14, 2024
deee39d
feat(project-description): call assets api only on contirbution tab i…
Oct 14, 2024
ed40275
Merge pull request #287 from hotosm/feat/update-assets-download-url
suzit-10 Oct 14, 2024
8ac63ff
fix: merge conflict
Oct 14, 2024
cfc9ae6
fix(backend): reslove merge conflict in pyproject.toml
Pradip-p Oct 14, 2024
def840d
fix(backend): returns values from calculations function
Pradip-p Oct 14, 2024
d959a81
fix(backend): reslove merge conflict in pyproject.toml
Pradip-p Oct 14, 2024
2c1274b
Merge pull request #289 from hotosm/fix/merge-conflict
Pradip-p Oct 14, 2024
4db47da
fix(backend): reslove merge conflict with main branch
Pradip-p Oct 14, 2024
335f409
fix: resolve merge conflict with main
Pradip-p Oct 14, 2024
baaeac1
fix: update the pdm lock file
Pradip-p Oct 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/backend/app/db/database.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@

async def get_db_connection_pool() -> AsyncConnectionPool:
"""Get the connection pool for psycopg."""
return AsyncConnectionPool(conninfo=settings.DTM_DB_URL.unicode_string())
pool = AsyncConnectionPool(conninfo=settings.DTM_DB_URL.unicode_string())
await pool.open() # Explicitly open the pool
return pool


async def get_db(request: Request) -> AsyncGenerator[Connection, None]:
Expand Down
23 changes: 9 additions & 14 deletions src/backend/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,13 @@
from fastapi.middleware.cors import CORSMiddleware
from fastapi.templating import Jinja2Templates
from fastapi.responses import RedirectResponse, JSONResponse

from psycopg_pool import AsyncConnectionPool
from app.config import settings
from app.projects import project_routes
from app.drones import drone_routes
from app.waypoints import waypoint_routes
from app.users import user_routes
from app.tasks import task_routes
from app.db.database import get_db_connection_pool


root = os.path.dirname(os.path.abspath(__file__))
Expand Down Expand Up @@ -105,23 +104,19 @@ def get_application() -> FastAPI:


@asynccontextmanager
async def lifespan(
app: FastAPI,
):
async def lifespan(app: FastAPI):
"""FastAPI startup/shutdown event."""
log.debug("Starting up FastAPI server.")

db_pool = await get_db_connection_pool()
await db_pool.open()
# Create a pooled db connection and make available in app state
# NOTE we can access 'request.app.state.db_pool' in endpoints
app.state.db_pool = db_pool

yield
async with AsyncConnectionPool(
conninfo=settings.DTM_DB_URL.unicode_string()
) as db_pool:
# The pool is now used within the context manager
app.state.db_pool = db_pool
yield # FastAPI will run the application here

# Shutdown events
# Pool will be closed automatically when the context manager exits
log.debug("Shutting down FastAPI server.")
await app.state.db_pool.close()


api = get_application()
Expand Down
26 changes: 24 additions & 2 deletions src/backend/app/projects/project_deps.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,35 @@
from fastapi import Depends, HTTPException, Path, File, UploadFile
from psycopg import Connection
from geojson import FeatureCollection

from psycopg.rows import dict_row
from app.db import database
from app.models.enums import HTTPStatus
from app.projects.project_schemas import DbProject
from app.utils import multipolygon_to_polygon


async def get_tasks_by_project_id(project_id: UUID, db: Connection):
"""Get tasks by project id."""
try:
async with db.cursor(row_factory=dict_row) as cur:
await cur.execute(
"""SELECT id FROM tasks WHERE project_id = %(project_id)s""",
{"project_id": project_id},
)

data = await cur.fetchall()

if data is None:
raise HTTPException(
status_code=HTTPStatus.FORBIDDEN,
detail="No tasks found for this project.",
)
return data

except Exception as e:
raise HTTPException(status_code=500, detail=str(e))


async def get_project_by_id(
project_id: Annotated[
UUID,
Expand All @@ -27,7 +49,7 @@ async def get_project_by_id(
try:
return await DbProject.one(db, project_id)
except KeyError as e:
raise HTTPException(status_code=HTTPStatus.NOT_FOUND) from e
raise HTTPException(status_code=HTTPStatus.FORBIDDEN) from e


async def geojson_upload(
Expand Down
27 changes: 22 additions & 5 deletions src/backend/app/projects/project_routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -388,17 +388,34 @@ async def process_imagery(


@router.get(
"/assets/{project_id}/{task_id}/",
"/assets/{project_id}/",
tags=["Image Processing"],
response_model=project_schemas.AssetsInfo,
)
async def get_assets_info(
user_data: Annotated[AuthUser, Depends(login_required)],
db: Annotated[Connection, Depends(database.get_db)],
project: Annotated[
project_schemas.DbProject, Depends(project_deps.get_project_by_id)
],
task_id: uuid.UUID,
task_id: Optional[uuid.UUID] = None,
):
"""
Endpoint to get the number of images and the URL to download the assets for a given project and task.
Endpoint to get the number of images and the URL to download the assets
for a given project and task. If no task_id is provided, returns info
for all tasks associated with the project.
"""
return project_logic.get_project_info_from_s3(project.id, task_id)
if task_id is None:
# Fetch all tasks associated with the project
tasks = await project_deps.get_tasks_by_project_id(project.id, db)

results = []

for task in tasks:
task_info = project_logic.get_project_info_from_s3(
project.id, task.get("id")
)
results.append(task_info)

return results
else:
return project_logic.get_project_info_from_s3(project.id, task_id)
27 changes: 4 additions & 23 deletions src/backend/app/projects/project_schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
import uuid
from typing import Annotated, Optional, List
from datetime import datetime, date
from app.projects import project_logic
import geojson
from loguru import logger as log
from pydantic import BaseModel, computed_field, Field, model_validator, root_validator
Expand Down Expand Up @@ -147,24 +146,6 @@ class TaskOut(BaseModel):
image_count: Optional[int] = None
assets_url: Optional[str] = None

@model_validator(mode="after")
def set_assets_url(cls, values):
"""Set image_url and image count before rendering the model."""
task_id = values.id
project_id = values.project_id

if task_id and project_id:
data = project_logic.get_project_info_from_s3(project_id, task_id)
if data:
return values.copy(
update={
"assets_url": data.assets_url,
"image_count": data.image_count,
}
)

return values


class DbProject(BaseModel):
"""Project model for extracting from database."""
Expand Down Expand Up @@ -549,12 +530,12 @@ def calculate_status(cls, values):
completed_task_count = values.completed_task_count
total_task_count = values.total_task_count

if ongoing_task_count == 0:
if completed_task_count == 0 and ongoing_task_count == 0:
values.status = "not-started"
elif ongoing_task_count > 0 and ongoing_task_count != completed_task_count:
values.status = "ongoing"
elif ongoing_task_count == total_task_count:
elif completed_task_count == total_task_count:
values.status = "completed"
else:
values.status = "ongoing"

return values

Expand Down
2 changes: 1 addition & 1 deletion src/backend/app/tasks/task_routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ async def get_task_stats(
raw_sql = """
SELECT
COUNT(CASE WHEN te.state = 'REQUEST_FOR_MAPPING' THEN 1 END) AS request_logs,
COUNT(CASE WHEN te.state = 'LOCKED_FOR_MAPPING' THEN 1 END) AS ongoing_tasks,
COUNT(CASE WHEN te.state IN ('LOCKED_FOR_MAPPING', 'REQUEST_FOR_MAPPING', 'IMAGE_UPLOADED', 'UNFLYABLE_TASK') THEN 1 END) AS ongoing_tasks,
COUNT(CASE WHEN te.state = 'IMAGE_PROCESSED' THEN 1 END) AS completed_tasks,
COUNT(CASE WHEN te.state = 'UNFLYABLE_TASK' THEN 1 END) AS unflyable_tasks
FROM (
Expand Down
Loading
Loading