Skip to content

Latest commit

 

History

History
158 lines (131 loc) · 7.41 KB

infrastructure.md

File metadata and controls

158 lines (131 loc) · 7.41 KB
Raw

Project Infrastructure Documentation

Overview

This documentation provides an overview of the infrastructure used for hosting a PostGIS database, GeoServer, and web services. The setup includes components hosted on AcuGIS and AWS, allowing for efficient geospatial data processing and delivery.

Components

  1. AcuGIS Hosting:

    • PostGIS Database: Stores geospatial data.
    • GeoServer: Provides OGC-compliant web services (WMS, WFS) for geospatial data served from PostGIS.

  2. AWS Services:

    • AWS Lambda: Runs serverless functions that interact with the PostGIS database via a REST API.
    • API Gateway: Serves as the entry point for the REST API, which is connected to the Lambda functions.
    • S3 Bucket: Used to store and serve large files, including exported data from the API.

Architecture Diagram

                +---------------------+
                |                     |
                |   AWS S3 Bucket     |
                | (Stores large files)|
                |                     |
                +---------------------+
                           ^
                           |
                           |
+-------------+  +-------------------+  +-----------------------+
|             |  |                   |  |                       |
|  API Gateway|-->| AWS Lambda       |-->| AcuGIS PostGIS        |
| (Public API)|  | (Processes requests|  | (Geospatial Database) |
|             |  | and queries PostGIS|  |                       |
+-------------+  +-------------------+  +-----------------------+
                                             |
                                             |
                                             v
                                     +-----------------+
                                     |                 |
                                     |   GeoServer     |
                                     | (Serves WMS/WFS |
                                     |    from PostGIS)|
                                     +-----------------+

## **Details of Each Component**

### **1. AcuGIS Hosting**

#### **1.1 PostGIS Database**
- **Description**: The PostGIS database is hosted on AcuGIS and is used to store and manage geospatial data. This database is directly connected to GeoServer for data visualization and query.
- **Connection Details**:
  - **Host**: `<your_acugis_host>`
  - **Port**: `5432`
  - **Database Name**: `<your_database_name>`
  - **Username**: `<your_username>`
  - **Password**: `<your_password>`
- **Usage**: 
  - Store and retrieve geospatial data.
  - Execute spatial queries using SQL via connected applications or directly through AcuGIS.

#### **1.2 GeoServer**
- **Description**: GeoServer is used to provide web services that interact with the geospatial data stored in PostGIS.
- **URL**: `<your_geoserver_url>`
- **Connected Datastore**: 
  - **Type**: PostGIS
  - **Workspace**: `<your_workspace>`
  - **Datastore Name**: `<your_datastore_name>`
- **Services Provided**:
  - **WMS (Web Map Service)**: For serving map images generated from your geospatial data.
  - **WFS (Web Feature Service)**: For accessing the raw geospatial data in a structured format.

### **2. AWS Infrastructure**

#### **2.1 AWS Lambda**
- **Description**: AWS Lambda functions are used to process API requests, connect to the PostGIS database, and perform operations like data retrieval and processing.
- **Function Names**: 
  - `<lambda_function_1>`: Connects to PostGIS, executes queries, and returns results.
  - `<lambda_function_2>`: Handles file exports and saves to S3.
- **Environment Variables**:
  - `DB_HOST`: `<your_acugis_postgis_host>`
  - `DB_PORT`: `5432`
  - `DB_NAME`: `<your_database_name>`
  - `DB_USER`: `<your_username>`
  - `DB_PASSWORD`: `<your_password>`
  - `S3_BUCKET_NAME`: `<your_s3_bucket_name>`
- **Trigger**: Invoked via API Gateway when an HTTP request is made.

#### **2.2 API Gateway**
- **Description**: API Gateway serves as the REST API endpoint for accessing the Lambda functions. It handles HTTP requests and forwards them to the appropriate Lambda function.
- **API Endpoint**: `https://<your_api_id>.execute-api.<region>.amazonaws.com/<stage>/`
- **Resources and Methods**:
  - **GET /query**: Triggers the Lambda function to query the PostGIS database.
  - **POST /export**: Triggers the Lambda function to process and save data to S3.
- **Security**:
  - **API Key**: (if applicable) Required for access.
  - **CORS**: Configured to allow cross-origin requests from specific domains.

#### **2.3 S3 Bucket**
- **Description**: The S3 bucket is used to store large files generated by the Lambda functions, such as exported datasets.
- **Bucket Name**: `<your_s3_bucket_name>`
- **Permissions**:
  - **Public Access**: (if applicable) Files can be publicly accessible via a URL.
  - **IAM Role**: Ensure that the Lambda function has the necessary permissions to write to the S3 bucket.
- **File Storage**:
  - **Folder Structure**: Organize files into folders (e.g., `/exports/`) for easier access.
  - **Access URL**: `https://<your_s3_bucket_name>.s3.amazonaws.com/<file_path>`

## **How to Deploy Changes**

### **1. GeoServer**
- **Data Update**: 
  - Connect to the PostGIS database and update data as necessary.
  - Reload layers in GeoServer if schema changes are made.
- **Service Management**: 
  - Access the GeoServer admin panel to manage workspaces, layers, and services.

### **2. AWS Lambda**
- **Code Deployment**:
  - Update the Lambda function code using the AWS Management Console, AWS CLI, or a CI/CD pipeline.
- **Environment Variables**:
  - Update environment variables if database credentials or S3 bucket details change.

### **3. API Gateway**
- **Deploy API Changes**:
  - After modifying resources or methods, deploy the API to the required stage (e.g., `dev`, `prod`).

### **4. S3 Bucket**
- **File Management**:
  - Upload or delete files directly via the AWS Management Console or programmatically via the AWS SDK.
  - Use lifecycle rules to manage storage (e.g., automatically delete old files).

## **Monitoring and Logging**

### **1. AWS CloudWatch**
- **Lambda Logs**: Monitor Lambda function executions and error logs.
- **API Gateway Logs**: Track API request metrics and errors.
- **S3 Access Logs**: (if enabled) Monitor access to files stored in the S3 bucket.

### **2. GeoServer Logs**
- **Access Logs**: Monitor requests made to GeoServer services.
- **Error Logs**: Track any issues with layer rendering or service availability.

## **Security Considerations**

- **Environment Variables**: Ensure sensitive data like database credentials are encrypted.
- **IAM Policies**: Grant the least privilege necessary for AWS Lambda and other AWS services.
- **GeoServer Access**: Restrict access to GeoServer admin interfaces using IP whitelisting or authentication.

## **Backup and Recovery**

- **PostGIS**: Set up regular backups of the PostGIS database using AcuGIS backup services or custom scripts.
- **S3 Bucket**: Enable versioning in S3 to recover deleted or overwritten files.
- **Lambda Code**: Keep your Lambda function code in version control (e.g., Git) for easy rollback if necessary.

## **Support and Maintenance**

- **AcuGIS Support**: Contact AcuGIS for issues related to PostGIS and GeoServer hosting.
- **AWS Support**: Utilize AWS Support for issues related to Lambda, API Gateway, and S3.
- **Internal Contacts**: List any internal contacts or team members responsible for different parts of the infrastructure.