Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infra for subql explorers #354

Merged
merged 8 commits into from
Oct 31, 2024
Merged

Infra for subql explorers #354

merged 8 commits into from
Oct 31, 2024

Conversation

DaMandal0rian
Copy link
Contributor

@DaMandal0rian DaMandal0rian commented Oct 24, 2024

PR Type

enhancement, configuration changes


Description

  • Transitioned infrastructure from Squid to SubQL, updating all relevant configurations and resources.
  • Added provisioning scripts for setting up and deploying SubQL nodes, including Docker and Nginx installations.
  • Configured DNS records for SubQL nodes using Cloudflare.
  • Defined output variables for SubQL node details and DNS records.
  • Updated provider configurations for AWS and Cloudflare.

Changes walkthrough 📝

Relevant files
Configuration changes
17 files
instances.tf
Update AWS instance configurations for SubQL nodes             

templates/terraform/subql/base/instances.tf

  • Renamed resources from squid to subql.
  • Updated security group and gateway references.
  • Modified installation commands and removed monitoring setup.
  • +44/-52 
    bootstrap_subql_provisioner.tf
    Add provisioning for SubQL node setup                                       

    templates/terraform/subql/base/bootstrap_subql_provisioner.tf

  • Added provisioning for blue and green SubQL nodes.
  • Configured SSH connections and file transfers.
  • Set up directories and copied configuration files.
  • +285/-0 
    main.tf
    Switch from Squid to SubQL module in Taurus                           

    explorer/terraform/aws/taurus/main.tf

  • Replaced squids module with subql.
  • Updated node configuration variables.
  • Removed unused node configurations.
  • +12/-60 
    network.tf
    Rename network resources for SubQL infrastructure               

    templates/terraform/subql/base/network.tf

  • Renamed VPC and related resources from squid to subql.
  • Updated tags and gateway references.
  • +18/-18 
    outputs.tf
    Define output variables for SubQL infrastructure                 

    templates/terraform/subql/base/outputs.tf

  • Added output variables for SubQL node details.
  • Included DNS records in outputs.
  • +120/-0 
    variables.tf
    Update variable definitions for SubQL nodes                           

    templates/terraform/subql/base/variables.tf

  • Renamed configuration variables from squid to subql.
  • Updated descriptions for clarity.
  • +10/-19 
    dns.tf
    Add DNS configuration for SubQL nodes                                       

    templates/terraform/subql/base/dns.tf

  • Added Cloudflare DNS records for SubQL nodes.
  • Configured DNS for blue and green environments.
  • +57/-0   
    outputs.tf
    Define outputs for SubQL node IPs and DNS                               

    explorer/terraform/aws/taurus/outputs.tf

  • Added outputs for SubQL node IP addresses.
  • Included DNS records output.
  • +24/-0   
    provider.tf
    Configure providers for SubQL infrastructure                         

    templates/terraform/subql/base/provider.tf

  • Added provider configurations for AWS and Cloudflare.
  • Set required provider versions.
  • +30/-1   
    ami.tf
    Define AMI data source for SubQL nodes                                     

    templates/terraform/subql/base/ami.tf

  • Defined data source for Ubuntu AMI.
  • Filtered AMI based on architecture and virtualization type.
  • +21/-1   
    backend.tf
    Update Terraform backend configuration for SubQL                 

    explorer/terraform/aws/taurus/backend.tf

    • Updated workspace name to subql-taurus-aws.
    +1/-1     
    install_nginx.sh
    Add Nginx installation script for SubQL                                   

    templates/terraform/subql/base/scripts/install_nginx.sh

  • Added script to install and configure Nginx.
  • Configured SSL and security headers.
  • +113/-1 
    install_subql_stack.sh
    Add script to install and run SubQL stack                               

    templates/terraform/subql/base/scripts/install_subql_stack.sh

  • Added script to install SubQL stack.
  • Utilized tmux for running services.
  • +43/-0   
    install_docker.sh
    Add Docker installation script for SubQL                                 

    templates/terraform/subql/base/scripts/install_docker.sh

  • Added script to install Docker and Docker Compose.
  • Configured system settings for Docker.
  • +26/-1   
    cors-settings.conf
    Add CORS configuration for Nginx                                                 

    templates/terraform/subql/base/config/cors-settings.conf

  • Configured CORS settings for Nginx.
  • Allowed various HTTP methods and headers.
  • +42/-1   
    postgresql.conf
    Add PostgreSQL configuration for SubQL                                     

    templates/terraform/subql/base/config/postgresql.conf

  • Added PostgreSQL configuration file.
  • Set connection, memory, and WAL settings.
  • +53/-1   
    nginx-subql.conf
    Configure Nginx for SubQL server                                                 

    templates/terraform/subql/base/config/nginx-subql.conf

  • Configured Nginx server for SubQL.
  • Set proxy and security settings.
  • +11/-11 

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    Copy link

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Configuration Consistency
    Ensure that the configuration parameters and resource allocations are consistent with the expected load and performance requirements. This includes instance types, disk sizes, and security settings.

    Script Robustness
    The script should handle potential errors during the execution of commands, such as failed downloads or installations, to prevent partial configurations.

    Database Optimization
    Review the PostgreSQL configuration settings for optimal performance based on the expected database size and query load.

    Copy link

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Best practice
    Use boolean values for boolean attributes to align with data type expectations

    Ensure consistent attribute naming by using 'map_public_ip_on_launch' as a boolean
    instead of a string to align with Terraform's expected data types.

    templates/terraform/subql/base/network.tf [18]

    -map_public_ip_on_launch = "true"
    +map_public_ip_on_launch = true
    Suggestion importance[1-10]: 9

    Why: This suggestion improves the code by aligning with Terraform's expected data types, enhancing clarity and preventing potential type-related issues.

    9
    Possible issue
    Remove duplicate tag keys to ensure correct resource tagging

    Remove the duplicate tag key 'Name' to avoid conflicts and ensure the VPC resource
    is correctly tagged.

    templates/terraform/subql/base/network.tf [6-9]

     tags = {
       Name = "${var.network_name}-subql-vpc"
    -  name = "${var.network_name}-subql-vpc"
     }
    Suggestion importance[1-10]: 8

    Why: The suggestion addresses a potential issue with duplicate tag keys, which can lead to conflicts and incorrect tagging. Removing the duplicate ensures clarity and correctness in resource tagging.

    8

    @DaMandal0rian
    Copy link
    Contributor Author

    DaMandal0rian commented Oct 24, 2024

    @marc-aurele-besner i have removed the old infra for squids, the last commit is the only one that needs to be checked. 🙏🏽

    @DaMandal0rian DaMandal0rian force-pushed the feat/subql-explorers branch 2 times, most recently from 18f018e to f324bcb Compare October 24, 2024 13:46
    Copy link

    @marc-aurele-besner marc-aurele-besner left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    LGTM

    @DaMandal0rian DaMandal0rian merged commit a8bfe12 into main Oct 31, 2024
    1 check passed
    @DaMandal0rian DaMandal0rian deleted the feat/subql-explorers branch October 31, 2024 16:52
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants