Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tiger team feature #1457

Merged
merged 146 commits into from
Dec 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
6ec3f85
WIP setting up docker compose
croyfish Sep 25, 2024
2b6ec21
update yarn lockfile
croyfish Sep 26, 2024
d900d49
Merge branch 'qa' into TGR-112-upgrade-investigations
croyfish Sep 26, 2024
3d58d32
more WIP changes
croyfish Sep 27, 2024
d662acc
Merge branch 'qa' into TGR-112-upgrade-investigations
croyfish Sep 27, 2024
ee07608
add example env file
croyfish Sep 27, 2024
090e3bc
Merge branch 'tiger-team-feature' into TGR-112-upgrade-investigations
croyfish Sep 27, 2024
a2b05b0
WIP commit
croyfish Sep 30, 2024
dcc1695
update service names locally
croyfish Sep 30, 2024
f0540ff
Merge branch 'TGR-112-upgrade-investigations' into TGR-112-upgrade-de…
croyfish Oct 1, 2024
e5ba209
add rpsec to gemfiles
croyfish Oct 1, 2024
65490df
change exists to exist
croyfish Oct 1, 2024
ca4a68b
change local to development
croyfish Oct 1, 2024
50474da
change a couple more files
croyfish Oct 1, 2024
24728ad
more changes to make stuff work locally
croyfish Oct 2, 2024
82e459a
make puma start in deployed environments as well
croyfish Oct 2, 2024
7bc2d7d
remove comments
croyfish Oct 2, 2024
59a6fc6
remove another comment
croyfish Oct 2, 2024
e5673e6
remove another coment
croyfish Oct 2, 2024
e3526b3
Merge pull request #1436 from NYPL/TGR-112-upgrade-investigations
croyfish Oct 2, 2024
b356df0
Merge branch 'qa' into tiger-team-feature
croyfish Oct 2, 2024
052bcb5
Merge branch 'tiger-team-feature' into TGR-118-convert-local-to-devel…
croyfish Oct 2, 2024
9c74ad0
cleaning up
croyfish Oct 2, 2024
3867e07
update db service names
croyfish Oct 2, 2024
0d65554
Merge branch 'TGR-118-convert-local-to-development' into TGR-117-add-…
croyfish Oct 2, 2024
644631f
set up and begin
croyfish Oct 2, 2024
7582d0d
add books specs
croyfish Oct 3, 2024
1b0e402
add home specs
croyfish Oct 3, 2024
a4c5d46
got some hold specs going
croyfish Oct 3, 2024
66463d7
good progress on fixing minitests
croyfish Oct 4, 2024
377eca6
more test fixes
croyfish Oct 4, 2024
c6d036b
more test fixes
croyfish Oct 4, 2024
30ed457
down to 30
croyfish Oct 4, 2024
d0496e8
fix es tests
croyfish Oct 4, 2024
e870912
Update settings_controller.rb
croyfish Oct 7, 2024
c0bab4b
push uncommitted changes
croyfish Oct 7, 2024
22bcf16
Merge branch 'qa' into TGR-117-add-rspec-coverage
croyfish Oct 7, 2024
52c60be
Merge branch 'TGR-117-add-rspec-coverage' of https://github.com/NYPL/…
croyfish Oct 7, 2024
0d50069
fix es mock
croyfish Oct 7, 2024
01982bb
fix news letter test
croyfish Oct 7, 2024
ba26766
fix hold change test
croyfish Oct 7, 2024
a350d2e
fix a news letter test
croyfish Oct 8, 2024
412ac56
fix routes tests
croyfish Oct 8, 2024
ce27af7
fix some holds controller tests
croyfish Oct 8, 2024
d2f9133
fix other holds controller tests
croyfish Oct 8, 2024
1b15eb1
fix datetime deserialization
croyfish Oct 9, 2024
db3df9d
fix response code in test
croyfish Oct 9, 2024
96a38c4
fix user flow tests
croyfish Oct 9, 2024
fdf3ee2
fix user tests
croyfish Oct 9, 2024
9946639
all tests passing before cleanup
croyfish Oct 9, 2024
10b8d60
commit one more
croyfish Oct 9, 2024
11bbb7f
Merge branch 'qa' into TGR-117-add-rspec-coverage
croyfish Oct 10, 2024
267cfa1
Merge branch 'qa' into TGR-118-convert-local-to-development
croyfish Oct 10, 2024
266f26d
Merge branch 'qa' into tiger-team-feature
croyfish Oct 10, 2024
cf24152
remove debug statements
croyfish Oct 10, 2024
65d9b86
Merge branch 'tiger-team-feature' into TGR-118-convert-local-to-devel…
croyfish Oct 10, 2024
b2a74c1
Merge branch 'TGR-118-convert-local-to-development' into TGR-117-add-…
croyfish Oct 10, 2024
01d8f20
Update settings_controller.rb
croyfish Oct 11, 2024
53a51a3
Merge pull request #1442 from NYPL/TGR-117-add-rspec-coverage
croyfish Oct 11, 2024
af80979
Merge branch 'qa' into tiger-team-feature
croyfish Oct 11, 2024
b9ff9b0
Merge branch 'tiger-team-feature' into TGR-118-convert-local-to-devel…
croyfish Oct 11, 2024
932598d
Merge branch 'TGR-118-convert-local-to-development' of https://github…
croyfish Oct 11, 2024
e4ec092
pin the gem versions
croyfish Oct 16, 2024
8b1026a
remove debug statement
croyfish Oct 16, 2024
7e3af3c
convert log format
croyfish Oct 17, 2024
ace771b
simplify test env
croyfish Oct 17, 2024
89c1f5b
Merge pull request #1450 from NYPL/TGR-116-convert-logstash-event-to-…
croyfish Oct 18, 2024
b0a8f1e
make es upgrade possible
croyfish Oct 23, 2024
09d3cd4
add some action cable config
croyfish Oct 23, 2024
a67231d
convert an overlooked local
croyfish Oct 23, 2024
dfd02b0
Merge branch 'TGR-118-convert-local-to-development' into TGR-124-upda…
croyfish Oct 23, 2024
f006483
Merge branch 'qa' into TGR-118-convert-local-to-development
croyfish Oct 23, 2024
000ad2e
Merge branch 'TGR-118-convert-local-to-development' into TGR-124-upda…
croyfish Oct 23, 2024
7adaef0
clean some config
croyfish Oct 23, 2024
2b182ae
Merge branch 'TGR-118-convert-local-to-development' into TGR-124-upda…
croyfish Oct 23, 2024
6d933d5
update dependencies
croyfish Oct 23, 2024
63a7678
fix broken test
croyfish Oct 23, 2024
2c688d6
Merge branch 'TGR-118-convert-local-to-development' into TGR-124-upda…
croyfish Oct 23, 2024
d6e77a2
Merge pull request #1440 from NYPL/TGR-118-convert-local-to-development
croyfish Oct 24, 2024
3d42028
make es upgrade possible
croyfish Oct 23, 2024
deb6e69
update docs
croyfish Oct 24, 2024
26d08b8
Merge pull request #1454 from NYPL/TGR-128-documentation-updates
croyfish Oct 24, 2024
d4a868f
Merge branch 'tiger-team-feature' into TGR-124-update-es-to-7.10
croyfish Oct 24, 2024
a4dd63f
first stab at gha builds
croyfish Oct 24, 2024
5623126
first stab at gh action builds
croyfish Oct 24, 2024
c1c060a
move workflow to correct location
croyfish Oct 24, 2024
b410069
also build production
croyfish Oct 24, 2024
63d9afb
remove travis deploys
croyfish Oct 24, 2024
78be15f
remove feature branch
croyfish Oct 25, 2024
1caf51a
Merge pull request #1455 from NYPL/TGR-127-implement-github-actions
croyfish Oct 25, 2024
27d2674
first stab
croyfish Oct 28, 2024
8634b26
add some docker
croyfish Oct 28, 2024
7f3d700
set up database
croyfish Oct 28, 2024
2f94128
chain commands correctly
croyfish Oct 28, 2024
eef226e
fix incorrect command
croyfish Oct 28, 2024
a16cd59
load schema not migrate
croyfish Oct 28, 2024
a6151e4
fix the indentation
croyfish Oct 28, 2024
7a6c026
add PRs to ci testing
croyfish Oct 28, 2024
52401df
Merge pull request #1456 from NYPL/TGR-130-unit-tests-run-automatically
croyfish Oct 28, 2024
44a9a1e
make pipelines deploy
croyfish Oct 28, 2024
a191cc0
remove epic branch from builds
croyfish Oct 28, 2024
1a050a7
clean up readme
croyfish Oct 28, 2024
86d7f89
clean up holds
croyfish Oct 28, 2024
11651e6
clean up lograge
croyfish Oct 28, 2024
9ee07b2
clean up user fixtures
croyfish Oct 28, 2024
35efda4
remove unwritten test
croyfish Oct 28, 2024
3e2b231
clean up hold test
croyfish Oct 28, 2024
87d7231
remove unused test
croyfish Oct 28, 2024
40d3634
clean up teacher set test
croyfish Oct 28, 2024
e73b973
Merge branch 'TGR-130-unit-tests-run-automatically' into tiger-team-f…
croyfish Oct 28, 2024
154ae22
update gh actions pipelines
croyfish Oct 30, 2024
faafdea
update build qa pipeline
croyfish Nov 1, 2024
d8bb252
add same changes to prod pipeline
croyfish Nov 1, 2024
7d9bc5e
merge latest qa
croyfish Nov 13, 2024
c9645fb
WIP
croyfish Nov 15, 2024
f48f9ae
try new approach
croyfish Nov 15, 2024
8a56ca5
update the schema
croyfish Nov 18, 2024
2964462
Merge branch 'tiger-team-feature' into TGR-141-set-up-db-automatically
croyfish Nov 18, 2024
80836b6
Merge pull request #1461 from NYPL/TGR-141-set-up-db-automatically
croyfish Nov 18, 2024
b85ad06
push the branch to qa
croyfish Nov 19, 2024
18ba453
actually deploy
croyfish Nov 19, 2024
765aefb
remove deploy condition
croyfish Nov 19, 2024
5df2829
downgrade active model serializers
croyfish Nov 20, 2024
b92a251
try reverting to 0.8
croyfish Nov 20, 2024
ccaf643
use exact ams version
croyfish Nov 20, 2024
ff5c647
Merge branch 'tiger-team-feature' into TGR-124-update-es-to-7.10
croyfish Nov 20, 2024
f9b88c0
update readme and add task
croyfish Nov 21, 2024
124859d
fix some formatting
croyfish Nov 21, 2024
442de69
Merge branch 'TGR-update-readme-and-setup' of https://github.com/NYPL…
deepikagonuguntla Nov 21, 2024
aa82b74
fix a migration for compatibility
croyfish Nov 21, 2024
a5cae73
Merge pull request #1463 from NYPL/TGR-fix-migrations
croyfish Nov 21, 2024
724d840
Merge branch 'tiger-team-feature' of https://github.com/NYPL/MyLibrar…
deepikagonuguntla Nov 21, 2024
50896d0
updated elastic search mappings
deepikagonuguntla Nov 21, 2024
5dd82d3
Update elastic search mappings
deepikagonuguntla Nov 21, 2024
f029dac
added mappings
deepikagonuguntla Nov 21, 2024
b2ab3e6
Merge pull request #1464 from NYPL/elasticsearch-6.8_to_7.10
croyfish Nov 22, 2024
71050ad
use port 443 for es client
croyfish Nov 22, 2024
8ba833b
make es work in all environments
croyfish Nov 25, 2024
e1b5d84
try fix to allow rake tasks to run
croyfish Nov 25, 2024
17fecc7
improve local es setup
croyfish Nov 25, 2024
af22384
updated mappings
deepikagonuguntla Nov 25, 2024
e476b6e
updated mappings
deepikagonuguntla Nov 25, 2024
743d4f9
Merge pull request #1466 from NYPL/elasticsearch-6.8_to_7.10
gonuguntla Nov 25, 2024
af98500
Fixed allowed user emails issue and fixed active checkbox alignment …
deepikagonuguntla Nov 27, 2024
d5027ed
Merge pull request #1467 from NYPL/elasticsearch-6.8_to_7.10
gonuguntla Nov 27, 2024
f782145
re-update ci pipeline
croyfish Dec 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env-example
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# .env
AWS_ACCESS_KEY_ID=<your-public-nypl-sandbox-cli-key>
AWS_SECRET_ACCESS_KEY=<your-private-nypl-sandbox-cli-key>
AWS_REGION=us-east-1
RAILS_ENV=development
46 changes: 46 additions & 0 deletions .github/workflows/build-production.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Deploy Production
on:
workflow_run:
workflows: ["CI"]
types:
- completed

permissions:
id-token: write
contents: read

jobs:
deploy_production:
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'production' }}
name: Deploy Production
runs-on: ubuntu-latest
environment: production
steps:
- name: checkout
uses: actions/checkout@v2

- name: Configure AWS credentials for Production environment account
uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: arn:aws:iam::946183545209:role/GithubActionsDeployerRole
aws-region: us-east-1

- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1

- name: Build, tag, and push image to Amazon ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: my-library-nyc-app
IMAGE_TAG: ${{ github.sha }}
run: |
DOCKER_BUILDKIT=1 docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
docker tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:production-latest
docker push $ECR_REGISTRY/$ECR_REPOSITORY:production-latest

- name: Force ECS Update
run: |
aws ecs update-service --cluster mylibrarynycapp-production --service mylibrarynycapp-production --force-new-deployment
aws ecs update-service --cluster mylibrarynycapp-production --service mylibrarynycapp-production-delayed-job --force-new-deployment
46 changes: 46 additions & 0 deletions .github/workflows/build-qa.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Deploy QA
on:
workflow_run:
workflows: ["CI"]
types:
- completed

permissions:
id-token: write
contents: read

jobs:
deploy_qa:
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'tiger-team-feature' }}
name: Deploy QA
runs-on: ubuntu-latest
environment: qa
steps:
- name: checkout
uses: actions/checkout@v2

- name: Configure AWS credentials for QA environment account
uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: arn:aws:iam::946183545209:role/GithubActionsDeployerRole
aws-region: us-east-1

- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1

- name: Build, tag, and push image to Amazon ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: my-library-nyc-app
IMAGE_TAG: ${{ github.sha }}
run: |
DOCKER_BUILDKIT=1 docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
docker tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:qa-latest
docker push $ECR_REGISTRY/$ECR_REPOSITORY:qa-latest

- name: Force ECS Update
run: |
aws ecs update-service --cluster mylibrarynycapp-qa --service mylibrarynycapp-qa --force-new-deployment
aws ecs update-service --cluster mylibrarynycapp-qa --service mylibrarynycapp-qa-delayed-job --force-new-deployment
35 changes: 35 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: CI

on:
push:
branches:
- qa
- production
pull_request:
branches:
- qa
- production

jobs:
build_and_test:
runs-on: ubuntu-latest
steps:
- name: check out code
uses: actions/checkout@v2

- name: set up docker
uses: docker/setup-buildx-action@v2

- name: install docker compose
run: |
sudo apt-get update
sudo apt-get install -y docker-compose

- name: build application
run: |
docker-compose build

- name: Run tests
run: |
docker-compose up -d
docker-compose run webapp sh -c 'RAILS_ENV=test bundle exec rake db:create db:schema:load && RAILS_ENV=test bundle exec rails test'
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@
# Ignore files in the data/private subdirectory
/data/private

# Ignore postgresql data from the local environment
/data/postgres-my-library-nyc-app

# Elastic Beanstalk Files
.elasticbeanstalk/*
!.elasticbeanstalk/*.cfg.yml
Expand All @@ -46,3 +49,7 @@ yarn-debug.log*

/app/assets/builds/*
!/app/assets/builds/.keep

# Ignore VIM swapfiles
*.swo
*.swp
2 changes: 2 additions & 0 deletions .rspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
--require spec_helper
--format documentation
79 changes: 0 additions & 79 deletions .travis.yml

This file was deleted.

79 changes: 36 additions & 43 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,56 +1,49 @@
# syntax = docker/dockerfile:1.3
FROM ruby:2.7.4 AS builder

# set env vars
ENV APP_HOME /home/app/MyLibraryNYCApp
ENV AWS_DEFAULT_REGION=us-east-1

ARG RAILS_ENV
ENV RAILS_ENV=${RAILS_ENV}

# Set base image and working directory
FROM ruby:3.3

# Install necessary packages, including curl and PostgreSQL client
RUN apt-get update -qq && apt-get install -y \
curl \
postgresql-client \
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
&& apt-get install -y nodejs \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Install Yarn globally
RUN npm install -g yarn

# Set environment variables
ENV RAILS_ENV=development
ENV APP_HOME=/app
RUN mkdir $APP_HOME
WORKDIR $APP_HOME

# install packages
RUN apt-get update -qq \
&& apt-get install -y \
curl \
postgresql-client \
git

RUN curl -sL https://deb.nodesource.com/setup_16.x | bash - \
&& apt-get -y install nodejs \
&& npm install --global yarn

# Install esbuild
# Install esbuild globally
RUN npm install -g esbuild

# set up app files
COPY . $APP_HOME
COPY Gemfile $APP_HOME
COPY Gemfile.lock $APP_HOME
WORKDIR $APP_HOME
# Copy Gemfile and Gemfile.lock first
COPY Gemfile Gemfile.lock $APP_HOME/

## bundle
ENV BUNDLER_VERSION=2.4.22
RUN gem install bundler -v $BUNDLER_VERSION
RUN bundle config --global github.https true \
&& bundle install --jobs 30
# Install bundler and Ruby dependencies
RUN gem install bundler -v 2.5.20
RUN bundle install --jobs 30

COPY package.json $APP_HOME/package.json
COPY package-lock.json $APP_HOME/package-lock.json
# Copy package.json and package-lock.json before running yarn install
COPY package.json $APP_HOME/

# Install JS dependencies
RUN yarn install

# build
# Now copy the rest of the application
COPY . $APP_HOME/

# Precompile assets
RUN yarn build
RUN yarn build:css
RUN --mount=type=secret,id=AWS_ACCESS_KEY_ID \
--mount=type=secret,id=AWS_SECRET_ACCESS_KEY \
AWS_ACCESS_KEY_ID=$(cat /run/secrets/AWS_ACCESS_KEY_ID) \
&& export AWS_ACCESS_KEY_ID \
AWS_SECRET_ACCESS_KEY=$(cat /run/secrets/AWS_SECRET_ACCESS_KEY) \
&& export AWS_SECRET_ACCESS_KEY \
&& bundle exe rails assets:precompile

# Expose the app port
EXPOSE 3000
CMD ["bundle", "exec", "rails", "server", "-p", "3000", "-b", "0.0.0.0"]

# Start the server
CMD ["bash", "-c", "rm -f /app/tmp/pids/server.pid && bundle exec rails server -b 0.0.0.0"]
Loading