diff --git a/README.md b/README.md index f1c566307..b5e66c43b 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,11 @@ # Hyku, the Hydra-in-a-Box Repository Application -Code: +Code: [![Build Status](https://circleci.com/gh/samvera/hyku.svg?style=svg)](https://circleci.com/gh/samvera/hyku) [![Coverage Status](https://coveralls.io/repos/samvera/hyku/badge.svg?branch=master&service=github)](https://coveralls.io/github/samvera/hyku?branch=master) [![Stories in Ready](https://img.shields.io/waffle/label/samvera/hyku/ready.svg)](https://waffle.io/samvera/hyku) -Docs: +Docs: [![Documentation](http://img.shields.io/badge/DOCUMENTATION-wiki-blue.svg)](https://github.com/samvera/hyku/wiki) [![Contribution Guidelines](http://img.shields.io/badge/CONTRIBUTING-Guidelines-blue.svg)](./CONTRIBUTING.md) [![Apache 2.0 License](http://img.shields.io/badge/APACHE2-license-blue.svg)](./LICENSE) @@ -25,7 +25,7 @@ Jump In: [![Slack Status](http://slack.samvera.org/badge.svg)](http://slack.samv * [Single Tenant Mode](#single-tenancy) * [Switching accounts](#switching-accounts) * [Development dependencies](#development-dependencies) - * [Postgres](#postgres) + * [Postgres](#postgres) * [Importing](#importing) * [enable Bulkrax](#bulkrax) * [from CSV](#from-csv) @@ -43,7 +43,7 @@ Jump In: [![Slack Status](http://slack.samvera.org/badge.svg)](http://slack.samv #### Dory -On OS X or Linux we recommend running [Dory](https://github.com/FreedomBen/dory). It acts as a proxy allowing you to access domains locally such as hyku.docker or tenant.hyku.docker, making multitenant development more straightforward and prevents the need to bind ports locally. You can still run in development via docker with out Dory, but to do so please uncomment the ports section in docker-compose.yml and then find the running application at localhost:3000 +On OS X or Linux we recommend running [Dory](https://github.com/FreedomBen/dory). It acts as a proxy allowing you to access domains locally such as hyku.test or tenant.hyku.test, making multitenant development more straightforward and prevents the need to bind ports locally. Be sure to adjust your ~/.dory.yml file to support the .test tld. You can still run in development via docker with out Dory, but to do so please uncomment the ports section in docker-compose.yml and then find the running application at localhost:3000 ```bash gem install dory @@ -55,7 +55,7 @@ dory up ```bash docker-compose up web workers ``` -This command starts the whole stack in individual containers allowing Rails to be started or stopped independent of the other services. Once that starts (you'll see the line `Passenger core running in multi-application mode.` to indicate a successful boot), you can view your app in a web browser with at either hyku.docker or localhost:3000 (see above). When done `docker-compose stop` shuts down everything. +This command starts the whole stack in individual containers allowing Rails to be started or stopped independent of the other services. Once that starts (you'll see the line `Passenger core running in multi-application mode.` to indicate a successful boot), you can view your app in a web browser with at either hyku.test or localhost:3000 (see above). When done `docker-compose stop` shuts down everything. #### Tests in Docker @@ -109,7 +109,7 @@ The [samvera-vagrant project](https://github.com/samvera-labs/samvera-vagrant) p ### With Kubernetes -Hyku provides a set of [Helm charts](chart/README.md) for deploying to a Kubernetes cluster. Detailed instructions are provided for deploying locally with Docker for Mac. +Hyku provides a set of [Helm charts](chart/README.md) for deploying to a Kubernetes cluster. Detailed instructions are provided for deploying locally with Docker for Mac. ## Single Tenant Mode @@ -117,7 +117,7 @@ Much of the default configuration in Hyku is set up to use multi-tenant mode. T To enable single tenant, in your settings.yml file change multitenancy/enabled to `false` or set `SETTINGS__MULTITENANCY__ENABLED=false` in your `docker-compose.yml` and `docker-compose.production.yml` configs. After changinig this setting, run `rails db:seed` to prepopulate the single tenant. -In single tenant mode, both the application root (eg. localhost, or hyku.docker) and the tenant url single.* (eg. single.hyku.docker) will load the tenant. Override the root host by setting multitenancy/root_host in settings.yml or `SETTINGS__MULTITENANCY__ROOT_HOST`. +In single tenant mode, both the application root (eg. localhost, or hyku.test) and the tenant url single.* (eg. single.hyku.test) will load the tenant. Override the root host by setting multitenancy/root_host in settings.yml or `SETTINGS__MULTITENANCY__ROOT_HOST`. To change from single- to multi-tenant mode, change the multitenancy/enabled flag to true and restart the application. Change the 'single' tenant account cname in the Accounts edit interface to the correct hostname. @@ -172,7 +172,7 @@ The Samvera community is here to help. Please see our [support guide](./SUPPORT. ## Acknowledgments -This software was developed by the Hydra-in-a-Box Project (DPLA, DuraSpace, and Stanford University) under a grant from IMLS. +This software was developed by the Hydra-in-a-Box Project (DPLA, DuraSpace, and Stanford University) under a grant from IMLS. This software is brought to you by the Samvera community. Learn more at the [Samvera website](http://samvera.org/). diff --git a/chart/README.md b/chart/README.md index 22c37a16a..cd3dd3b66 100644 --- a/chart/README.md +++ b/chart/README.md @@ -208,4 +208,4 @@ There are equivalent kubectl commands for logs and accessing a shell, eg. ``` kubectl kubectl exec -it POD --namespace NAMESPACE -- /bin/bash kubectl kubectl logs POD --namespace NAMESPACE -``` \ No newline at end of file +``` diff --git a/chart/sample.yaml b/chart/sample.yaml index 2d9daba4c..4aef1b1f3 100644 --- a/chart/sample.yaml +++ b/chart/sample.yaml @@ -75,7 +75,7 @@ rails: ingress: tlsSecretName: demoapp-puma-tls - host: hyku.docker + host: hyku.test # use "0" for unlimited annotations: { kubernetes.io/ingress.class: "nginx", @@ -104,9 +104,9 @@ env: SETTINGS__CONTACT_EMAIL: admin@example.org SETTINGS__DEVISE__INVITATION_FROM_EMAIL: admin@example.org SETTINGS__FITS_PATH: /opt/fits/fits.sh - SETTINGS__MULTITENANCY__ADMIN_HOST: hyku.docker + SETTINGS__MULTITENANCY__ADMIN_HOST: hyku.test SETTINGS__MULTITENANCY__ADMIN_ONLY_TENANT_CREATION: true - SETTINGS__MULTITENANCY__DEFAULT_HOST: "%{tenant}.hyku.docker" + SETTINGS__MULTITENANCY__DEFAULT_HOST: "%{tenant}.hyku.test" SETTINGS__MULTITENANCY__ENABLED: true secret: diff --git a/chart/templates/setup-job.yaml b/chart/templates/setup-job.yaml index 12c440b82..2999cd49a 100644 --- a/chart/templates/setup-job.yaml +++ b/chart/templates/setup-job.yaml @@ -40,7 +40,7 @@ spec: - name: setup image: {{ .Values.rails.image.repository }}:{{ .Values.rails.image.tag }} volumeMounts: - - mountPath: /home/app/webapp/public/assets + - mountPath: /app/samvera/hyrax-webapp/public/assets name: shared subPath: assets command: ["/bin/bash"] diff --git a/chart/templates/sidekiq-deploy.yaml b/chart/templates/sidekiq-deploy.yaml index 9be242bcc..ad8e4f85a 100644 --- a/chart/templates/sidekiq-deploy.yaml +++ b/chart/templates/sidekiq-deploy.yaml @@ -52,28 +52,28 @@ spec: # initialDelaySeconds: 30 # Use sub-path for individual folders volumeMounts: - - mountPath: /home/app/webapp/tmp/imports + - mountPath: /app/samvera/hyrax-webapp/tmp/imports name: shared subPath: import_path - - mountPath: /home/app/webapp/tmp/exports + - mountPath: /app/samvera/hyrax-webapp/tmp/exports name: shared subPath: export_path - - mountPath: /home/app/webapp/tmp/derivatives + - mountPath: /app/samvera/hyrax-webapp/tmp/derivatives name: shared subPath: derivatives_path - - mountPath: /home/app/webapp/tmp/network_files + - mountPath: /app/samvera/hyrax-webapp/tmp/network_files name: shared subPath: network_files - - mountPath: /home/app/webapp/tmp/uploads + - mountPath: /app/samvera/hyrax-webapp/tmp/uploads name: shared subPath: upload_path - - mountPath: /home/app/webapp/public/uploads + - mountPath: /app/samvera/hyrax-webapp/public/uploads name: shared subPath: uploads - - mountPath: /home/app/webapp/public/assets + - mountPath: /app/samvera/hyrax-webapp/public/assets name: shared subPath: assets - - mountPath: /home/app/webapp/public/branding + - mountPath: /app/samvera/hyrax-webapp/public/branding name: shared subPath: branding envFrom: diff --git a/chart/templates/web-deploy.yaml b/chart/templates/web-deploy.yaml index 0e1723e66..cd84fbb67 100644 --- a/chart/templates/web-deploy.yaml +++ b/chart/templates/web-deploy.yaml @@ -41,28 +41,28 @@ spec: imagePullPolicy: IfNotPresent # Use sub-path for individual folders volumeMounts: - - mountPath: /home/app/webapp/tmp/imports + - mountPath: /app/samvera/hyrax-webapp/tmp/imports name: shared subPath: import_path - - mountPath: /home/app/webapp/tmp/exports + - mountPath: /app/samvera/hyrax-webapp/tmp/exports name: shared subPath: export_path - - mountPath: /home/app/webapp/tmp/derivatives + - mountPath: /app/samvera/hyrax-webapp/tmp/derivatives name: shared subPath: derivatives_path - - mountPath: /home/app/webapp/tmp/network_files + - mountPath: /app/samvera/hyrax-webapp/tmp/network_files name: shared subPath: network_files - - mountPath: /home/app/webapp/tmp/uploads + - mountPath: /app/samvera/hyrax-webapp/tmp/uploads name: shared subPath: upload_path - - mountPath: /home/app/webapp/public/uploads + - mountPath: /app/samvera/hyrax-webapp/public/uploads name: shared subPath: uploads - - mountPath: /home/app/webapp/public/assets + - mountPath: /app/samvera/hyrax-webapp/public/assets name: shared subPath: assets - - mountPath: /home/app/webapp/public/branding + - mountPath: /app/samvera/hyrax-webapp/public/branding name: shared subPath: branding readinessProbe: diff --git a/config/initializers/0db_created.rb b/config/initializers/0db_created.rb new file mode 100644 index 000000000..8564f0161 --- /dev/null +++ b/config/initializers/0db_created.rb @@ -0,0 +1,3 @@ +def db_created? + ::ActiveRecord::Base.connection_pool.with_connection(&:active?) rescue false +end diff --git a/config/initializers/apartment.rb b/config/initializers/apartment.rb index fa934a5a7..ca4c18c81 100644 --- a/config/initializers/apartment.rb +++ b/config/initializers/apartment.rb @@ -1,4 +1,4 @@ -unless ENV['DB_ADAPTER'] == 'nulldb' +if ENV['DB_ADAPTER'] != 'nulldb' && db_created? # You can have Apartment route to the appropriate Tenant by adding some Rack middleware. # Apartment can support many different "Elevators" that can take care of this routing to your data. # Require whichever Elevator you're using below or none if you have a custom one. diff --git a/docker-compose.production.yml b/docker-compose.production.yml deleted file mode 100644 index a1c794ae2..000000000 --- a/docker-compose.production.yml +++ /dev/null @@ -1,200 +0,0 @@ -version: '2' - -volumes: - fcrepo: - solr: - db: - redis: - zk1: - zk2: - zk3: - zkconf: - app: - assets: - -networks: - internal: - -services: - zoo1: - image: zookeeper:3.4 - restart: always - environment: - - ZOO_MY_ID=1 - - ZOO_SERVERS=server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 - expose: - - 2181 - - 2888 - - 3888 - volumes: - - zk1:/data - - zkconf:/conf - networks: - internal: - aliases: - - zookeeper_cluster - zoo2: - image: zookeeper:3.4 - restart: always - environment: - - ZOO_MY_ID=2 - - ZOO_SERVERS=server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 - expose: - - 2181 - - 2888 - - 3888 - volumes: - - zk2:/data - - zkconf:/conf - networks: - internal: - aliases: - - zookeeper_cluster - zoo3: - image: zookeeper:3.4 - restart: always - environment: - - ZOO_MY_ID=3 - - ZOO_SERVERS=server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 - expose: - - 2181 - - 2888 - - 3888 - volumes: - - zk3:/data - - zkconf:/conf - networks: - internal: - aliases: - - zookeeper_cluster - - solr: - image: solr:7.1 - command: solr -c -f -z zookeeper_cluster:2181 - depends_on: - - zoo1 - - zoo2 - - zoo3 - expose: - - 8983 - volumes: - - solr:/opt/solr/server/solr - networks: - internal: - - fcrepo: - image: cbeer/fcrepo4:4.7 - env_file: - - .env - expose: - - 8080 - volumes: - - fcrepo:/data - environment: - # - JAVA_OPTS=-Dfcrepo.modeshape.configuration="classpath:/config/jdbc-postgresql-s3/repository.json" -Dfcrepo.postgresql.host="db" -Dfcrepo.postgresql.username="postgres" -Dfcrepo.postgresql.password="DatabaseFTW" -Daws.accessKeyId="REPLACE_WITH_AWS_ID" -Daws.secretKey="REPLACE_WITH_AWS_SECRET_KEY" -Daws.bucket="REPLACE_WITH_BUCKET_NAME" - - JAVA_OPTS=${JAVA_OPTS} -Dfcrepo.modeshape.configuration="classpath:/config/file-simple/repository.json" -Dfcrepo.object.directory="/data/objects" -Dfcrepo.binary.directory="/data/binaries" - networks: - internal: - - db: - image: postgres:11.1 - env_file: - - .env - environment: - - POSTGRES_DB=${DATABASE_NAME} - - POSTGRES_PASSWORD=${DATABASE_PASSWORD} - - POSTGRES_USER=${DATABASE_USER} - volumes: - - db:/var/lib/postgresql/data - networks: - internal: - - app: - image: hyku/main:latest - env_file: - - .env - environment: - - FEDORA_URL=http://fcrepo:8080/fcrepo/rest - - IN_DOCKER=true - - LD_LIBRARY_PATH=/opt/fits/tools/mediainfo/linux - - PASSENGER_APP_ENV=production - - RAILS_CACHE_STORE_URL=memcache - - RAILS_LOG_TO_STDOUT=true - - REDIS_HOST=redis - - SECRET_KEY_BASE=asdf - - SETTINGS__ACTIVE_JOB__QUEUE_ADAPTER=sidekiq - - SETTINGS__FITS_PATH=/opt/fits/fits.sh - - SETTINGS__MULTITENANCY__ADMIN_HOST=hyku.docker - - SETTINGS__MULTITENANCY__ADMIN_ONLY_TENANT_CREATION=false - - SETTINGS__MULTITENANCY__DEFAULT_HOST=%{tenant}.hyku.docker - - SETTINGS__MULTITENANCY__ENABLED=true - - SETTINGS__SOLR__URL=http://solr:8983/solr/ - - SOLR_URL=http://solr:8983/solr/ - volumes: - - app:/home/app/webapp/tmp/uploads - - assets:/home/app/webapp/public/assets - networks: - internal: - - web: - extends: - service: app - depends_on: - - db - - solr - - fcrepo - - redis - - zoo1 - - zoo2 - - zoo3 - - memcache - - initialize_app - - db_migrate - expose: - - 3000 - - workers: - extends: - service: app - command: bundle exec sidekiq - depends_on: - - db - - solr - - fcrepo - - redis - - zoo1 - - zoo2 - - zoo3 - - initialize_app - - db_migrate - - initialize_app: - extends: - service: app - restart: on-failure - command: bundle exec rails zookeeper:upload - depends_on: - - zoo1 - - zoo2 - - zoo3 - - db_migrate: - extends: - service: app - restart: on-failure - command: bundle exec rails db:migrate - depends_on: - - db - - redis: - image: redis:3 - command: redis-server - volumes: - - redis:/data - networks: - internal: - - memcache: - image: memcached:1.5.14 - networks: - internal: diff --git a/docker-compose.yml b/docker-compose.yml index fe0460d76..7af4bd781 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -7,7 +7,7 @@ volumes: redis: zk1: zkconf: - app: + uploads: assets: cache: @@ -48,7 +48,7 @@ services: - SOLR_PORT_NUMBER=8983 - SOLR_ZK_HOSTS=zoo1 - VIRTUAL_PORT=8983 - - VIRTUAL_HOST=solr.hyku.docker + - VIRTUAL_HOST=solr.hyku.test depends_on: zoo1: condition: service_healthy @@ -88,7 +88,7 @@ services: app: build: . - image: hyku:latest + image: ghcr.io/samvera/hyku:latest env_file: - .env environment: @@ -108,13 +108,13 @@ services: - SETTINGS__BULKRAX__ENABLED=false - SETTINGS__FITS_PATH=/opt/fits/fits.sh # Comment out these 5 for single tenancy / Uncomment for multi - - SETTINGS__MULTITENANCY__ADMIN_HOST=hyku.docker + - SETTINGS__MULTITENANCY__ADMIN_HOST=hyku.test - SETTINGS__MULTITENANCY__ADMIN_ONLY_TENANT_CREATION=false - - SETTINGS__MULTITENANCY__DEFAULT_HOST=%{tenant}.hyku.docker - - SETTINGS__MULTITENANCY__ROOT_HOST=hyku.docker + - SETTINGS__MULTITENANCY__DEFAULT_HOST=%{tenant}.hyku.test + - SETTINGS__MULTITENANCY__ROOT_HOST=hyku.test - SETTINGS__MULTITENANCY__ENABLED=true # Comment out these 2 for multi tenancy / Uncomment for single - # - SETTINGS__MULTITENANCY__ROOT_HOST=hyku.docker + # - SETTINGS__MULTITENANCY__ROOT_HOST=hyku.test # - SETTINGS__MULTITENANCY__ENABLED=false - SETTINGS__SOLR__URL=http://admin:admin@solr:8983/solr/ - SOLR_ADMIN_USER=admin @@ -124,10 +124,10 @@ services: - SOLR_COLLECTION_NAME=hydra-development - PASSENGER_APP_ENV=development volumes: - - app:/home/app/webapp/tmp/uploads - - assets:/home/app/webapp/public/assets - - cache:/home/app/webapp/tmp/cache - - .:/home/app/webapp + - uploads:/app/samvera/hyrax-webapp/tmp/uploads + - assets:/app/samvera/hyrax-webapp/public/assets + - cache:/app/samvera/hyrax-webapp/tmp/cache + - .:/app/samvera/hyrax-webapp networks: internal: @@ -135,9 +135,10 @@ services: extends: service: app environment: - - VIRTUAL_PORT=80 - - VIRTUAL_HOST=hyku.docker + - VIRTUAL_PORT=3000 + - VIRTUAL_HOST=hyku.test depends_on: + - check_volumes - db - solr - fcrepo @@ -154,6 +155,7 @@ services: service: app command: bundle exec sidekiq depends_on: + - check_volumes - db - solr - fcrepo @@ -163,6 +165,21 @@ services: - db_migrate + check_volumes: + extends: + service: app + user: root + entrypoint: ["sh", "-c"] + command: + - > + # Do not recurse through all of tmp. derivitives will make booting + # very slow and eventually just time out as data grows + chown -f app:app /app/samvera/hyrax-webapp/tmp + chown -fR app:app /app/samvera/hyrax-webapp/tmp/pids + chown -fR app:app /app/samvera/hyrax-webapp/tmp/uploads + chown -fR app:app /app/samvera/hyrax-webapp/public/assets + chown -fR app:app /app/samvera/hyrax-webapp/tmp/cache + initialize_app: extends: service: app diff --git a/ops/nginx.sh b/ops/nginx.sh index a41dc1a6a..737c68509 100755 --- a/ops/nginx.sh +++ b/ops/nginx.sh @@ -15,14 +15,14 @@ fi if [[ $PASSENGER_APP_ENV == "development" ]] || [[ $PASSENGER_APP_ENV == "test" ]] then # this should be a volume in dev, it makes rails assets much faster - /bin/bash -l -c 'chown -fR app:app /home/app/webapp/tmp/cache' - /sbin/setuser app /bin/bash -l -c 'cd /home/app/webapp && bundle exec rails db:test:prepare' + /bin/bash -l -c 'chown -fR app:app /app/samvera/hyrax-webapp/tmp/cache' + /sbin/setuser app /bin/bash -l -c 'cd /app/samvera/hyrax-webapp && bundle exec rails db:test:prepare' fi if [[ $PASSENGER_APP_ENV == "production" ]] || [[ $PASSENGER_APP_ENV == "staging" ]] then - /bin/bash -l -c 'chown -fR app:app /home/app/webapp/public/assets' # mounted volume may have wrong permissions - /sbin/setuser app /bin/bash -l -c 'cd /home/app/webapp && rsync -a public/assets-new/ public/assets/' + /bin/bash -l -c 'chown -fR app:app /app/samvera/hyrax-webapp/public/assets' # mounted volume may have wrong permissions + /sbin/setuser app /bin/bash -l -c 'cd /app/samvera/hyrax-webapp && rsync -a public/assets-new/ public/assets/' fi exec /usr/sbin/nginx diff --git a/ops/webapp.conf b/ops/webapp.conf index 4f0dcdec2..9f513e18c 100644 --- a/ops/webapp.conf +++ b/ops/webapp.conf @@ -1,7 +1,7 @@ server { listen 80; server_name _; - root /home/app/webapp/public; + root /app/samvera/hyrax-webapp/public; client_body_in_file_only clean; client_body_buffer_size 32K; diff --git a/tmp/pids/.keep b/tmp/pids/.keep new file mode 100644 index 000000000..e69de29bb