Sprint 16 and 17 updates #570
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
afwillia
commented
Aug 31, 2023
afwillia
commented
- DCA & Schematic versions added to UI
- Schematic manifest get option strict_validation set to false. Fixes issues editing cells in Google Sheets and Excel
- Staging DCA now uses staging branch of data_curator_config for the data model and template dropdown menu
- Fixed a bug to enable selecting different templates without a refresh of the app
- Display an error to users who don’t have access to projects instead of receiving a disconnect error from the app
Fix the conflicts error from #462
* Set instance size to xxxlarge and log level to verbose during app deploy. * Only install schematic from pip, never use develop branch. * configureApp() will fail if the app does not exist yet. Put after deployApp() in case of first deployment. * try setting upload to FALSE and then configure it. The configApp() fails after deployApp() * if app exists, configure then deploy. Otherwise, deploy then configure. * change appName to testingXXX so it creates a new app instance and tests if the rsconnect configuration logic works. * Logic worked, go back to testing1 name
* hone in where python is failing 2 * debug python venv path * use newest reticulate * update all packages * hone in where python is failing 3 * hone in where python is failing 4 * hone in where python is failing 4 * hone in where python is failing 5 * hone in where python is failing 5 * Revert "hone in where python is failing 5" Revert changes trying to use python3.10. It's not possible for shinyapps.io at the moment. This reverts commit 1d7c566. * Revert "hone in where python is failing 5" This reverts commit a833ea9. * Revert "hone in where python is failing 4" This reverts commit 34bfab8. * Revert "hone in where python is failing 4" This reverts commit 6766939. * Revert "hone in where python is failing 3" This reverts commit 5cd8c7d. * Revert "update all packages" This reverts commit 08bd7db. * Revert "use newest reticulate" This reverts commit 49d835d. * Revert "debug python venv path" This reverts commit c9e16b5. * Revert "hone in where python is failing 2" This reverts commit 35cdfe0. * Revert "hone in where python is failing." This reverts commit 221ecfa. * Revert "Test py commands in global.R" This reverts commit 388b183. * Revert "use pip" This reverts commit 058c698. * Revert "check env var in shiny log" This reverts commit fad46ec. * Revert "Remove blank line" This reverts commit 68bc652. * Revert "don't use poetry shell" This reverts commit c6bad6a. * Revert "read venv in schematic set up func" This reverts commit 79cd7f4. * Revert "Try not using a virtual env" This reverts commit 3c5ae28. * Revert "Try hard coding python path" This reverts commit 1a2c087. * Revert "set reticulate python env in workflow" This reverts commit 76753bb. * Revert "Try setting python path in deployApp()" This reverts commit 287a921. * Revert "apt-get remove instead of uninstall" This reverts commit 66be0bb. * Revert "Don't load reticulate by default." This reverts commit 67d14ef. * Revert "Uninstall python3.8" This reverts commit 8d6ae51. * Revert "Add reticulate python env var to deployment workflow" This reverts commit 661490b. * Revert "Remove reticulate python from global.R" This reverts commit 6cb177f. * Revert "Add asset view back to waiter." This reverts commit 79d3e75. * Revert "Don't use renv to avoid png error" This reverts commit 27ab92c. * Revert "Explicitly use python from virtualenv" This reverts commit 7e97c3f. * Revert "Use renv" This reverts commit 15c4845. * Revert "Add renv" This reverts commit 81c4a0a. * Revert "Add color theme rds files." This reverts commit 38e90d6. * Revert "Add missing }" This reverts commit 924af32. * Revert "Use python3.10 to run config_schema.py" This reverts commit 1e82bf6. * Revert "Don't read template menu config file in global.R" This reverts commit bf6b67a. * Revert "Actuallly write updated .Renviron this time" This reverts commit 1054726. * Revert "Add DCA_APP_URL env var." This reverts commit e9299fc. * Revert "Use jammy ubuntu repo" This reverts commit 28e533e. * Revert "Remove config generation in workflow. The app does that." This reverts commit d72d62a. * Revert "Fix double quites" This reverts commit 1b8daf9. * Revert "Update package repo." This reverts commit 6aadb4b. * Revert "Add missing quote." This reverts commit 70ae71f. * Revert "Use rocker/rstudio:latest to get most recent ubuntu (jimmy) and install python3.10 correctly" This reverts commit 7e36fce. * Revert "Use python 3.10 explicitly." This reverts commit 118145b. * Revert "Print out python version to debug" This reverts commit 49b4fbf. * Revert "Add python venv" This reverts commit 71bafb2. * Revert "Try setting DEBIAN_FRONTEND env var to disable keyboard prompt during apt-get update. Move into apt get command again" This reverts commit 915d970. * Revert "Try setting DEBIAN_FRONTEND env var to disable keyboard prompt during apt-get update. Move into apt get command" This reverts commit a4fd440. * Revert "Try setting DEBIAN_FRONTEND env var to disable keyboard prompt during apt-get update." This reverts commit c024468. * Revert "Add -y flag to apt-get update so interactive text doesn't hinder workflow." This reverts commit 01264d6. * Revert "Use python 3.10 because schematic doesn't support 3.11" This reverts commit 35dac98. * Revert "Use python 3.11 because schematic doesn't support 3.8" This reverts commit 24dc62f. * Revert changes related to debugging python. Shinyapps.io requires python 3.8. * Revert "Use python 3.11 because schematic doesn't support 3.8" Stick with python 3.8 because shinyapps.io doesn't support other versions. This commit undoes all of the tweaking related to updating and debugging python 3.10. This reverts commit 24dc62f. * Add color theme rds files. * Update asset view text in waiter after selecting asset view. * Add offline mode. If can't access synapse OAuth redirect. Go to landing page with no access token. * Add try() to redirect, if it fails, go to landing page without an access token. * Add offline mode to first and second tabs. Use dca_schematic_api env to be reticulate, rest, or offline. * use dca_schematic_api set to offline to go into ui without an access token. * use dca_schematic_api var to determine if running in offline mode. * Add a template dropdown for offline use. * Add offline mode to manifest generation tab. * Set instance size and log level during deployment. * Don't ever install develop schematic * Remove single quotes from echo. * remove APP_URL env var * Add appUrl to Renviron. * Add missing quote * Add mock config when not using python * add dashboard to ui * source dashboard functions * add dashboard to server * Make waiter screen default color slightly transparent. * Update include config file * Revert make waiter transparent by default. It was too transparent. * Add get_component_requirements for reticulate * Use reticulate wrapper for get_component requirements * Get syn_store object from synapse_driver * Allow manifest name to have component * Write manifest name with component * Hide dropdown menu from first page. * Remove parallel processing from validating data in loop. For some reason, it will not run properly when schematic encounters errors. But lapply() works fine. * Read in reactive config file outside of reticulate config file update. * dashboardFuns can use schematic via reticulate or rest api * dashboard can use schematic via reticulate or rest api * Add rest api arguments to dashboard * Read manifest basename from schematic_config.yml * Move reticulate/rest switch inside of get_schema_nodes(). Add () to synapse_user_profile. * Merge develop-add-rest-api into schematic-rest-api. This adds reticulate and REST api access to schematic. * Remove browser() from server.R * Add use_schema_label and manifest_record_type to manifest/validate. * Add waiter until data type menu updates * Simplify logic in manifest download for excel sheet * Expand downloadHandler content function to download the manifest from schematic. This allows users to skip downloading manifests if they don't need it. * Expose use_schema_label in model/submit to user. This can be toggled with DCA_SCHEMATIC_SUBMIT_USE_SCHEMA_LABEL env var. * Update contributors (#488) * If a server error is encountered with manifest/validate, return a list with an error in a similar format to other schematic validation errors. That way validationResult() can handle it as a proper error. (#490) * Develop offline mode (#491) * turn off dashboard in offline mode for now * fix logic for offline manifest to show up. * Add offline mode for manifest download. It will download a mock data csv. Add an offline option for manifest validation as well. * Add another download handler for downloading a corrected mock offline manifest * Add logic to download a corrected mock manifest in offline mode. * Develop update logo (#492) * Update default logo to transparent background * Update sage logo in waiter to transparent background * Add sage logo with transparent background * Remove extra heart from footer. * Remove synapse ID from asset view labels and waiter screens. (#493) * fix grammar (#494) * Develop toggle db (#495) * Add UI toggle for dashboard based on DCA_COMPLIANCE_DASHBOARD env var. * use example config.json * For validationresult in dashboard, concatenate the error messsage * Change branch in install_github() * Update waiter color to dcc color scheme. * When validation fails, remove option to download manifest as google sheet. * Add gihub PAT to renviron to avoid API error on deployment. * Dev beta fix templates (#502) * Remove project dirs in www/ since their relevant files are in the directory itself and in dca_themes/ * Update htan config to use file/record type * Move template config.json files into www subdirectory. * Update htan config so it can be read by jsonlite::fromJSON() * Add other assay to default template config. * Add other assay to example template config. * Read template config file and synID mapping env var, DCA_TEMPLATE_MENU_CONFIG, as a variable in global.R * Clean up template config code. Ensure the template dropdown menu updates to match the selected asset view. * Update config path in global.R * Update path of default template in server.R * Test deploying to shinyapps.io by moving R/ files into functions/ during deployment. * add covr and rex to fix release error * Don't use Renviron when installing packages. * try renv * add png and xml libs * add rsconnect * Don't source renv files. * Beta shinyapps deploy (#504) * show files to see if R/ directory is still present. * change devel branch in deployment * remove NAMESPACE and DESCRIPTION * install packages manually to avoid shinypop error. * add packages missing from rsconnect deploy error. * Add missing env vars to deployment script * Add template menu config env var to deployment workflow * Change url to testing1 * Add data model url env var to deployment. * Use single quotes * Fix typo to use .Renviron instead of .Rprofile. * Download and save data model before writing schematic config. * Revert listing files for debugging. * use download.file() instead of system(curl...) * Change branch in shinyapps deploy * Only use one DCC because shinyapps.io workers share the same reticulate environment. So different sessions can only use the same schematic config. * Add 'Step X' to sidebar tabs to guide users. * Revert add step X to sidebar. * Beta schematic rest api (#507) * Set up test instance for fork * Move parse_env_vars() and update_logo() to utils.R * Add test to parse_env_vars that checks if it errors out on empty string. * source files at top of global.R * Revert change to appName * Add naming logic back to deployment workflow * Beta schematic rest api (#508) * Set up test instance for fork * Move parse_env_vars() and update_logo() to utils.R * Add test to parse_env_vars that checks if it errors out on empty string. * source files at top of global.R * Revert change to appName * Add naming logic back to deployment workflow * Make manifest_url() reactive so link shows up correctly. * Beta submit body (#513) * Add 'Step X' to sidebar tabs to guide users. * Revert add step X to sidebar. * Update submit and validate to upload data in the request body, not header. * Copy R/schematic_rest_api.R to functions/ so it works on shinyappps.io * Update validate and submit calls to use a file name instead of passing a json object. * Add renv::isolate to Dockerfile * source renv activate on startup. * Simplify renv install and use jammy repo * Use updated shiny-base image that has libglpk-dev to avoid networkD3 error installing from binary. * Try renv and setting the repo to focal binaries. * Add renv::isolate inside of rsconnect deploy step * Only do deployApp() to test where deployment fails * try avoiding cache * use checkout v3 * use ubuntulatest and jammy repo. * Use focal once again. * update renv.lock * Update renv lock from ubuntu * Back to install-pkgs script * fix spacing * ubuntu 20.04 * checkout v3 and pandoc v2 * Don't source renv/activate.R * Remove R package install script to use renv * Update renv.lock for R4.1.3 * Use renv in workflow. restore and isolate in deploy step * Remove repo from Dockerfile because it's in the renv.lock file. * Use shinydashboardplus from github, not cran * need to source activate.R in Rprofile * Update renv activate to v0.17 * Update base image for stringi lib. * Update shiny-base image to fix stringi error * Update base image to afwillia/shiny-base:release-1.2 * Use same versioon of R as the AWS deployment. * Test removing renv calls from deployApp step * confirmed removing renv from deployApps crashes. Add renv::restore() back. * Install rsconnect in deployapps step * Remove rsconnect, remotes, and packrat from renv.lock because these are specific to shinyapps.io deployment workflow. * Add updated renv/activate.R and .gitignore * Revert changes to renv/activate.R because this breaks the shinyapps.io deployment * use sagebionetworks/shiny-base and install extra libs in DCA dockerfile * Try installing from source to avoid stringri error on jammy * Add rstudio jammy repo to dockerfile to speed up package installation. * Test overriding renv repo in workkflow to jammy binary * Still uses ubuntu20 cache, despite running on ubuntu22. What happens if we bypass_cache. * Add 'with' in setup-renv * Don't indent 'with' in setup-renv * shinydashboardPlus dependency fastmap fails to install because there is no 'GLIBCXX_3.4.29'. Abandon this quest and use previously functional setup. * Try updating glibc and avoid shinydashboardplus and fastmap error * Updating glibc-source did not work. Go back to existing workflow. * Update node for security patch * Remove default node then install from nodesource * Update renv gitignore with defaults from v0.17 * Beta dcc config (#515) * Add config file for DCC settings * Use dcc config file instead of env vars. * Use dcc config file in ui instead of env vars. * Use dcc config file in server instead of env vars. * Add theme colors to dcc config file * Change theme switching to use dcc config file instead of rds files. * Remove rds theme files. * rename csbc template file to mc2 * Make waiter background transparent. * Update waiter background to be slightly transparent * Update waiter landing text from ux feedback * Change waiter text and add things may take a minute. * welcome user by name not synapse username. * Remove env variables from shinyapps.io workflow that are in dcc_config.csv * Update staging app to use the schematic rest api service. * Don't install schematic with rest api deploy * Remove data model config from workflow since those files are hard coded. * Set host instead of port in Renviron. * Update README for new configuration options with environment variables. * Renviron instead of Rprofile. * Format files for github rendering * clean up readme for github formatting * Fixes FDS-73. An issue where validation was failing. * move shinyapps_deploy documentation to docs/ * Update link to shinyapps.io deployment docs * Update link to OAuth client setup docs * Update shinyapps.io deployment for the REST/reticulate version of DCA. * Fix extra characters in code chunks. * Add error checking for DCA_SCHEMATIC_API_TYPE env var * Update workflow to conditionally install python if using reticulate * Remove space between $ and {. * fix variable name in if statement * add shell: bash to venv activate * Write .Renviron after checking out repo * checkout repo first. * update system then checkout * try using setup-python for caching * Don't use setup-py yet * try caching venv * put IFs into one line * update python cache * try using setup-python with pip not pipenv * fix typo * set env path * update cache dir * update cache dir 2 * set hash files in key * update cache key * create cache key from release 23.1.1 schematic poetry lock * download with wget * fix expression * use double & * only cache venv.zip * move if statements * don't use pip cache dir * add pip cache dir * add pip cache dir 2 * add pip cache dir 4 * add pip download cache dir * print out pip version * move schematic install * Try installing schematicpy outside of venv for caching * upgrade pip * use default pip * create data model config * update testing1 instance with schematic rest api * hotfix when validationResult returns a warning to highlight particular columns, if the vector is length 0 then DTable quietly breaks. This results in the submit button not showing up. To address this, check the length of cells to highlight. If it's 0, skip highlighting. Need to determine why validationResult is not returning the correct cells to highlight. * add error checking to model_submit * write manifest to temp dir * Add error handling and table_manipulation arguments to model_submit. * Update tests for model_submit * Update model_submit arguments to use dcc_config and add arguments. * Redploy app with reticulate to staging instance. * Add restrict rules to validate * update validate tests * Update README * Don't cache python installation, it doesn't speed things up much. * Install schematicpy in its own step
* add rsconnect * Don't source renv files. * Beta shinyapps deploy (#504) * show files to see if R/ directory is still present. * change devel branch in deployment * remove NAMESPACE and DESCRIPTION * install packages manually to avoid shinypop error. * add packages missing from rsconnect deploy error. * Add missing env vars to deployment script * Add template menu config env var to deployment workflow * Change url to testing1 * Add data model url env var to deployment. * Use single quotes * Fix typo to use .Renviron instead of .Rprofile. * Download and save data model before writing schematic config. * Revert listing files for debugging. * use download.file() instead of system(curl...) * Change branch in shinyapps deploy * Only use one DCC because shinyapps.io workers share the same reticulate environment. So different sessions can only use the same schematic config. * Add 'Step X' to sidebar tabs to guide users. * Revert add step X to sidebar. * Beta schematic rest api (#507) * Set up test instance for fork * Move parse_env_vars() and update_logo() to utils.R * Add test to parse_env_vars that checks if it errors out on empty string. * source files at top of global.R * Revert change to appName * Add naming logic back to deployment workflow * Beta schematic rest api (#508) * Set up test instance for fork * Move parse_env_vars() and update_logo() to utils.R * Add test to parse_env_vars that checks if it errors out on empty string. * source files at top of global.R * Revert change to appName * Add naming logic back to deployment workflow * Make manifest_url() reactive so link shows up correctly. * Beta submit body (#513) * Add 'Step X' to sidebar tabs to guide users. * Revert add step X to sidebar. * Update submit and validate to upload data in the request body, not header. * Copy R/schematic_rest_api.R to functions/ so it works on shinyappps.io * Update validate and submit calls to use a file name instead of passing a json object. * Add renv::isolate to Dockerfile * source renv activate on startup. * Simplify renv install and use jammy repo * Use updated shiny-base image that has libglpk-dev to avoid networkD3 error installing from binary. * Try renv and setting the repo to focal binaries. * Add renv::isolate inside of rsconnect deploy step * Only do deployApp() to test where deployment fails * try avoiding cache * use checkout v3 * use ubuntulatest and jammy repo. * Use focal once again. * update renv.lock * Update renv lock from ubuntu * Back to install-pkgs script * fix spacing * ubuntu 20.04 * checkout v3 and pandoc v2 * Don't source renv/activate.R * Remove R package install script to use renv * Update renv.lock for R4.1.3 * Use renv in workflow. restore and isolate in deploy step * Remove repo from Dockerfile because it's in the renv.lock file. * Use shinydashboardplus from github, not cran * need to source activate.R in Rprofile * Update renv activate to v0.17 * Update base image for stringi lib. * Update shiny-base image to fix stringi error * Update base image to afwillia/shiny-base:release-1.2 * Use same versioon of R as the AWS deployment. * Test removing renv calls from deployApp step * confirmed removing renv from deployApps crashes. Add renv::restore() back. * Install rsconnect in deployapps step * Remove rsconnect, remotes, and packrat from renv.lock because these are specific to shinyapps.io deployment workflow. * Add updated renv/activate.R and .gitignore * Revert changes to renv/activate.R because this breaks the shinyapps.io deployment * use sagebionetworks/shiny-base and install extra libs in DCA dockerfile * Try installing from source to avoid stringri error on jammy * Add rstudio jammy repo to dockerfile to speed up package installation. * Test overriding renv repo in workkflow to jammy binary * Still uses ubuntu20 cache, despite running on ubuntu22. What happens if we bypass_cache. * Add 'with' in setup-renv * Don't indent 'with' in setup-renv * shinydashboardPlus dependency fastmap fails to install because there is no 'GLIBCXX_3.4.29'. Abandon this quest and use previously functional setup. * Try updating glibc and avoid shinydashboardplus and fastmap error * Updating glibc-source did not work. Go back to existing workflow. * Update node for security patch * Remove default node then install from nodesource * Update renv gitignore with defaults from v0.17 * Beta dcc config (#515) * Add config file for DCC settings * Use dcc config file instead of env vars. * Use dcc config file in ui instead of env vars. * Use dcc config file in server instead of env vars. * Add theme colors to dcc config file * Change theme switching to use dcc config file instead of rds files. * Remove rds theme files. * rename csbc template file to mc2 * Make waiter background transparent. * Update waiter background to be slightly transparent * Update waiter landing text from ux feedback * Change waiter text and add things may take a minute. * welcome user by name not synapse username. * Remove env variables from shinyapps.io workflow that are in dcc_config.csv * Update staging app to use the schematic rest api service. * Don't install schematic with rest api deploy * Remove data model config from workflow since those files are hard coded. * Set host instead of port in Renviron. * Update README for new configuration options with environment variables. * Renviron instead of Rprofile. * Format files for github rendering * clean up readme for github formatting * Fixes FDS-73. An issue where validation was failing. * move shinyapps_deploy documentation to docs/ * Update link to shinyapps.io deployment docs * Update link to OAuth client setup docs * Update shinyapps.io deployment for the REST/reticulate version of DCA. * Fix extra characters in code chunks. * Add error checking for DCA_SCHEMATIC_API_TYPE env var * Update workflow to conditionally install python if using reticulate * Remove space between $ and {. * fix variable name in if statement * add shell: bash to venv activate * Write .Renviron after checking out repo * checkout repo first. * update system then checkout * try using setup-python for caching * Don't use setup-py yet * try caching venv * put IFs into one line * update python cache * try using setup-python with pip not pipenv * fix typo * set env path * update cache dir * update cache dir 2 * set hash files in key * update cache key * create cache key from release 23.1.1 schematic poetry lock * download with wget * fix expression * use double & * only cache venv.zip * move if statements * don't use pip cache dir * add pip cache dir * add pip cache dir 2 * add pip cache dir 4 * add pip download cache dir * print out pip version * move schematic install * Try installing schematicpy outside of venv for caching * upgrade pip * use default pip * create data model config * update testing1 instance with schematic rest api * hotfix when validationResult returns a warning to highlight particular columns, if the vector is length 0 then DTable quietly breaks. This results in the submit button not showing up. To address this, check the length of cells to highlight. If it's 0, skip highlighting. Need to determine why validationResult is not returning the correct cells to highlight. * add error checking to model_submit * write manifest to temp dir * Add error handling and table_manipulation arguments to model_submit. * Update tests for model_submit * Update model_submit arguments to use dcc_config and add arguments. * Redploy app with reticulate to staging instance. * Add restrict rules to validate * update validate tests * Update README * Don't cache python installation, it doesn't speed things up much. * Install schematicpy in its own step * use github_token in secrets context instead of github context to hopefully resolve an error with renv installing a github repo * upgrade to renv 0.17.2 * Print dca_schematic_rest_api env var to console for debugging on shinyapps.io * Check if just deploying an existing app speeds up deployment. * Switch names in asset view display the label instead of synapse id * Add synapse REST API wrappers for getting project data * Use synapse API instead of schematic to get storage projects. * Use synapse API instead of schematic to get the dataset folders within a project. * Use synapse API to get files in a dataset folder. * Fix storage_projects_files_py so it returns output. * remove debugging message * configuring and deploying app in same step does not change deploy times. * remove duplicate github_token in workflow * fix merge conflicts * Remove extra quote * Add toggle to download file to return data.frame * Update manifest/download * Download manifest from synapse api. * Update manifest_record_type in submit to table_and_file * Add optional code for using synapse REST api. * Show project name instead of synapse Id in asset view menu * Add roxygen documentation to synapse functions * Toggle synapse and schematic APIs by using DCA_SYNAPSE_PROJECT_API env var. * Move waiter screens on data selection page before any business logic is performed * Update testing instance with schematic REST API * Use dev schematic api service. * Don't use synapse API to download manifest file. * Update schematic rest api functions * make use_annotations FALSE instead of 'false' * set output format for manifest/generate * Add manifest output env var to shinyapps deployment * In waiter, display project name. * only create files vector if files are in dataset folder * Use updated shiny-base with longer http_keepalive_timeout * Use correct image repo * Use promises package * Use manifest_generate asynchronously with future_promise(). Need to test this on AWS with concurrent users. Also, need to add a waiter screen if successful * Add future package to renv * Load future package and create multisession plan * Move waiter screens to work with future_promise. * Add async call to model/validate. * Set up global.R to work in offline mode with synapse REST API and dcc_config.csv * Get offline mode to work with synapse REST API and dcc_config.csv. Let async schematic calls work in offline mode. * WIP move all of validation step into an async call. Need to add formatting validation table to async. * Set up submit for offline mode mock submission. * update shiny-baes image with simple_scheduler = 1 * Add async functionality to model_submit * Use shiny-base 1.4 which does not set simple_scheduler. * remove a reactive variable from inside a future_promises() statement. * Add test manifests. * Add a shiny server config file * Copy this repo's shiny server conf into the docker image * Create one tab for each menu, to add a waiter between each screen for async. * Add tabs to global.R * Refactor server to handle one tab per dropdown menu. Add async calls to synapse API. * Set default waiter sleep to 0. Sleep messes with concurrent users, causing unnecessary delays. * Have multiple observers for buttons with various priorities. * Remove async call from fast synapse api calls, but change the waiter default to not sleep. * Simplify waiter screen messages. * Fix bug in data_list if no files were downloaded from Synapse. * use all available cores for asynchronous calls. * Say asset view loading may take a minute in the waiter. MC2 took much longer than the demo. * Add async to synapse asset view scope call. It takes a long time with MC2. * Fix waiter screens after selecting asset view so they don't overlay. * Add async calls to synapse folder and file apis. * Disable the asset view button after it has been clicked. Then enable it only if the asset view dropdown has changed. This avoids a bug with the asynchronous code where the waiter will not disappear on subsequent clicks of the same button. * Disable project, template, and folder buttons after clicking. Enable them again if the input changes. This avoids a bug with the asynchronous code. * Remove header dropdown from UI. * Remove header selection bar from server. * Hide the side bar selectors so users progress through the app once tab at a time. Allow manifest download to trigger after clicking the download tab or proceed button from folder selection screen. * add config options for AD Knowledge Portal (#525) * temp not ignore config json and csv * AD template config * add AD config * add AD portal logo * re-ignore json and csv * fix typo * use json-ld for data model * Fix typo in include logo href * Add adknowledgeportal logo * Make tab icons unique * Various UI text tweaks to simplify layout * Give boxes in validation page an id so we can hide them. * Hide and show the validation tab boxes to guide users through the submission process * Add reminder below file upload to remove blank rows from csv. * Add reminder that spreadsheet apps may add extra rows to template. * Add help text to prompt user to click download template tab to continue. * Set waiter sleeps to 0 to help with concurrent users. * Only show submit box if validation passes. * Remove placeholder text for get template tab. * Use available cores for futures * Add manifest_record_type to dcc_config * In submit, use manifest_record_type from config file * Add manifest_record_type to dcc_config in README * Change text of download template header to show the template and folder * Set name of download template header based on selected folder and template * Use file_only instead of file for AD manifest_record_type. * Move the right button closer to the left sidebar * Remove the hover animation from disabled buttons. * Use schematic storage/project/datasets to get folders within a project. Don't use synapse entity children because it doesn't return the corect information for nested file views. * After selecting folder automatically switch to download template tab. * update csvInFile module to remove empty columns and rows. * Remove Synapse from waiter messages. * After trimming blank columns and rows from uploaded csv, write it so it can be passed to schematic submit endpoint. * Use shiny base image with updated node. * Don't deploy to shinyapps.io with AD configuration. * add config info for VEOIBD DCA (#526) * Disable dashboard for now. * Add VEOIBD logo * Clarify instructions in README * Add BTC * Fix bug where validation error table doesn't display. Show error message waiter for 2.5 seconds like current app does. * remove defunct code from ui. * Update UI description comments * Clean up indentation in UI code * On folder selection tab, remove note and button. * Clean up indentation in server code. * Suppress rowname warning on tibble in csv upload * don't show column type message reading dcc_config.csv * Clean up whitespace in global.R * Remove theme files from global.R * download template as excel by default. Opt in to google sheet. * Add DCA_SYNAPSE_PROJECT_API env var to global var. * add google sheet and excel download button in the same box, just display the button based on what is actually generated. * use dcc_config.csv instead of env var to generate a google sheet or excel. output a link or download button depending on what output format is specified. also, fix template name issue in google sheet * Add an example template config file for BTC. Use google_sheet for Demo instance template as a test. * remove browser statement from server. * Sort items in folder dropdown menu --------- Co-authored-by: Abby Vander Linden <[email protected]> Co-authored-by: Dan Lu <[email protected]>
* update config files, logo, and data models * update config * Update dcc_config.csv * Update dcc_config.csv
* Change input_token to access_token for schematic update * Change input_token to access_token for schematic update in REST API functions * Change input_token to access_token for schematic update in REST API functions in functions dir
* Add a new demo fileview that submits with upsert * Add template dropdown config for upsert demo
* Update template config script name * Clarify how to set up config.json for a simple data model. * Remove header link
* Read dcc_config file from an environment variable to customize its source. * Change default template config to download from demo on github * Read template config file from file or from github * remove template config files. These are in sage-bionetworks/data_curator_config * Update server code to get logo image and link from dcc_config. * remove reading template config file from global.R * remove dcc_config.csv
* if user has access to only one DCC, simulate a click of asset view button to skip the selection page. (#536) * Dev selection banner (#537) * Add data selection banner to top. * Update header selection banner with project, template, and folder. * Add placeholder text to selection header boxes * Read use_annotations argument to manifest/generate from dcc_config (#538) * Dev skip download (#539) * On folder selection page, add a button to skip ahead to validation tab without downloading a manifest * Add logic to skip template download if user desires. * show download and validate tabs if skipping download * show selection header automatically after DCC page. Remove badge from the button. * remove badge from selection header dropdown and change text to 'selected data' * Folder selection in header now updates. * show data selection banner after selecting project * hide data selection banner until template selection * Hide sidebar at startup
* Use selectizeInput with options maxItems = '1' to remove the dropdown icon from the selection header. * Remove grey background from header selection boxes.
…on variables. Point to a branch of data_curator_config that does uses existing shinyapps.io DCCs. (#543)
* Use hide_blanks from config file * Add hide_blanks argument to model_submit
…d to not be NULL in manifest_id observer (#550)
* Set manifest_id reactive to NULL after submitting. Require manifest_id to not be NULL in manifest_id observer * Swap order of folder and template selection tabs. * Switch template download buttons to template tab instead of folder tab. The two tabs were reordered in ui.R * Move template and folder name display to template download tab logic. Make tab transition to template download tab after clicking download. * Change text of buttons for clarity. * Change text of download header
* Add service desk link in feature_request.md Add service desk link to issues template. * Add service desk link in bug_report.md Add service desk link in bug report issues.
* Fix links README.md Update links to dcc_config documentation. * Update README.md remove links to sage apps
* Develop (#522) * Develop no poetry (#477) * Set instance size to xxxlarge and log level to verbose during app deploy. * Only install schematic from pip, never use develop branch. * configureApp() will fail if the app does not exist yet. Put after deployApp() in case of first deployment. * try setting upload to FALSE and then configure it. The configApp() fails after deployApp() * if app exists, configure then deploy. Otherwise, deploy then configure. * change appName to testingXXX so it creates a new app instance and tests if the rsconnect configuration logic works. * Logic worked, go back to testing1 name * coerce warning to character so strsplit() does not fail. (#520) --------- Co-authored-by: Rongrong Chai <[email protected]> * Update trigger for docker build workflow to use release version tag and dev branches * Remove empty line at EOF --------- Co-authored-by: Rongrong Chai <[email protected]>
* Upgrade renv to v1.0.0 * quote dev branch to capture regex pattern
* Remove schematic_config.yml from .github * Use example data model in schematic_config.ym
* use future = TRUE in renderDT to allow asynchronous table processing. * Use future plan multicore instead of multisession. This appears to be more stable with renderDT(future=TRUE,...) * Add Renviron with RENV_CONFIG_SANDBOX_ENABLED = FALSE to avoid long startup times with renv v1.0.0
#560) * Refactor trimEmptyRows to avoid an error uploading manifests with 74+ columns. This issue only happens in the Ubuntu docker image, not Mac. The error is * Revert to renv 0.17.3 * Remove Renviron with RENV_CONFIG_SANDBOX_ENABLED = FALSE since we have reverted to renv 0.17.3
…acter to avoid uppercase booleans. (#562)
…ashing without error.
…s dcc_config.csv (#565)
* Get dca-template-config from the same branch in data_curator_config as dcc_config.csv * Add precommit file from precommit::use_precommit * Add .pre-commit-config.yml to .Rbuildignore * Don't build container on dev branches. Build with rc tag * Don't run test schematic workflow on push * Don't run shiny deploy workflow * Run workflows on pr into main * Make pr trigger pull_request
…message with more information about the error and how to proceed.
… projects are returned. If no projects are returned, update the waiter screen with a message about fileview and project permissions (#567)
#563) * enable template download button after changing the template dropdown input * Enable the download template button if previous tab inputs change but manifest type stays the same.
* WIP: Add DCA and Schematic versions to footer of dashboard. * Get schematic version via api call * Use schematic version from variable in global.R * Add version as an env variable to docker container * When building docker container, use the github ref as a build arg to note the version. * Change version variable to DCA_VERSION * Set DCA_VERSION env var to a variable in the global env * Show DCA version from the env var * Use GITHUB_REF_NAME env var to set version env var. * specify github context for github_ref_name var * Update github ref name to github.REF_NAME * For clarity, set build arg to DCA_VERSION instead of TAG. * Add DCA_VERSION env var to .Renviron so it is loaded as env var in shiny server. * Make DCA_VERSION from docker build-arg an env var in the container. Then in dca_startup.sh, add the env var to Renviron * Add error handling for schematic version api call
…ndicates a permission issue instead of no folders.
…Bionetworks/data_curator into dev-dca-project-no-folders
…ndicates a permission issue instead of no folders.
When a project contains no folders, display a popup box instead of cr…
…rror Dev manifest generate error
Add error handling to submit
|
These PRs/commits have been reviewed in the develop branch - merging this without more review. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.