Skip to content

Releases: shapiromatron/hawc

2024 - Q3

24 Oct 20:35
@caseyhans caseyhans
2024-Q3
35e1a97
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2024 - Q3 Latest
Latest

October 24th, 2024

We're proud to release 2024.Q3. This release features a new labelling tool, two new visualizations, and various export improvements.

Assessment members can now create and apply labels to visuals, data pivots, and summary tables to help categorize content within an assessment. Users can filter list pages by these labels and view all items with a given label on one page. This allows for better organization of various content within an assessment on HAWC. This is especially useful for large assessments with lots of visualizations in different health domains, for example.

Two new visualizations were introduced this quarter: PRISMA flow diagrams and literature Venn diagrams. PRISMA flow diagrams are available as a new visual type used for conveying an assessment's systematic review process. They were inspired by a recent paper by Walker et al. 2024. In the literature review module, users can compare tagged references using our new Venn diagram visualization option (alongside the existing tag tree visualization).

Export changes include a new style of reference export, a BMDS-friendly animal endpoint export, and exports for User Defined Fields. Existing imports have been improved, with new filters, new data fields, and drastic speed improvements for large animal exports.

A total of 51 pull requests were merged from 2024-07-10 to 2024-10-24. The complete changelog is described in summary below; the emoji legend describes how change bullets are categorized:

Global Updates:

Study Evaluation / Risk of Bias:

  • 🐞 Fix for conflict between SET url setting and url validation by @rabstejnek in #1111 (September 26)

Assessment Updates:

User Interface (UI) Updates:

Literature Review:

Animal Data Extraction:

Epidemiology Data Extraction:

Visualizations:

Developer/Security updates:

  • 🛠️ Update Ruff Rules by @caseyhans in #1080 (August 28)
  • 🛠️ remove user_can_edit_assessment by @caseyhans in #1071 (August 28)
  • 🛠️ Add scientific notation filter by @munnsmunns in #1077 (August 28)
  • 🛠️ fix create_superuser command by @caseyhans in #1084 (August 28)
  • 🛠️ Alert tag rewrite by @caseyhans in #1074 (August 29)
  • 🧪 Add path for new HERO API in HERO-ID based import by @munnsmunns in #1070 (September 04)
  • 🛠️ make all hawc imports relative instead of absolute by @shapiromatron in #1095 (September 05)
  • 🛠️ Add coverage artifact by @shapiromatron in #1106 (September 18)
  • 🛠️ add a user agent for external requests by @shapiromatron in #1096 (September 18)
  • 🛠️ Change pandas pattern for empty dataframe by @caseyhans in #1109 (September 26)
  • 🛠️ Add minimum Python coverage number by @shapiromatron in #1120 (October 16)
  • 🐞 Fix display choices for swagger display /admin/api/swagger/ by @shapiromatron in #1121 (October 22)
  • 🔒 Bump webpack from 5.88.2 to 5.94.0 in /frontend in #1101 (September 10)
  • 🔒 update npm express by @shapiromatron in #1102 (September 10)
  • 🔒 Bump micromatch from 4.0.5 to 4.0.8 in /frontend in #1103 (September 10)
  • 🔒 Bump wagtail from 6.1.2 to 6.1.3 in #1068 (July 12)
  • 🔒 update django from 5.0.6 to 5.0.7 by @shapiromatron in #1069 (July 12)
  • 🔒 Upgrade django to 5.1 by @munnsmunns in #1083 (August 28)

Emoji legend:

  • 🚀 Feature - this is a new feature or improvement to the existing application
  • 🧪 In Progress - a feature actively under development; may not yet be available in production
  • 🐞 Bug - fixed an error in the current software
  • 📄 Documentation - updated user/developer documentation
  • 🔒 Security Updates - security related updates
  • 🛠️ Developer Tools - updated internal tooling for a better developer experience

Contributors

shapiromatron, munnsmunns, and 3 other contributors
Assets 2
Loading

2024 - Q2

12 Jul 13:44
@shapiromatron shapiromatron
2024-Q2
e115598
Compare
Choose a tag to compare
2024 - Q2

July 11, 2024

This release is a collection of smaller user-facing updates to improve new features we've added over the last year, and larger infrastructure changes required to maintain the overall healthiness of the project.

For user-facing updates, we've continued to expand the functionality of User Defined Forms (UDF) released last quarter, adding a reference UDF display, data export downloads, and fixing a few bugs identified during testing. We've made some minor visualization updates as well - adding study evaluation data to epiv2 data pivot, and allowing users to determine which column to use to generate counts in exploratory heatmaps.

Our infrastructure for generating data exports was overhauled for speed and code readability; this was an 8-month effort and thousands of manual visualization inspections to ensure backwards compatibility with our previous approach. We've also expanded our test coverage to 92%, writing dozens of new tests to improve the stability and quality guarantees of HAWC.

A total of 50 updates were made from 2024-04-16 to 2024-07-11. The complete changelog is described in summary below; see the emoji legend which describes how changes are categorized:

Global Changes:

Assessment Updates:

Literature Screening:

Data Tables and Visualizations:

User Defined Fields:

Ecological Data Extraction:

Bugfixes:

Developer/Security updates:

  • 🛠️ New permission classes by @shapiromatron in #1016 (April 19)
  • 🛠️ Remove react-datepicker by @munnsmunns in #1017 (May 13)
  • 🛠️ Update to PostgreSQL v 16 by @shapiromatron in #1029 (May 14)
  • 🛠️ Remove uswds by @shapiromatron in #1031 (May 14)
  • 🛠️ Remove requirements.txt by @shapiromatron in #1033 (June 05)
  • 🛠️ Refactor HTMX Viewsets by @shapiromatron in #1012 (June 14)
  • 🛠️ Developer quality of life improvements by @shapiromatron in #1040 (June 15)
  • 🛠️ python package updates by @shapiromatron in #1046 (June 17)
  • 🛠️ Replace and with bs4 tags by @munnsmunns in #1049 (June 27)
  • 🛠️ Improve test coverage by @munnsmunns in #1051 (June 27)
  • 🛠️ Add dev-header to full width of body instead of as a tab by @shapiromatron in #1057 (June 28)
  • 🛠️ Add form helpers instead of repeating code by @shapiromatron in #1053 (June 28)
  • 🛠️ Remove supported browser check - no longer relevant by @shapiromatron in #1059 (July 01)
  • 🛠️ Add more tests by @shapiromatron in #1062 (July 01)
  • 🔒 Update gunicorn by @shapiromatron (April 24)
  • 🔒 Bump @babel/traverse from 7.22.8 to 7.24.5 in /frontend in #1025 (May 14)
  • 🔒 Bump webpack-dev-middleware from 6.1.1 to 6.1.2 in /frontend in #1030 (May 14)
  • 🔒 Bump glob-parent from 3.1.0 to 5.1.2 in /frontend in #1032 (May 14)
  • 🔒 Update github action versions by @shapiromatron in #1024 (May 15)
  • 🔒 Bump jinja2 from 3.1.2 to 3.1.4 in #1042 (June 05)
  • 🔒 Bump requests from 2.31.0 to 2.32.2 in #1041 (June 05)
  • 🔒 Bump express from 4.18.2 to 4.19.2 in /frontend in #1043 (June 15)
  • 🔒 Bump postcss from 8.4.27 to 8.4.38 in /frontend in #1044 (June 15)
  • 🔒 Bump braces from 3.0.2 to 3.0.3 in /frontend in #1045 (June 15)
  • 🔒 Bump es5-ext from 0.10.62 to 0.10.64 in /frontend in #1047 (June 17)
  • 🔒 Bump djangorestframework from 3.15.1 to 3.15.2 in #1063 (June 28)
  • 🔒 Switch from python:3.12-slim-bullseye to centos:stream9-minimal by @shapiromatron in #1065 (July 02)

Emoji legend:

  • 🚀 Feature - this is a new feature or improvement to the existing application
  • 🧪 In Progress - a feature actively under development; may not yet be available in production
  • 🐞 Bug - fixed an error in the current software
  • 📄 Documentation - updated user/developer documentation
  • 🔒 Security Updates - security related updates
  • 🛠️ Developer Tools - updated internal tooling for a better developer experience

Contributors

shapiromatron, munnsmunns, and 2 other contributors
Assets 4
Loading

2024 - Q1

15 Apr 23:01
@munnsmunns munnsmunns
2024-Q1
e0746b1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2024 - Q1

April 15, 2024

This release includes a new feature we've been developing for the last six months or so - User Defined Forms (UDF). HAWC has always had a fixed schema for data extraction; this was an intentional decision because we wanted to be able to visualize content and perform standardized data analysis across assessments. However, it came at the expense of flexibility. Adding UDFs will allow users to augment existing forms with custom fields for their assessment. Right now, we've added UDFs to literature tags, studies, and bioassay endpoints, but we expect to add more UDF options to other items in the future. We plan on adding UDF data exports and integration with existing visuals in the next release. Documentation is limited, but please reach out with any questions or comments - we'd love to hear your feedback!

Other notable updates include new options in Data Pivot visualizations (calculated columns, columns to the right of the plot), updated task table filtering, updated assessment analytics for epidemiology and study evaluation domains, and many minor user-interface improvements throughout HAWC based on user feedback.

A total of 42 pull requests were merged into HAWC from 2024-02-02 to 2024-04-15. The complete changelog is described in summary below; see the emoji legend which describes how changes are categorized:

Global Changes:

Assessment:

Literature Screening and Tagging:

Data Extraction:

Visualizations:

Developer/Security updates:

Emoji legend:

  • 🚀 Feature - this is a new feature or improvement to the existing application
  • 🧪 In Progress - a feature actively under development; may not yet be available in production
  • 🐞 Bug - fixed an error in the current software
  • 📄 Documentation - updated user/developer documentation
  • 🔒 Security Updates - security related updates
  • 🛠️ Developer Tools - updated internal tooling for a better developer experience

Contributors

shapiromatron, DookTibs, and 4 other contributors
Assets 2
Loading

2023 - Q4

01 Feb 19:13
@shapiromatron shapiromatron
2023-Q4
c4020ae
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2023 - Q4

February 1, 2024

We're proud to release 2023.Q4. This release makes a number of improvements to our literature screening workflow, an assessment dashboard to highlight key analytics for your project, and a long-needed refresh for improving flexibility and performance of our data exports used in visualizations. Finally, we added a new visual type - an image upload, so we can more easily allow for other visualization types for which HAWC may not be capable of creating.

In the last six months we've released a number of updates in the literature screening module, and we've had some real-world applications which helped identify a few improvements to make the module more intuitive. The first is a workflow system, which allows users to set rules to create workflows for literature (for example, a title/abstract review, a full-text review, etc.). In addition, we added a bulk merge feature to conflict resolutions - this should make it easier to rapidly resolve conflicts, or accept all tags even if only one user has reviewed them (which can be helpful in some situations). We hope you'll find these features useful!

We have a few features that we're close to releasing that we made great progress on with this release. The first is an internal documentation site, so we can better add content for how to use HAWC and keep it up to date for our user base. The second is dramatic performance improvements to our exports, which should make them more flexible, and easier to add to new parts of our app (along with new visualization types). The third are user defined fields, which should be added in the future to our literature screening workflow as well as key data extraction models. We hope that these features will be available in our next release.

A total of 50 pull requests were merged into HAWC from 2023-10-07 to 2024-01-18. The complete changelog is described in summary below; see the emoji legend which describes how changes are categorized:

  • Global Updates:

    • 🚀 Allow Sharepoint (sharepoint.com), Zenodo (zenodo.org), and Canadian Government (canada.ca) hyperlinks #924 #949
    • 🚀 Add new option for users to show HAWC in debug mode #951
    • 📄 Add a documentation application to add internal documentation for HAWC editors #950
    • 🛠️ Update EPA flavor of application to use 2023 style guide #945 #960

  • Assessment:

    • 🚀 Create new assessment-level analytics pages #917 #927
    • 🚀 Add study-level (one row per study) data summaries to downloads page #944
    • 🚀 Add new options for downloading published/unpublished data to assessment downloads #956
    • 🐞 Fix assessment lists from showing duplicates #930

  • Literature Review:

    • 🚀 Add literature screening workflows #952
    • 🚀 Add bulk merge tag conflicts, and more filters for conflict resolution #919 #932
    • 🚀 Add additional duplicate reference check on imports #965
    • 🚀 Improve layout for references #933 #953

  • Study Evaluation / Risk of Bias:

    • 🐞 Restore user study evaluation review dashboard #964
    • 🐞 Fix bug in creating scores via API #936
    • 🛠️ Improve speed and standardize all study evaluation APIs #934
    • 🧪 Rewrite export engine for improved performance #921

  • Data Extraction: Animal Bioassay:

    • 🚀 Update animal dose response plots #780
    • 🚀 add Effect Tag creation (Bioassay, Epi, etc.) to API #937

  • Data Extraction: Epidemiology:

    • 🚀 Add client for modifying epi v2 data #888
    • 🧪 Rewrite exports for improved performance #911 #922

  • Data Extraction: In Vitro:

    • 🧪 Rewrite exports for improved performance #958

  • Visualizations:

    • 🚀 Add new visual type - user uploaded image #941
    • 🚀 Overhaul data prefilters and add new prefilter options #939 #962
    • 🚀 Automate downloading visuals using the client #810
    • 🐞 Use preferred assessment naming convention for risk of bias/study evaluation in visuals #946

  • Developer/Security updates:

    • 🔒 Update to Python 3.12 #967
    • 🔒 Update to Django 5 #959
    • 🔒 Update python packages to most recent versions #928 #929
    • 🔒 Update pydantic to v2 #948
    • 🛠️ Improve view/form test code coverage #905
    • 🛠️ Remove/refactor unused and duplicative code #799 #925 #926 #931
    • 🛠️ Standardize template tags for easier use #920 #923
    • 🛠️ Standardize assessment level APIs #935
    • 🛠️ Standardize caching patterns #940
    • 🛠️ Replace black with ruff formatter #942
    • 🐞 Remove broken Zenodo badge from readme #954
    • 🐞 Fix flaky tests to improve code stability #943 #968

Emoji legend:

  • 🚀 Feature - this is a new feature or improvement to the existing application
  • 🧪 In Progress - a feature actively under development; may not yet be available in production
  • 🐞 Bug - fixed an error in the current software
  • 📄 Documentation - updated user/developer documentation
  • 🔒 Security Updates - security related updates
  • 🛠️ Developer Tools - updated internal tooling for a better developer experience
Assets 2
Loading

2023 - Q3

17 Oct 16:12
@dannypeterson dannypeterson
2023-Q3
0aac8d2
Compare
Choose a tag to compare
2023 - Q3

October 17th, 2023

We're proud to release 2023.Q3! We've made a number of quality-of-life improvements this quarter based on user feedback. First, we've refined searching and filtering during literature screening and have updated how references are displayed. Second, we've added a new data export for study evaluation heatmaps; this export is now available on all existing visuals. Finally, we've added some new features to data pivots and exploratory heatmaps to enable more customization of their look and feel.

There's more work that we've been doing behind the scenes that we hope will allow us to add some substantial new features in future release. First, we've rewritten how data can be filtered for visualizations so we can filter more quickly and flexibly. This should allow us to add more visuals and filtering options in the future. Second, we're adding User Defined Forms (UDF), so that users can augment the existing tagging and extractions in HAWC with additional information that may be required for an assessment. This has long been requested over the years and we're excited to break ground on this work.

The complete changelog is described in summary below; see the emoji legend which describes how changes are categorized:

  • User Interface (UI) Updates:

    • 🚀 Added Unpublished badge to studies, visuals, etc to better show team members the status of content under development #860
    • 🚀 Text areas can be resized globally on all text boxes #883
    • 🧪 Download filtered data for outcome lists (animal, epidemiology, etc.) #895
    • 🐞 Converted assessment Log search form to new inline filter form layout #872

  • Literature Review:

    • 🚀 Added authors and year filter to literature review reference search #886
    • 🚀 Add better full-text search capabilities #877
    • 🚀 UI updates for References - simplify layout for detail and list reference views #878
    • 🚀 Added a new "accept all" to accept all proposed tags during conflict resolution #908
    • 🚀 Improve validation for full text URL uploads and display errors to users #887
    • 🐞 Add db restraint to prevent duplicate (reference, user) pairs in User tags #890
    • 🐞 Fix bug in keyword highlighting for literature tagging with special characters #878

  • Study Evaluation / Risk of Bias:

    • 🐞 Fix duplicative studies shown in ROB assignment views #885

  • Data Extraction: Animal:

    • 🚀 Update experiment comments label on detail view to be consistent with form #904
    • 🚀 Animal summary endpoint exports now contain sample size 'N' #898
    • 🚀 Calculate percent change relative to control calculated for dichotomous when possible #909 #915
    • 🐞 Fixed bug where users couldn't set or view NOEL/LOEL for endpoints where data was not extracted #907

  • Data Extraction: Epidemiology:

    • 🚀 Added adverse direction field to epidemiological study population Data Extractions #902
    • 🚀 Rewrite export view for performance #870
    • 🚀 Update API to filter study designs by study identifier #918

  • Visualizations:

    • 🚀 Study Evaluation Heatmaps - download data in tabular format using the actions button #892
    • 🚀 Exploratory Heatmap - toggle display of counts and colors #916
    • 🚀 Data Pivot - show arrows for confidence intervals when out of domain #874
    • 🚀 Data Pivot - allow more flexibility with background colors and gridlines #874
    • 🚀 Data Pivot - add more flexibility and precision with setting x-axis domains and display #874
    • 🚀 Data Pivot - full read/write control via API #896
    • 🚀 Visuals - full read/write control via API #896
    • 🐞 Data pivot file upload error #894
    • 🐞 Fix typos in visual list text #893
    • 🐞 Fix deprecated Endpoint Aggregation visual form and remove dependencies #883

  • User Defined Forms (UDF):

    • 🧪 Allow users to define their own forms #891 #882 #897
    • 🧪 Bind UDF to data extraction components in an assessment #899
    • 🧪 Bind UDF to tags in an assessment #899

  • Developer/Security updates:

    • 🛠️ Installed django-browser-reload package for automated browser reloading in debug #881
    • 🛠️ Remove dead code #900
    • 🛠️ Convert text fields to JSON fields for visuals and data pivots #901
    • 🛠️ Refactor prefilters for visuals and datapivots #876
    • 🛠️ Standardize actions button menus #860
    • 🛠️ Added utility method for generating tabular flat exports to remove duplicate code #873
    • 🔒 Add setting to disable login attempts on some domains #879
    • 🔒 Fixes from security scans #875 #884 #889
    • 🔒 Updated Django to latest release #903
    • 🔒 Updated docker container versions #910

Emoji legend:

  • 🚀 Feature - this is a new feature or improvement to the existing application
  • 🧪 In Progress - a feature actively under development; may not yet be available in production
  • 🐞 Bug - fixed an error in the current software
  • 📄 Documentation - updated user/developer documentation
  • 🔒 Security Updates - security related updates
  • 🛠️ Developer Tools - updated internal tooling for a better developer experience
Assets 2
Loading

2023-Q2

09 Aug 18:56
@dannypeterson dannypeterson
2023-Q2
51177c6
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2023-Q2

August 9th, 2023

We're proud to release 2023-Q2. Over the last few quarters, we've been internally developing new data extraction and visualization components in partnership with our scientists, and with this release, those features are made generally available. Specifically:

  • Last year we released new data extraction modules for Epidemiology. This update integrates data extraction with existing data visualizations in HAWC, such as exploratory heatmaps and data pivots.
  • We've released ecological data extraction, which, like epidemiology, is now fully integrated with our exploratory heatmap and data pivot visualizations.
  • We've added an assessment metadata module to add high level project data to assessments (assessment type, current status, related documentation), as well as adding assessment values, if a HAWC assessment results in quantitative values as a conclusion.

In addition to these changes, we've overhauled searching and filtering throughout HAWC; filters should be faster and more intuitive. We added new customizations to assessments to show/hide items on the sidebar, to make it easier to only have links throughout an assessment to pages which have content.

The complete changelog is described below (features marked with the lab icon 🧪 indicate they're not yet available and are behind feature flags):

New Features:

  • General improvements
    • Redesigned search/filters to include an inline search bar and expandable 'advanced search' filter form #817 #838
    • Improve styling for Excel-based tabular exports #816
    • 🧪 New REST API endpoints for searching assessments by chemical and reference (admin only) #826
  • Literature screening
    • Add wildcard keyword searching for literature screening #833
    • Remove ability to change source on already executed literature import #849
    • Steer users to importing references from PubMed/HERO instead of manual extraction #819
  • Assessment configuration
    • Assessment sidebar customization allows users to show/hide all sidebar links #827
    • Restyled Assessment Downloads page which includes multiple different export types #807
    • Improve flexibility for searching and filtering assessments #839 #857
  • Study Evaluation
    • Add new study evaluation judgment choice: Minor/Critical concerns (sensitivity) #848
    • Improve display of study evaluation criteria when empty #821
  • Data Extraction: Animal Bioassay
    • Update export for bioassay data to include diagnostics field #850
    • Added Pubertal and Juvenile to animal constants #864
  • Data Extraction: Epidemiology
    • Added the ability to create data pivots and exploratory heatmaps using extracted data #832 #869
    • Add REST API endpoints for epiv2 data extraction (create, read, update, delete) #820 #866
    • Added bulk-cleanup features to clean manually extracted data #855
  • Data Extraction: Ecology
    • Added ecological data pivots and exploratory heatmap visualizations using extracted data #845
    • Make ecological data extraction generally available #868
    • Added schema to developer and API documentation #841 #866
  • Assessment Values and Details
    • Additional revisions to assessment values and details per scientist feedback #792 #831
    • New REST API endpoints for CRUD operations of assessment values and details #858 #862
    • Make values and details generally available #868
  • Summary Tables and Visualizations
    • Streamline exploratory heatmap user interface #846
    • Update Evidence Profile Table language per scientist recommendation #837
    • 🧪 New visualization types using plotly (admin only) #804
  • Assorted User Requests
    • Assorted UI improvements #818 #823 #824 #822 #863
    • Updated notifications to streamline interactions with pages #819 #821 #844 #843 #855
    • Rebuild mgmt views to use templates instead of JS #808
    • Update CCTE Dashboard API to newly release version for chemical information #828
Assets 2
Loading

2023-Q1

13 Apr 14:08
@shapiromatron shapiromatron
2023-Q1
42b2cc3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2023-Q1

April 13th, 2023

We're proud to release 2023.Q1. The primary feature in this release is the updated literature screening module with optional conflict resolution of tags. This features allows dual-screening of references in HAWC, and we hope this expands the types of systematic review methodologies that can be conducted from within HAWC.

In addition, we've made assorted quality of life updates throughout the application - assessment data downloads should be better organized and with more information about the underlying reference, study evaluation metrics and domains are now organized by health domain, and summary tables can now be set to filter and paginate for longer tables.

Finally, we're enabling a feature-flag system for future development to make it easier to add features that we can progressively roll out to users. We hope that this approach makes it easier for developers to update code with less context-switching, which ultimately should make us more efficient.

The complete changelog is described below (features marked 🧪 indicate they're not yet generally available and are behind feature flags):

New Features:

  • Literature Screening #756 #774 #752 #794 #789

    • release literature tagging with conflict resolution
    • rewrote literature summary pages and add new summary stats
    • added reference counts to all pages where tag lists are displayed
    • add user tag exports and filterable web views

  • Study Evaluation/Risk of Bias

    • update the study evaluation domain/metric view to show questions by study type #790
    • update study evaluation questions from NTP/OHAT and EPA/IRIS to current approaches; created tools to automate syncing in the future #771
    • add study evaluation metric key field for easier copying of metrics across assessments #771

  • Data Extraction

    • add API endpoint for epiv2 metadata settings, useful for automated machine extractions #766
    • updated animal data extraction help text #764
    • 🧪 release candidate for ecological data extraction #740 #746 #783 #784 #791
      • revised browsing of controlled vocabulary; added excel-based exports
      • add data export for extracted data
    • 🧪 release updated version of BMDS execution using 2022 version #788

  • Summary tables

    • add searchable, filterable, paginated generic summary tables #786
    • add a caption to summary tables and their corresponding exports #772
    • swap (again) columns in EPT tables #773

  • Summary visualizations

    • No changes this quarter

  • General Improvements

    • simplify assessment downloads page to only show exports with data #784
    • add reference IDs (PubMed, HERO, DOI) to common assessment download exports #745
    • 🧪 updates for assessment values #759
      • added admin API to retrieve a report of all values across assessments

  • Authentication/authorization

  • Client updates #765 #803

    • release version 2023.2
    • Added client.riskofbias.compare_metrics - align metrics when copying across assessments via fuzzy text matching
    • Added client.summary.datapivot_list - retrieve all data pivots for an assessment
    • Added client.summary.table_list - retrieve all tables for an assessment
    • Added login parameter to client.summary.set_authentication_token, to setup a client which can be used to browse the HAWC website
    • Added client.lit.reference_user_tags - retrieves all user tag for all references for an assessment.

  • Administrative

    • (optionally) add sentry #768
    • added assessment admin notes, to capture details on individual assessments #763

Bug/regressions fixed:

  • fix bug in data pivot initial settings #770
  • fix bug in hawc client and API sessions #776
  • fix bug in references "manually added" layout in html #779
  • fix bug in user ordering for study evaluation pages #781
  • remove duplicate study evaluation rows #782

Software upgrades/security/stability:

  • update python, javascript, and CSS dependencies #787 #777
  • remove old unused code #767
  • remove redundant views, old javascript patterns, and catch errors with invalid data inputs #803

Developer experience:

  • simplify django permissions for views and APIs #732 #801 #803
    • the old approach was confusing and difficult to infer or test
  • make browsable API available for PandasRenderers API views #800
    • this will be useful to debug query counts in django debug toolbar
  • added plotly template tag for easier plotting on django templates #793
  • added feature flags for features still undergoing internal review #802 #788
    • this makes it easier for develop to develop this features, but means we'll need to make sure code that we don't yet available to standard deployments is appropriately captured behind feature flags
    • in staging environments, we will scan with all features enabled, to ensure that all possible routes are undergoing security review
Assets 2
Loading

2022-Q4

09 Jan 19:46
@caseyhans caseyhans
2022-Q4
ff70df3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2022-Q4

January 5th, 2023

The bulk of development this quarter was focused on new conflict resolution workflows for literature screening. If activated for an assessment, multiple users are required to select tags to a reference before they are applied. If two users tag a reference identically, the tags are marked as "resolved" and the reference advances to the next stage of screening. If two users tag a reference differently, then the tags are marked as conflicting, and the reference is added to the conflict resolution view. Note that existing assessments will not have this feature enabled, and tagging will work as it has in the past. This work is still not yet live, and is undergoing testing by our users (indicated by 🧪).

Another large-scale update was migrating our search/filter capabilities throughout HAWC to use django-filters instead of selectable, which is no longer maintained. Performance should be greatly improved with this migration, resulting in faster queries and page loads.

The complete changelog is described below:

New Features:

  • General Improvements

  • Literature Screening

    • Add PubMed and DOI identifiers on HERO import #755
    • Handle and report 500 errors from HERO requests #705
    • Hide RIS import button in some HAWC deployments #705
    • 🧪 Conflict resolution to our literature screening/tagging workflow #739
      • Create wireframes, new models, new view stubs, and new backend logic as a foundation for further conflict resolution development #708 #721 #716 #735 #741 #750
      • Add new view for users to identify and resolve tag conflicts #734 #749
      • Reorganize screening page layout to account for user feedback and new conflict resolution #714, #731, #738, #747

  • Risk of Bias/Study Evaluation

    • Update API to allow project managers to copy study evaluations across assessments #706
    • Make cloning risk of bias data across assessments easier using the HAWC client #724

  • Data Extraction

    • Bioassay
      • prepare for updating BMDS in HAWC; disable execution of current versions #711
    • Epidemiology (v1)
      • Result group names now include comparison set name on data entry forms #748

  • Summary tables

    • Moved factor column locations in EPTs #744
    • Add client API method to fetch all summary tables #717

  • Summary visualizations

    • Add Tableau data filters and modernize API #758 #761
    • Enable JPEG exports of visualizations #712
    • Improve to tagtree hover/help text #709
    • Only show units available in an assessment when configuring a data pivot #709
    • Fix data pivot plot layout to always show lower-bound on plot instead of on axis #710
    • Add client API method to fetch all summary visualizations #717

  • Authentication/authorization

    • Add deployment feature flag to prevent account creation by users #718
    • Make session cookies configurable #754
    • Add token-based authentication to access django views #762

  • Administrative

    • Update docs to show latest database schema #720

Bug/regressions fixed:

  • Fixed an edge case where dose-group counts in data pivots did not add up correctly #713
  • Fix case where user could unintentionally delete tagged references #753
  • Change test to use regex for content type ID stability #725
  • Various style fixes #726

Software upgrades/security/stability:

  • Replace type hints from typing module with built in object types #723
  • Standardize buttons for easier integration testing #743
  • Add eslint plugin for sorting imports #728
  • Reduce scope of API responses in production #715
  • Update github actions #730
  • Rename urls that use /new/ to use /create/ for consistency across HAWC #707
  • Remove dead code #722 #727
Assets 2
Loading

2022-Q3

06 Oct 21:37
@rabstejnek rabstejnek
2022-Q3
713eda6
Compare
Choose a tag to compare
2022-Q3

October 6, 2022

Our major update this quarter includes deployment of a new extraction workflow and schema for epidemiology data. Named epiv2 in the software; all new assessments and existing assessments without epidemiology data extraction have been migrated to the new schema; assessments with the old schema, epiv1, are unchanged. The updated schema simplifies the old schema and user interface for data extraction and makes it easier to extract data at a high level (for systematic evidence maps, SEM), as well as a more detailed granular level, depending on the nature of the assessment. This release includes all the required components for data extraction; future releases will include the components needed to build summary tables and visualizations - in the meantime, try out the extraction forms and let us know what you think!

In addition, we made the following larger changes:

  • We updated assessment audit-logs support to create new exports and improving our current logs to show more information
  • We updated and standardized many of the forms that we use in HAWC. We've rewritten the text input fields, as well as the autocomplete/autosuggest fields using more modern, stable, and secure technologies. We hope this makes for a more seamless experience for users
  • We enhanced summary visuals and tables to reflect the needs of users integrating the visuals into their projects - we hope these updates make the current visuals more flexible

Two other major changes have been released in our next branch this quarter. These features are available under deployment feature flags and are undergoing some testing by users prior to merging into our main release branch; they are indicated below by the 🧪 emoji. Some production deployments may not have these features available; we hope to have them live soon:

  • 🧪 Capturing additional assessment level metadata on a HAWC assessment and quantitative values for a final conclusion. In larger organizations, HAWC is frequently one of many tools and resources used for conducting a systematic review and health assessment. Therefore, we've added some new high-level assessment level fields to capture information regarding the larger project (class of project, relevant documents or URLs, QA procedure and methodology), and fields for capturing health values of interest (points of departure, reference values, etc). #674
  • 🧪 Capturing ecological data of interest in a health assessment. Some health assessments include impacts on the environment and ecological health as well as human health. We've created a new data extraction module to extract fields in this domain as well, when appropriate. #634

The complete changelog is described below:

New features:

  • Overall Assessment Settings

    • Added downloads to assessment audit logs; expanded log views show additional details #683
    • 🧪 Created new fields for assessment-level metadata and assessment-level values #674

  • Literature Screening

    • Standardize DOIs in HAWC (lowercase; improved for deduplication) #666
    • Expand RIS import validation to catch more errors prior to import #647
    • Added additional warnings to prevent accidental search deletions #662
    • Update reference ID export to include references with no external IDs #659

  • Data extraction

    • Study
      • User-interface updates for high-level study information #656 #663
    • Animal
      • Added treatment related effect to animal endpoint groups #681
    • Epidemiology
      • Finalized an updated epiv2 data extraction module; basic Excel exports #642 #673 #687
      • Added "Ecological" and "Other" as choices for epiv1 study design; consistent with epiv2 #702
    • Ecology
      • 🧪 Added forms #634

  • Risk of Bias/Study evaluation

    • Expand filters for risk of bias assignments to include user #665
    • Updated default study evaluation metrics and domains #682

  • Summary tables and visualizations

    • Literature tagtrees
      • Added a legend #684
      • Added an option to show or hide reference counts, hiding counts is useful for documenting the approach in the planning and protocol phase #684
      • Added the ability to drag and drop nodes for complex tree layouts #684
      • Fixed bug where reference lists for a node were improperly filtered #654
    • Data Pivots
      • Updated forest plots to include treatment related effects in addition to statistical significance; can be used for conditional formatting #681
      • Updated barcharts to handle standard deviation for error bars (previously only confidence intervals were handled correctly) #651 #697
      • Added numeric duration exposure, useful for sorting #651
    • Study Evaluation Heatmaps/Barcharts
      • Improved default positioning of legend #685
    • Evidence Profile Tables
      • Improvements to evidence profile tables and reporting #675 #657
    • Study Evaluation Tables
      • Update Study Evaluation Table (SET) data entry based for easier data entry #678 #653

  • General improvements

    • Updated the EPA header and footer to match the new style guidelines #693 #703
    • Fix mobile rendering of website (sidebar collapse) #672 #700 #664
    • Modernized HTML text data entry fields to improve consistency and stability #655 #699
    • Modernized text autocomplete data entry fields to improve consistency and stability #667 #698 #701

  • Administrative

    • Updates to diagnostic API endpoints #668 #669

Bug/regressions fixed:

  • Fixed bug in downloading SVG exports of visuals #677
  • Fixed bug which prevented updating assessment file attachments #695
  • Better handle improper Unicode in Excel exports #652 #670
  • Report an error on extremely large exploratory heatmaps instead of crashing the browser #671
  • Improve validation error messages for users #658 #643
  • Fix assorted errors #646 #696 #679 #676 #692

Software upgrades/security/stability:

  • Quality of life improvements to developer experience #689 #660
  • Rewrote integration tests using Playwright instead of Selenium #650
  • Rewrote documentation using more user-friendly markdown instead of sphinx #680
  • Fix deployment on servers with FIPS cryptography enforcement (RHEL8) #688
  • Update HAWC configuration to reasonable and secure defaults #649 #691
Assets 2
Loading

2022-Q2

12 Jul 22:15
@shapiromatron shapiromatron
2022-Q2
d094598
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2022-Q2

July 12, 2022

Updates were light this quarter; we were focused on other projects. One major change is rewriting how rasterized visuals are created; we're using native tooling in the browser now instead of generating server side; this should result in increased performance and accuracy to the rendered visual. #630

The complete changelog is described below:

  • Visualizations:
    • Client side rasterization of SVG images #630
  • Literature screening:
    • Fixed literature tagtree overflow #638
    • Delete untagged references if search scope decreases #639
  • Study evaluations:
    • Add a "Save and continue editing" button for study evaluations #645
    • Standardize "completion" on frontend/backend for study evaluations #648
  • Client:
    • Update hawc client (version 2022.7) with new commands #573
  • Admin:
    • New admin dashboard - changes per day #636
    • New configurable hawc admin root #637
    • Improved admin performance #638
  • Developer updates:
    • Database commands for windows environments #640
    • Standardize fake data names #641
  • Current in development (behind feature flags):
    • ecology data extractions #620
    • epidemiology version 2 updates #632
Assets 2
Loading