diff --git a/.Rbuildignore b/.Rbuildignore
index 30a1020c..53aaa071 100644
--- a/.Rbuildignore
+++ b/.Rbuildignore
@@ -10,3 +10,6 @@ google-analytics.js
droptoken.rds
^cran-comments\.md$
^CRAN-RELEASE$
+^_pkgdown\.yml$
+^docs$
+^pkgdown$
diff --git a/.github/workflows/pkgdown.yaml b/.github/workflows/pkgdown.yaml
new file mode 100644
index 00000000..087f0b05
--- /dev/null
+++ b/.github/workflows/pkgdown.yaml
@@ -0,0 +1,46 @@
+# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
+# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
+on:
+ push:
+ branches: [main, master]
+ pull_request:
+ branches: [main, master]
+ release:
+ types: [published]
+ workflow_dispatch:
+
+name: pkgdown
+
+jobs:
+ pkgdown:
+ runs-on: ubuntu-latest
+ # Only restrict concurrency for non-PR jobs
+ concurrency:
+ group: pkgdown-${{ github.event_name != 'pull_request' || github.run_id }}
+ env:
+ GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
+ steps:
+ - uses: actions/checkout@v3
+
+ - uses: r-lib/actions/setup-pandoc@v2
+
+ - uses: r-lib/actions/setup-r@v2
+ with:
+ use-public-rspm: true
+
+ - uses: r-lib/actions/setup-r-dependencies@v2
+ with:
+ extra-packages: any::pkgdown, local::.
+ needs: website
+
+ - name: Build site
+ run: pkgdown::build_site_github_pages(new_process = FALSE, install = FALSE)
+ shell: Rscript {0}
+
+ - name: Deploy to GitHub pages 🚀
+ if: github.event_name != 'pull_request'
+ uses: JamesIves/github-pages-deploy-action@v4.4.1
+ with:
+ clean: false
+ branch: gh-pages
+ folder: docs
diff --git a/.gitignore b/.gitignore
index e9c1d972..d720c4a9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -46,7 +46,7 @@ inst/doc
.Renviron
# pkgdown site
-docs/
+#docs/
# System files
*.ini
@@ -55,3 +55,5 @@ docs/
# ignore old datasets but don't remove
inst/shiny/
+#docs
+docs
diff --git a/DESCRIPTION b/DESCRIPTION
index e3d6dd03..8c942667 100644
--- a/DESCRIPTION
+++ b/DESCRIPTION
@@ -35,7 +35,8 @@ Description: Raman and (FT)IR spectral analysis tool for plastic particles and
Analyzed spectra can be shared with the Open Specy community. A Shiny app is
available via run_app() or online at
LICENSE.md
+ Creative Commons Corporation (“Creative Commons”) is not a law firm and does not provide legal services or legal advice. Distribution of Creative Commons public licenses does not create a lawyer-client or other relationship. Creative Commons makes its licenses and related information available on an “as-is” basis. Creative Commons gives no warranties regarding its licenses, any material licensed under their terms and conditions, or any related information. Creative Commons disclaims all liability for damages resulting from their use to the fullest extent possible.
+Creative Commons public licenses provide a standard set of terms and conditions that creators and other rights holders may use to share original works of authorship and other material subject to copyright and certain other rights specified in the public license below. The following considerations are for informational purposes only, are not exhaustive, and do not form part of our licenses.
+++Considerations for licensors: Our public licenses are intended for use by those authorized to give the public permission to use material in ways otherwise restricted by copyright and certain other rights. Our licenses are irrevocable. Licensors should read and understand the terms and conditions of the license they choose before applying it. Licensors should also secure all rights necessary before applying our licenses so that the public can reuse the material as expected. Licensors should clearly mark any material not subject to the license. This includes other CC- licensed material, or material used under an exception or limitation to copyright. More considerations for licensors: wiki.creativecommons.org/Considerations_for_licensors
+
++Considerations for the public: By using one of our public licenses, a licensor grants the public permission to use the licensed material under specified terms and conditions. If the licensor’s permission is not necessary for any reason–for example, because of any applicable exception or limitation to copyright–then that use is not regulated by the license. Our licenses grant only permissions under copyright and certain other rights that a licensor has authority to grant. Use of the licensed material may still be restricted for other reasons, including because others have copyright or other rights in the material. A licensor may make special requests, such as asking that all changes be marked or described. Although not required by our licenses, you are encouraged to respect those requests where reasonable. More considerations for the public: wiki.creativecommons.org/Considerations_for_licensees
+
By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution 4.0 International Public License (“Public License”). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions.
+Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image.
Adapter’s License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.
Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights.
Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements.
Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material.
Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License.
Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license.
Licensor means the individual(s) or entity(ies) granting rights under this Public License.
Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them.
Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world.
You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning.
License grant.
+Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to:
+reproduce and Share the Licensed Material, in whole or in part; and
produce, reproduce, and Share Adapted Material.
Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions.
Term. The term of this Public License is specified in Section 6(a).
Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material.
Downstream recipients.
+Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License.
No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material.
No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i).
Other rights.
+Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise.
Patent and trademark rights are not licensed under this Public License.
To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties.
Your exercise of the Licensed Rights is expressly made subject to the following conditions.
+Attribution.
+If You Share the Licensed Material (including in modified form), You must:
+retain the following if it is supplied by the Licensor with the Licensed Material:
+identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);
a copyright notice;
a notice that refers to this Public License;
a notice that refers to the disclaimer of warranties;
a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
indicate if You modified the Licensed Material and retain an indication of any previous modifications; and
indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License.
You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information.
If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable.
If You Share Adapted Material You produce, the Adapter’s License You apply must not prevent recipients of the Adapted Material from complying with this Public License.
Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material:
+for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database;
if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material; and
You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights.
+UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.
This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.
Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:
+automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.
+For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License.
Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed.
Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License.
For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License.
To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions.
No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.
Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.
++Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” The text of the Creative Commons public licenses is dedicated to the public domain under the CC0 Public Domain Dedication. Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at creativecommons.org/policies, Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses.
+
Creative Commons may be contacted at creativecommons.org.
+Jessica +Meyers, Jeremy Conkle, Win Cowger, Zacharias Steinmetz, Andrew Gray, +Chelsea Rochman, Sebastian Primpke, Jennifer Lynch, Hannah Hapich, +Hannah De Frond, Keenan Munno, Bridget O’Donnell
+ +vignettes/sop.Rmd
+ sop.Rmd
Open Specy Raman and (FT)IR spectral analysis tool for plastic
+particles and other environmental samples. Supported features include
+reading spectral data files (.asp, .csv, .jdx, .spc, .spa, .0),
+smoothing spectral intensities with smooth_intens()
,
+correcting background with subtr_bg()
, and identifying
+spectra using an onboard reference library. Analyzed spectra can be
+shared with the Open Specy community. A Shiny app is available via
+run_app()
or online at https://openanalysis.org/openspecy/.
This document outlines a common workflow for using Open Specy and +highlights some topics that users are often requesting a tutorial on. If +the document is followed sequentially from beginning to end, the user +will have a better understanding of every procedure involved in using +Open Specy as a tool for interpreting spectra. It takes approximately 45 +minutes to read through and follow along with this standard operating +procedure the first time. Afterward, knowledgeable users should be able +to thoroughly analyze individual spectra at an average speed of 1 +min-1.
+Here we will describe the basic workflow of spectral analysis in +maximum 3 paragraphs.
+To get started with the Open Specy user interface, access https://openanalysis.org/openspecy/ +or start the Shiny GUI directly from R typing
+
+run_app()
Then click the Upload File tab at the top of the +page.
+ +Accessibility is extremely important to us and we are making strives +to improve the accessibility of Open Specy for all spectroscopists. +Please reach out if you have ideas for improvement.
+We added a Google translate plugin to all pages in the app so that +you can easily translate the app. We know that not all languages will be +fully supported but we will continue to try and improve the +translations.
+ +If you don’t have your own data to use right away, that is ok. You +can download test data to try out the tool by clicking on the test data +button. A .csv file of HDPE Raman spectrum will download on your +computer. This file can also be used as a template for formatting .csv +data into an Open Specy accepted format. The following line of code does +the same:
+
+data("raman_hdpe")
Before uploading, indicate if you would like to share the uploaded +data or not using the slider. If selected, any data uploaded to the tool +will automatically be shared under CC-BY 4.0 +license and will be available for researchers and other ventures to +use to improve spectral analysis, build machine learning tools, etc. +Some users may choose not to share if they need to keep their data +private. If switched off, none of the uploaded data will be stored or +shared in Open Specy.
+Open Specy allows for upload of .csv, .asp, .jdx, .0, .spc, and .spa +files. .csv files should always load correctly but the other file types +are still in beta development, though most of the time these files work +perfectly. It is best practice to cross check files in the proprietary +software they came from and Open Specy before use in Open Specy. Due to +the complexity of these file types, we haven’t been able to make them +fully compatible yet. If your file is not working, please contact the +administrator and share the file so that we can get it to work.
+For the most consistent results, files should be converted to .csv +format before uploading to Open Specy. The specific steps to converting +your instrument’s native files to .csv can be found in its software +manual or you can check out Spectragryph, which +supports many spectral file conversions (see Mini Tutorial section: File +conversion in Spectragryph to Open Specy accepted format).
+If uploading a .csv file, label the column with the wavenumbers
+wavenumber
and name the column with the intensities
+intensity
.
wavenumber | +intensity | +
---|---|
301.040 | +26 | +
304.632 | +50 | +
308.221 | +48 | +
311.810 | +45 | +
315.398 | +46 | +
318.983 | +42 | +
Wavenumber units must be cm-1. Any other columns are not +used by the software. Always keep a copy of the original file before +alteration to preserve metadata and raw data for your records.
+To upload data, click Browse and choose one of your +files to upload, or drag and drop your file into the gray box. At this +time you can only upload one file at a time.
+Upon upload and throughout the analysis, intensity values are min-max +normalized (Equation 1).
+\[\frac{x - \mathrm{min}(x)}{\mathrm{min}(x) +- \mathrm{max}(x)}\] ++Equation 1: Max-Min Normalization +
+The following R functions from the Open Specy package will also read +in spectral data accordingly:
+ +After spectral data are uploaded, it will appear in the main window. +This plot is selectable, zoomable, and provides information on hover. +You can also save a .png file of the plot view using the camera icon at +the top right when you hover over the plot. This plot will change the +view based on updates from the Intensity Adjustment +selection.
+Open Specy assumes that intensity units are in absorbance units but +Open Specy can adjust reflectance or transmittance spectra to absorbance +units using this selection in the upload file tab. The transmittance +adjustment uses the \(\log_{10} 1/T\) +calculation which does not correct for system or particle +characteristics. The reflectance adjustment use the Kubelka-Munk +equation \(\frac{(1-R)^2}{2R}\). If +none is selected, Open Specy assumes that the uploaded data is an +absorbance spectrum and does not apply an adjustment.
+This is the respective R code:
+
+library(magrittr)
+
+raman_adj <- raman_hdpe %>%
+ adj_intens()
+
+head(raman_adj)
+#> wavenumber intensity
+#> 1: 301.040 0.00000000
+#> 2: 304.632 0.03037975
+#> 3: 308.221 0.02784810
+#> 4: 311.810 0.02405063
+#> 5: 315.398 0.02531646
+#> 6: 318.983 0.02025316
To share metadata about your spectrum, click the metadata input +button. When sharing data, please provide as much metadata as you can. +Metadata helps make shared data as useful as possible. Metadata inputs +each have examples provided in the input. The examples disappear when +the box is clicked and will not be saved if nothing is input by the +user. Mandatory inputs are marked with a red asterisk. If these inputs +are not filled, the data will be considered uninterpretable and will be +discarded. Inputs left blank will be left blank in the metadata sheet +and interpreted as “unknown” or “not applicable”. To share metadata, +click the share data button at the bottom of the metadata inputs.
+When the user clicks the Share Data button their +current uploaded data and metadata is sent to an open-access online repository.
+ +All inputs from the metadata (described below) are input to a +metadata sheet. The metadata sheet is given the same unique name as the +data, but it ends with “_form”. The exact same data is saved as would be +downloaded using the download data button (described below). All high +quality uploaded data with metadata will eventually be reviewed by +spectroscopy experts and added to the internal library if it passes +review. If multiple files are going to be uploaded with metadata, the +cells will also stay filled with the last input after the share data +button is clicked. Just upload the next dataset and change the metadata +inputs that are different. If more than 50 files will be shared at once, +you can contact the website administrator to get a bulk upload sheet for +more rapid upload.
+Type
+
+share_spec(raman_hdpe,
+ metadata = c(user_name = "Win Cowger",
+ contact_info = "wincowger@gmail.com",
+ spectrum_type = "Raman",
+ spectrum_identity = "HDPE")
+ )
to share your spectral data from the R console.
+After uploading data, you can preprocess the data using baseline +correction, smoothing, and range selection and save your preprocessed +data. Go to the Preprocess Spectrum tab to select your +parameters for processing the spectrum.
+The preprocess spectra plot shows the uploaded spectra in comparison +to the processed spectra that has been processed using the processing +inputs on the page. It will automatically update with any new slider +inputs. This allows the user to tune the inputs to optimize the signal +to noise ratio. The goal with preprocessing is to make peak regions have +high intensities and non-peak regions should have low intensities.
+When the slider is green for the tool type, that means that that tool +is being used to preprocess the spectrum. If the slider is clicked +blank, the cog button to the right will disappear to indicate that the +tool is no longer being used.
+ +If the cog button is clicked, any functions associated with that tool +will be displayed and can be manipulated to process the spectrum.
+The first step of the Open Specy preprocessing routing is spectral +smoothing. The goal of this function is to increase the signal to noise +ratio (S/N) without distorting the shape or relative size of the peaks. +The value on the slider is the polynomial order of the Savitzky-Golay +(SG) filter. The SG filter is fit to a moving window of 11 data +points where the center point in the window is replaced with the +polynomial estimate. The number of data points in the window is not user +adjustable. Higher numbers lead to more wiggly fits and thus less +smooth, lower numbers lead to more smooth fits, a 7th order polynomial +will make the spectrum have almost no smoothing. If smoothing is set to +0 then no smoothing is conducted on the spectrum. When smoothing is done +well, peak shapes and relative heights should not change. Typically a +3rd order polynomial (3 on the slider) works to increase the signal to +noise without distortion, but if the spectrum is noisy, decrease +polynomial order and if it is already smooth, increase the polynomial +order to the maximum (7). Examples of smoothing below:
+The different degrees of smoothing were achieved with the following R +commands:
+
+smooth_intens(raman_hdpe, p = 1)
+smooth_intens(raman_hdpe, p = 4)
The intensity-adjusted sample spectrum raman_adj
is
+smoothed accordingly:
+raman_smooth <- raman_adj %>%
+ smooth_intens()
+
+head(raman_smooth)
+#> wavenumber intensity
+#> 1: 301.040 0.00000000
+#> 2: 304.632 0.01568318
+#> 3: 308.221 0.02461353
+#> 4: 311.810 0.02828915
+#> 5: 315.398 0.02820811
+#> 6: 318.983 0.02586852
The second step of Open Specy’s preprocessing routine is baseline +correction. The goal of baseline correction is to get all non-peak +regions of the spectra to zero absorbance. The higher the polynomial +order, the more wiggly the fit to the baseline. If the baseline is not +very wiggly, a more wiggly fit could remove peaks which is not desired. +The baseline correction algorithm used in Open Specy is called +“iModPolyfit” (Zhao et al. 2007). This algorithm iteratively fits +polynomial equations of the specified order to the whole spectrum. +During the first fit iteration, peak regions will often be above the +baseline fit. The data in the peak region is removed from the fit to +make sure that the baseline is less likely to fit to the peaks. The +iterative fitting terminates once the difference between the new and +previous fit is small. An example of a good baseline fit below.
+The smoothed sample spectrum raman_smooth
is
+background-corrected as follows:
The final step of preprocessing is restricting the spectral range. +Sometimes the instrument operates with high noise at the ends of the +spectrum and sometimes the baseline fit can produce distortions at the +ends of the spectrum, both can be removed using this routine. You should +look into the signal to noise ratio of your specific instrument by +wavelength to determine what wavelength ranges to use. Distortions due +to baseline fit can be assessed from looking at the preprocess spectra +plot. Additionally, you can restrict the range to examine a single peak +or a subset of peaks of interests. This function allows users to isolate +peaks of interest for matching, while removing noise and influence from +less relevant spectral data.
+After you have the preprocessing parameters set, we recommend that +you download the preprocessed data for your records. The download data +button will append the uploaded data to three columns created by the +preprocessing parameters. “Wavelength” and “Absorbance” are columns from +the data uploaded by the user. “NormalizedIntensity” is the max-min +normalized value (Equation 1) of the “Absorbance”. “Smoothed” is the +Savitzky-Golay filter specified by the slider explained above. +“BaselineRemoved” is the smoothed and baseline corrected value that is +visible on the center plot.
+After uploading data and preprocessing it (if desired) you can now +identify the spectrum. To identify the spectrum go to the Match +Spectrum tab.
+You will see your spectrum and the top matches, but before looking at +matches, you need to check the three selectable parameters below.
+The spectra type input on the “Match spectra” tab specifies the type +of spectra (Raman or FTIR) that the user has uploaded and wants to match +to. This input will tell the website whether to use the FTIR library or +the Raman library to make the match.
+The spectra to analyze input specifies if the tool will match the +Uploaded spectra (unaltered by the inputs on the +Preprocess Spectra tab) or the Processed +Spectra (manipulated by the inputs in the Preprocess Spectra +Tab).
+The region to match input specifies if the “Full Spectrum” will match +the entire range of the spectra (including non peak regions) in the +reference database. This is the most intuitive match. Or should the +Peaks Only match just the peak regions in the reference +database. This is an advanced feature proposed in Renner et al. (2017). +This can be a less intuitive approach but in cases where there are few +peaks and high baseline interference, it could be the best option. In +cases where non-peak regions are important for the interpretation of the +match, this is not the best approach.
+The selectable table shows the top material matches returned by the +tool, their Pearson’s r value, and the organization they were provided +by. When rows are selected their spectra are added to the match plot. +The spectrum being matched and reference library are determined by the +previously mentioned parameters. During the matching process, one final +cleaning step happens using a simple minimum subtraction algorithm +(Equation 2) which in many cases will allow unprocessed spectra to +remove subtle baseline, but will not harm the spectra which has no +baseline. Then, these aligned data are tested for correlation using the +Pearson’s r. The Pearson’s r is used as a match quality indicator and +the spectra from the top 1000 best matches are returned from the +library. You can restrict the libraries which are displayed in the table +by clicking the box that says All under the +Organization column.
+Similarly you can restrict the range of Pearson's r values or search +for specific material types.
+\[\mathrm{for~each}~peak~group^{1,n}: x - +\mathrm{min}(x)\] ++Equation 2: Minimum Subtraction +
+The same table can be returned using the Open Specy library commands +in the R console.
+ +Whatever match is selected from the match table may have additional +metadata about it. That metadata will be displayed below the plot. Some +of this metadata may assist you in interpreting the spectra. For +example, if the spectra has metadata which says it is a liquid and you +are analyzing a solid particle, that spectrum may not be the best +match.
+The R command for manual metadata selection using
+sample_name == 5381
as example is:
+find_spec(sample_name == 5381, library = spec_lib, which = "raman")
This plot is dynamically updated by selecting matches from the match +table. The red spectrum is the spectrum that you selected from the +reference library and the white spectrum is the spectrum that you are +trying to identify. Whenever a new dataset is uploaded, the plot and +data table in this tab will be updated. These plots can be saved as a +.png by clicking the camera button at the top of the plot.
+There are several important things to consider when interpreting a +spectral match including the library source, the Pearson’s r, and other +metrics.
+When you click on a spectrum, all of the metadata that we have in +Open Specy about that source will be displayed in a metadata window +below to the matches table. Each library has different methodologies +used to develop it. It is useful to read up on the library sources from +the literature that they came from. E.g. Chabuka et al. 2020 focuses on +weathered plastics, so matching to it may suggest that your spectrum is +of a weathered polymer. Primpke et al. 2018 only has a spectral range up +to 2000, so some polymers may be difficult to differentiate with it. +Make sure to cite the libraries that you use during your search when you +publish your results. The authors were kind enough to make their data +open access so that it could be used in Open Specy and we should return +the favor by citing them.
+Correlation values are used to identify the closest matches available +in the current Open Specy spectral libraries to improve material +identification and reduce sample processing times. Pearson’s r values +range from 0 - 1 with 0 being a completely different spectrum and 1 +being an exact match. Some general guidelines that we have observed from +using Open Specy. If no matches are > ~0.3 the material may require +additional processing or may not exist in the Open Specy library. +Correlation values are not the only metric you should use to assess your +spectra’s match to a material in the library, matches need to make +sense.
+Peak position and height similarities are more important than +correlation and need to be assessed manually. Peak position correlates +with specific bond types. Peak height correlates to the concentration of +a compound. Therefore, peak height and peak position should match as +closely as possible to the matched spectrum. When there are peaks that +exist in the spectra you are trying to interpret that do not exist in +the match, there may be additional materials to identify. In this case, +restrict the preprocessing range to just the unidentified peak and try +to identify it as an additional component (see also https://www.compoundchem.com/2015/02/05/irspectroscopy/).
+Also, check the match metadata to see if the match makes sense. +Example: A single fiber cannot be a “cotton blend” since there would be +no other fibers to make up the rest of the blend. Example: Cellophane +does not degrade into fibers, so a match for a fiber to cellophane +wouldn’t make sense. Example: You are analyzing a particle at room +temperature, but the matched material is liquid at room temperature. The +material may be a component of the particle but it cannot be the whole +particle.
+You can choose to be specific about how you classify a substance +(e.g. polyester, cellophane) or more general (e.g. synthetic, +semi-synthetic, natural, etc.). The choice depends on your research +question. Using more general groups can speed up analysis time but will +decrease the information you have for interpretation. To identify +materials more generally, you can often clump the identities provided by +Open Specy to suit your needs. For example, matches to “polyester” and +“polypropylene” could be clumped to the category “plastic”.
+One common challenge is differentiating between LDPE and HDPE. But, +even with a low resolution instrument (MacroRAM, 2 cm-1 +pixel-1), you can still see some differences. From a wide +view, these low, medium, and high density PE samples all look relatively +similar (figures courtesy of Bridget O'Donnell, Horiba Scientific):
+ +But, a closer look at the 1450 cm-1 band reveals clear +differences:
+ +When you overlay them, you start to see differences in other spectral +regions too:
+ +So, the question is, how do we deal with samples that are very +similar with only subtle differences? Usually, researchers will use MVA +techniques after they’ve collected multiple reference spectra of known +samples (LDPE and HDPE in this case). They can then develop models and +apply them to distinguish between different types of PE. With a +reference database like Open Specy, this is complicated by the fact that +researchers are measuring samples on different instruments with +correspondingly different spectral responses and spectral resolutions. +That makes it even more difficult to accurately match definitively to +LDPE and HDPE as opposed to generic ‘PE’.
+One possibility is to place more emphasis (from a computational +perspective) on the bands that show the most difference (the triplet at +1450 cm-1) by restricting the range used to match in Open +Specy.
+The other, much simpler option is to just match any PE hit to generic +‘PE’ and not specifically HDPE or LDPE.
+Another challenge is in differentiating between types of nylons. But, +Raman has a pretty easy time distinguishing nylons. These spectra were +recorded of a series of nylons and the differences are much more +distinguishable compared to the PE results above (nylon 6, 6-6, 6-9, +6-10, and 6-12 top to bottom):
+ +The differences are even more pronounced when you overlay the +spectra:
+ +Download Spectragryph from https://www.effemm2.de/spectragryph/down.html
Open Spectragryph and upload your file by dragging and dropping +it into the console.
Chabuka BK, Kalivas JH (2020). “Application of a Hybrid Fusion +Classification Process for Identification of Microplastics Based on +Fourier Transform Infrared Spectroscopy.” Applied Spectroscopy, +74(9), 1167–1183. doi: 10.1177/0003702820923993.
+Cowger W, Gray A, Christiansen SH, Christiansen SH, Christiansen SH, +De Frond H, Deshpande AD, Hemabessiere L, Lee E, Mill L, et al. (2020). +“Critical Review of Processing and Classification Techniques for Images +and Spectra in Microplastic Research.” Applied Spectroscopy, +74(9), 989–1010. doi: 10.1177/0003702820929064.
+Cowger W, Steinmetz Z, Gray A, Munno K, Lynch J, Hapich H, Primpke S, +De Frond H, Rochman C, Herodotou O (2021). “Microplastic Spectral +Classification Needs an Open Source Community: Open Specy to the +Rescue!” Analytical Chemistry, 93(21), +7543–7548. doi: 10.1021/acs.analchem.1c00123.
+Primpke S, Wirth M, Lorenz C, Gerdts G (2018). “Reference Database +Design for the Automated Analysis of Microplastic Samples Based on +Fourier Transform Infrared (FTIR) Spectroscopy.” Analytical and +Bioanalytical Chemistry, 410(21), 5131–5141. doi: +10.1007/s00216-018-1156-x.
+Renner G, Schmidt TC, Schram J (2017). “A New Chemometric Approach +for Automatic Identification of Microplastics from Environmental +Compartments Based on FT-IR Spectroscopy.” Analytical +Chemistry, 89(22), 12045–12053. doi: 10.1021/acs.analchem.7b02472.
+Savitzky A, Golay MJ (1964). “Smoothing and Differentiation of Data +by Simplified Least Squares Procedures.” Analytical Chemistry, +36(8), 1627–1639.
+Zhao J, Lui H, McLean DI, Zeng H (2007). “Automated Autofluorescence +Background Subtraction Algorithm for Biomedical Raman Spectroscopy.” +Applied Spectroscopy, 61(11), 1225–1232. doi: +10.1366/000370207782597003.
+