Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CO2 Data Collection of the Mauna Loa Observatory #26

Merged
merged 92 commits into from
Nov 5, 2024

Conversation

haiderabbas007
Copy link
Contributor

Pull Request: CO2 Data Collection of the Mauna Loa Observatory

Linting

  • All Python scripts in this repository are pylint compliant, ensuring adherence to PEP8 standards for readability, maintainability, and consistent style.

Contents

This PR contains data files and scripts to process CO₂ concentration data collected from the Mauna Loa Observatory. The data includes various types of CO₂ measurements, such as discrete, monthly, daily, and hourly readings, stored in subdirectories corresponding to the sampling type (e.g., flask monthly, insitu daily).

  • Top-Level Files:

    • README.md: Provides comprehensive instructions, directory overview, file descriptions, citation information, and sample data formats.
    • LICENSE: Indicates that all data files are in the public domain under the Creative Commons Zero (CC0) 1.0 Universal Public Domain Dedication.
    • NOTES.md: Additional notes for repository users.
  • Subdirectories:

    • flask-discrete/
    • flask-monthly/
    • insitu-daily/
    • insitu-hourly/
    • insitu-monthly/
    • surface-discrete/
  • Subdirectory Files (for each of flask-discrete, flask-monthly, insitu-daily, insitu-hourly, insitu-monthly, and surface-discrete):

    • .csv: Processed CO₂ data in CSV format.
    • .json: Processed CO₂ data in JSON format.
    • _no_nan.json: JSON format without NaN values.
    • .md: Markdown format of the CO₂ data.
    • .py: Python script to process the raw data file, pylint compliant.
    • _README.html: Detailed README from NOAA for data usage and license information.
    • _raw.txt: Original raw data file as downloaded from NOAA.

Suggested Reviewer

@SchrodingersStruggle

@haiderabbas007
Copy link
Contributor Author

haiderabbas007 commented Nov 4, 2024

I modified all the .json files by adding the following line of code in my *.py files and added the modified files to the directory:

df_extracted['Timestamp'] = pd.to_datetime(df_extracted['Timestamp'], utc=True)

Are these acceptable now?

@laserlab

@laserlab
Copy link
Contributor

laserlab commented Nov 4, 2024

@haiderabbas007
flask_monthly.json:

[{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2 (ppm)":null,"Timestamp":null},{"CO2

Seems not to work

@haiderabbas007
Copy link
Contributor Author

haiderabbas007 commented Nov 4, 2024

Seems to be fixed now @laserlab

flask_monthly.json:

[{"Timestamp":"1969-08-01T00:00:00.000Z","CO2 (ppm)":322.51},{"Timestamp":"1969-09-01T00:00:00.000Z","CO2 (ppm)":321.36},{"Timestamp":"1969-10-01T00:00:00.000Z","CO2 (ppm)":320.74},{"Timestamp":"1969-11-01T00:00:00.000Z","CO2 (ppm)":321.98},{"Timestamp":"1969-12-01T00:00:00.000Z","CO2 (ppm)":323.78},{"Timestamp":"1970-01-01T00:00:00.000Z","CO2 (ppm)":325.13},{"Timestamp":"1970-02-01T00:00:00.000Z","CO2 (ppm)":325.55},{"Timestamp":"1970-03-01T00:00:00.000Z","CO2 (ppm)":325.67},{"Timestamp":"1970-04-01T00:00:00.000Z","CO2 (ppm)":326.58},{"Timestamp":"1970-05-01T00:00:00.000Z","CO2 (ppm)":327.69},{"Timestamp":"1970-06-01T00:00:00.000Z","CO2 (ppm)":327.3},{"Timestamp":"1970-07-01T00:00:00.000Z","CO2 (ppm)":326.12},{"Timestamp":"1970-08-01T00:00:00.000Z","CO2 (ppm)":324.91},{"Timestamp":"1970-09-01T00:00:00.000Z","CO2 (ppm)":322.99},{"Timestamp":"1970-10-01T00:00:00.000Z","CO2 (ppm)":322.64},{"Timestamp":"1970-11-01T00:00:00.000Z","CO2 (ppm)":324.03},{"Timestamp":"1970-12-01T00:00:00.000Z","CO2 (ppm)":325.54},{"Timestamp":"1976-07-01T00:00:00.000Z","CO2 (ppm)":332.29},{"Timestamp":"1976-08-01T00:00:00.000Z","CO2 (ppm)":331.04},{"Timestamp":"1976-09-01T00:00:00.000Z","CO2 (ppm)":329.67},{"Timestamp":"1976-10-01T00:00:00.000Z","CO2 (ppm)":329.33},{"Timestamp":"1976-11-01T00:00:00.000Z","CO2 (ppm)":330.47},{"Timestamp":"1976-12-01T00:00:00.000Z","CO2 (ppm)":331.97},{"Timestamp":"1977-01-01T00:00:00.000Z","CO2 (ppm)":332.77},{"Timestamp":"1977-02-01T00:00:00.000Z","CO2 (ppm)":333.1}

Fixed the file to produce correct json files. There was problem with formatting of the raw data file actually.
Raw file modified a little to make code work. Only last line of header lines changed
@laserlab laserlab self-requested a review November 4, 2024 20:23
@laserlab
Copy link
Contributor

laserlab commented Nov 4, 2024

Taking this as a reviewer since I already discussed it in class.

linting compliance
Copy link
Contributor

@laserlab laserlab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far, but please select one data set that is useful for determining the seasonal trend for Mount Lua (meaning contiguous and equidistant data) and only keep the json as the data file.

@haiderabbas007
Copy link
Contributor Author

I have selected the flask sampling (monthly) dataset. The new repository as directed is found here

https://github.com/haiderabbas007/CP1-24-HW5/tree/005c14c480ff29a09d0d21974917d933996305da/mauna-loa-data

@laserlab

@laserlab
Copy link
Contributor

laserlab commented Nov 4, 2024

@SchrodingersStruggle let’s merge this now so we have a data set

Copy link
Contributor

@laserlab laserlab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still issues, but we can fix those along the way. Let’s merge now

@SchrodingersStruggle
Copy link
Collaborator

Okay sounds good, merging.

@SchrodingersStruggle SchrodingersStruggle merged commit 6da5009 into ubsuny:main Nov 5, 2024
0 of 2 checks passed
@haiderabbas007
Copy link
Contributor Author

Still issues, but we can fix those along the way. Let’s merge now

Thanks, out of curiosity, what issues need to be fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation Organization python python related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants