GCReW Weather Station Data for L1

[wilsonsj100]

@bpbond @roylrich do you think we could include the GCReW weather station data in the L1 release? This may mean pulling that data from the dropbox or mirroring it to the G-drive at some point. Roy would this be okay with you?

[bpbond]

Why do issues opened by @wilsonsj100 always result in more work for me 🤣

Seriously, though, this is a GREAT idea--lots and lots of TEMPEST data users will need met data, and if we can serve it up to them just like we do for the other sites that would be great.

Short-term, could you send me a sample raw file from the GCReW met station? That would help me assess how easily the existing pipeline will be able to process it, or if we'll need to make modifications.

[wilsonsj100]

GCREW_MET_GCREW_MET_15min_20230322020009.txt

That really is true... so sorry :)

Attached here is an example MET file that I think is a good example. Let me know if you need something else!

[bpbond]

Thank you!

Hurray, totally consistent TOA5 format. This will be easy. I ❤️ @roylrich !

[bpbond]

@stephpenn1 Does this make sense to you?

[bpbond]

Here are the data fields in the file posted above by @wilsonsj100 . I assume that we want any field that with a match in the COMPASS-FME ClimateVue data. What others?

Will need to think about how to handle unit differences.

TIMESTAMP	TS	Do we want?
RECORD	RN
Statname
Rain_in_Tot	inch
Rain_cm_Tot	cm	Yes but COMPASS var in mm
AirTC_Avg	°C	Yes
RH	%	Yes
CleanRH	%	?
VP	kPa	Yes - but COMPASS var is in mbar
VPD	Pa
SVP	Pa
Barometric_Pressure_PB110B	mbar	What's the difference from next one?
Barometric_Pressure_PB110B_Avg	mbar	Yes
SlrW_Avg	W/m²	Yes but COMPASS in MJ/m2
SlrMJ_Tot	MJ/m²	Yes but COMPASS in W/m2
PAR_Den_C_Avg	micromols/s/m^2	Yes
PAR_Tot_C_Tot	mmol/m^2second	Yes
Temp10cm_Avg	C
Temp25cm_Avg	C
Temp90cm_Avg	C
Above75cm_plate_Avg	C
Aquatroll_ID(2)
Depth	cm
Temperature	C
Actual_Conductivity	uS/cm
Specific_Conductivity	uS/cm
Salinity	PSU	Yes
TDS	ppt
Water_Density	g/cm3
Pressure	PSI
Resistivity	ohm-cm
WindDir_RM_Avg	degrees	What are these "RM" vars?
WindDir_RM	degrees
WS_ms_RM_Max	m/s
WS_ms_RM_Min	m/s
WS_ms_RM_Avg	m/s
WS_ms_RM	m/s	?
WindDir_WS_Avg	degrees	Yes
WindDir_WS	degrees	?
WS_ms_WS_Max	m/s	Yes
WS_ms_WS_Min	m/s	Yes
WS_ms_WS_Avg	m/s	Yes
WS_ms_WS	m/s	?

[bpbond]

GCReW met should be mapped to both TMP and GCW, right? So the same climate data appears in both sites' files?

[wilsonsj100]

I haven't looked at the parameters list yet, but I think if we have what is available from the other sites that will be great. I will look for others, but I think just matching would be fine.

One the TMP and GCW, I think yes. This would make it easy for people working with TMP data to use GCW met data.

[bpbond]

Thanks. That makes sense to me, at least for a first pass; we could expand the available data in subsequent releases.

[bpbond]

Initial mapping for going from GCReW met data to COMPASS-FME L1 variables:

[stephpenn1]

Hey all, how is this met station output different than the compass_gcw_w_climavue data streams already in our pipeline?

[roylrich]

This is the permanent gcrew met station Sent from my T-Mobile 5G Device Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Stephanie Pennington ***@***.***> Sent: Thursday, May 2, 2024 9:59:40 AM To: COMPASS-DOE/data-workflows ***@***.***> Cc: Rich, Roy ***@***.***>; Mention ***@***.***> Subject: Re: [COMPASS-DOE/data-workflows] GCReW Weather Station Data for L1 (Issue #158) External Email - Exercise Caution Hey all, how is this met station output different than the compass_gcw_w_climavue data streams already in our pipeline? — Reply to this email directly, view it on GitHub<#158 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFIDRBSSSIVNWKRS4E7LKHDZAJBEZAVCNFSM6AAAAABHB4MFT2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJQGU3TCNRVGI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[stephpenn1]

Is our COMPASS climavue station in a different location than the gcrew one?

I'm just wondering the benefits of having two met station datasets since it increase file sizes and might be confusing. @wilsonsj100 might be a good person to answer this, im just curious why use the gcrew one over the synoptic one. Thanks!

[bpbond]

Hmm. You're right @stephpenn1 there are existing ClimaVUE50 outputs for GCW...but all their values appear to be zero.

Paging @wilsonsj100 for clarifications 😕

[roylrich]

There is no climate at gcrew because it was an add on site Sent from my T-Mobile 5G Device Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Ben Bond-Lamberty ***@***.***> Sent: Friday, May 3, 2024 8:29:02 AM To: COMPASS-DOE/data-workflows ***@***.***> Cc: Rich, Roy ***@***.***>; Mention ***@***.***> Subject: Re: [COMPASS-DOE/data-workflows] GCReW Weather Station Data for L1 (Issue #158) External Email - Exercise Caution Hmm. You're right @stephpenn1<https://github.com/stephpenn1> there are existing ClimaVUE50 outputs for GCW...but all their values appear to be zero. Paging @wilsonsj100<https://github.com/wilsonsj100> for clarifications 😕 — Reply to this email directly, view it on GitHub<#158 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFIDRBTP56HVUMDV53WQIC3ZAN7I5AVCNFSM6AAAAABHB4MFT2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSHEZDAMRUGA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[bpbond]

Right, thanks @roylrich . I wonder why those zero rows are appearing in our L1...

Anyway, @stephpenn1 that's the rationale: there's a big convenience factor for users if weather data are easily available GCW and TMP as for the other synoptic sites.

[wilsonsj100]

@roylrich @bpbond @stephpenn1 I don't think we have a "COMPSS" climaVue at GCW because we were planning to use the existing met station. Roy can confirm, but I think the reason they are all zero is because it doesn't exist and there is nothing to populate that part of the dataset, but the code is there in case we added one. That is why I wanted to add the existing GCW weather station data in.

[stephpenn1]

Awesome thank you all for the clarification! We should remove the GCW Clima vue fields from the design table if it doesn't exist so that it doesn't end up in L1, and continue mapping the gcrew met station data as planned above :)

[bpbond]

OK, cool, thanks.

@roylrich maybe @selinalcheng or SW or SP, does the mapping in my comment above, from the GCReW datalogger fields to our COMPASS synoptic fields, look right?

Related note, for the wind speed and direction fields in the GCReW datalogger, what's the difference between the "RM" and "WS" fields? I wasn't sure which one to use.

[selinalcheng]

There are a few things in the mapping doc that are incorrect. Where is this document? / could you share with me on GDrive? I could edit and fix. So for the variables you didn’t fill out, are you just not including them in the L1 release?

remind me again, what does the prefix “wx” mean? Does that refer to the WXT met instrument?

I believe RM refers to one anemometer made by RM Young. WS refers to another anemometer called Windsonic. You may want to include both? IDK @roylrich ? How would you choose one over the other?

[bpbond]

@selinalcheng I shared a google sheet with you

[selinalcheng]

Ok, I am actually off today and Monday so I may not get to it till Tuesday if that’s ok!On May 3, 2024, at 11:39 AM, Ben Bond-Lamberty ***@***.***> wrote: @selinalcheng I shared a google sheet with you —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[bpbond]

I have one question for @wilsonsj100

• I assume that for consistency you'd like the GCReW weather station data mapped to GCW-W, following the pattern of other synoptic sites?

And one question for @roylrich (probably)

• If we're going to do this, could I be added to the Dropbox share for these data? Or if the data are distributed some other way, let me know options to gain access?

Thanks!

[wilsonsj100]

Yes, I think that would make the most sense. This would match the rest of the sites and make it easier to utilize the data across sites.

Thank you!

[selinalcheng]

Ok, I updated the google sheet with fixes and comments. The VP measurement is dubious and I am still not sure how you would want to deal with / choose the RM Young vars over the Windsonic vars

[bpbond]

Thank you! This will let me do some robust testing while I wait for data access.

[bpbond]

@wilsonsj100 The L1 data on the google drive have been updated with the GCReW met data.

[wilsonsj100]

Great! Thank you - is that as of right now? Ie. if I downloaded the data earlier today, do I need to redownload now?

[bpbond]

I'm sorry, yes, just pushed up a new version.

[bpbond]

Actually it's not quite done! Sit tight for 5 minutes

[wilsonsj100]

Okie doke! No problem! I can redownload tomorrow

[bpbond]

All done

[wilsonsj100]

Yay! Thank you!