Change of CSPP NUTNR CTD Variables

HelpDesk
1

I am looking at some CSPP NUTNR data post variable name change. I noticed that variables like ctd_temp were converted to sea_water_temperature, but there are also variables like ctdpf_j_cspp_instrument_recovered-sea_water_temperature. sea_water_temperature contains fill values while ctdpf* does not. I was previously under the assumption that ctd_temp was from the SUNA instrument header and could be ignored. Are the ctdpf* variables the ones used in corrections?

int_ctd_pressure also continues to exist and retain data and sea_water_pressure contains fill values.
Will int_ctd_pressure continue to be the interpolated pressure for the NUTNR using data from the co-located CTD on the CSPPs?

Thanks!

2

Ian,

We are working through changing the variable names in batches. The CSPP instruments weren’t in the first round so those ctdpf* variables, even though they are CTD related, slipped through on this last round. I’ll get those updated in time for our planned December release of further Preload changes.

In the meantime, yes the ctdpf* variables and int_ctd_pressure are the ones used in the corrections, and those are the values you should be using.

Cheers,
Chris

3

Hi Chris and others.

My issue builds off Ian’s… for those of us who have code that pulls data from the THREDDS server, the code no longer works after all the variable names have been updated. I’m finding it really frustrating to have to rewrite all of my code and try to discern which variables are what. Is there a resource available that tells us what the previous variable names were and what the current variable name is? In case it matters, I’m trying to do this right now with DOSTA data and it’s co-located CTD data.

Thanks!
Kristen

4

Kristen,

The CSV list below should have the list of changed variable names (sorry, Discourse seems to be blocking uploading a CSV file).

Chris

Original_Name,Corrected_Name,ID
conductivity,sea_water_electrical_conductivity,PD1
pressure,sea_water_pressure,PD2
salinity,sea_water_practical_salinity,PD3
temperature,sea_water_temperature,PD4
density,sea_water_density,PD5
temp,sea_water_temperature,PD6
practical_salinity,sea_water_practical_salinity,PD13
seawater_temperature,sea_water_temperature,PD908
seawater_pressure,sea_water_pressure,PD909
seawater_conductivity,sea_water_electrical_conductivity,PD910
vent_fluid_temperaure,vent_fluid_temperature,PD965
calibrated_phase,m_science_clothesline_lag,PD1410
sci_water_cond,sea_water_electrical_conductivity,PD1526
sci_water_temp,sea_water_temperature,PD1528
conductivity,sea_water_electrical_conductivity,PD1552
ctdpf_ckl_seawater_pressure,sea_water_pressure,PD1959
ctdpf_ckl_seawater_temperature,sea_water_temperature,PD1960
ctdpf_ckl_seawater_conductivity,sea_water_electrical_conductivity,PD1961
pressure_depth,pressure,PD2606
ctd_psu,sea_water_practical_salinity,PD2615
ctd_temp,sea_water_temperature,PD2616
ctd_dbar,sea_water_pressure,PD2617
sci_seawater_density,sea_water_density,PD2749
phsen_abcdef_ph_seawater,ph_seawater,PD2767
ctdbp_seawater_temperature,sea_water_temperature,PD2819
ctdbp_seawater_pressure,sea_water_pressure,PD2820
ctdbp_seawater_conductivity,sea_water_electrical_conductivity,PD2821
ctdmo_seawater_pressure,sea_water_pressure,PD2926
ctdmo_seawater_temperature,sea_water_temperature,PD2927
ctdmo_seawater_conductivity,sea_water_electrical_conductivity,PD2928
pressure_mbar,sea_water_pressure_mbar,PD3248
seawater_pressure_mbar,sea_water_pressure_mbar,PD3249
ctdav_n_auv_conductivity,sea_water_electrical_conductivity,PD3459
ctdbp_no_seawater_pressure,sea_water_pressure,PD3647
ctdbp_no_seawater_conductivity,sea_water_electrical_conductivity,PD3648
ctdpf_ckl_mmp_cds_seawater_conductivity,sea_water_electrical_conductivity,PD3666
dpc_ctd_seawater_conductivity,sea_water_electrical_conductivity,PD3706
sci_water_pressure_dbar,sea_water_pressure,PD3837
ctdmo_seawater_pressure,sea_water_pressure,PD7987
ctdmo_seawater_temperature,sea_water_temperature,PD7988
ctdmo_seawater_conductivity,sea_water_electrical_conductivity,PD7989
presf_tide_pressure,sea_water_pressure_at_sea_floor,PD8035
presf_tide_temperature,sea_water_temperature,PD8036
presf_wave_burst_pressure,sea_water_pressure_at_sea_floor,PD8037
5

Chris,

This is great and something others might find useful-- is it on the website somewhere?. Do you have a similar table for the oxygen instruments?

Thanks!
Kristen

6

Kristin,

I’m not sure if it is on the website or not. I’m pinging @dtrewcrist to see if it is. These are the only variable names that should be changed in this first round. Most of the other changes were focused on correcting unit strings (several used nonstandard ASCII characters that some applications didn’t support), fixing descriptions and updating/adding CF standard names were appropriate.

Are you seeing changed DO variable names?

Cheers,
Chris

7

Hi Kristen and Ian-

The changes are on the website here: https://oceanobservatories.org/changes-affecting-data/ under “Data Stream Parameters Simplified to Ease Access.” I’ve attached a list of variable name changes… On the website, you can access downloadable .csv and Excel files. I’ll also see if I can add these files to Discourse so they are more widely available.

Cheers.

Darlene

image001.jpg
image001.jpg1152×319 118 KB

(Attachment preload_variable_name_changes_20220824 (1).csv is missing)

8

Hi Chris,

I just looked closely at differences between DOSTA and associated CTD files I downloaded before the variable update and ones that I downloaded today. The variables ‘temp’ and ‘practical salinity’ were updated to ‘sea_water_temperature’ and ‘sea_water_practical_salinity’ as expected, but now occur in the DOSTA file and not the CTD file. However, the CTD files previously had variables that are no longer available in either file. I’ve highlighted those in yellow in the table.

Thoughts?

Untitled picture
Untitled picture863×511 13.5 KB

9

Kristen,

What reference designator and what time frame?

Cheers,
Chris

10

deployment0009_CE04OSSM-RID27-04-DOSTAD000-recovered_host-dosta_abcdjm_dcl_instrument_recovered_20191021T174307.464000-20200711T141801.400000

11

Kristen,

OK, there two issues at play here. The first is a known feature of the system but one that isn’t well advertised. When data from a co-located instrument is required to calculate a parameter in the dataset of interest (in this case temperature, salinity and pressure to correct the DO measurement and convert to umol/kg), the system will grab the data and those instrument records will be added to the THREDDS catalog. However, they will be incomplete. You can see an earlier post on this issue here: Missing variables in data sets. As a rule of thumb, I only use the data from the sensor of interest in each catalog, ignoring any other instrument files as they are often incomplete. Instead of getting the CTDBP data from the DOSTA directory, you need to get that data from the CTDBP directory.

For the DO data, explicitly use Catalog http://thredds.dataexplorer.oceanobservatories.org/thredds/catalog/ooigoldcopy/public/CE04OSSM-RID27-04-DOSTAD000-recovered_host-dosta_abcdjm_dcl_instrument_recovered/catalog.html and the *DOSTA*.nc files in there.

For the CTD data, explicitly use Catalog http://thredds.dataexplorer.oceanobservatories.org/thredds/catalog/ooigoldcopy/public/CE04OSSM-RID27-03-CTDBPC000-recovered_host-ctdbp_cdef_dcl_instrument_recovered/catalog.html and the *CTDBP*.nc files in there.

The second issue, with the missing QARTOD variables, looks to be an unintended consequence of the changes of migrating to QARTOD for QC processing and the parameter name changes. We are going to need to dig into that further to better understand where the disconnect is and how to correct it. I’m going to open a ticket in our internal system so we can start working on this.

Cheers,
Chris

12

Hi Chris,

Can you expand on this a little?

Does this mean the qc* variables will become deprecated in the future?

I am also looking at some instances where _qartod_executed variables have a value of 3. I do not know what that is and it is difficult to find documentation. The _qartod_results column also shows a 3, which I understand as suspect or of high interest. But it is unclear what 3 means in the _qartod_executed column. I previously operated under the assumption that only two options existed, 0 = not executed and 1 = executed.

Thanks,
Ian