File parsing glitches during observation upload

[nicholas512]

Problem: Observation uploader can no longer parse out junk such as:

Lines: ‘Delta Time: 318 secs’
Columns: ‘#HK-Bat:V’ and ‘#HK-Temp:oC’.
Text: '(NO VALUE)' not parsed as NaN

Solutions:

A temporary solution is to clean FG2 and GP5W data files using dbpf_GP5W_file_formatter and dbpf_FG2toGP5W functions in PermafrostDB.
Perhaps a wee git rollback of SensorDB to a previous version that worked to identify where software went awry.

Example file below.

E50BED_20210811192228.csv

[nicholas512]

I tried uploading that file to a dockerized sensordb (without any changes to the file parser).
it complained about the name of the sensor:

{
   "skippedCount": 0,
   "collisionCount": 0,
   "type": "import",
   "inTransitCount": 0,
   "error": {
      "error": "Sensor with label (#2:rH) not found for device: Device(id=d39076f3-7ced-4d26-91d2-85f3cc48ef27,sn=E50BED)"
   },
   "insertedCount": 0
}

When I renamed the sensor to #2:%RH (matching the M-Log5W-US-HUMIDITY profile), it uploaded without complaint:

{
   "importId": "e5a446c1-e96c-4886-92c0-38306c67c6c4",
   "skippedCount": 0,
   "collisionCount": 0,
   "type": "import",
   "inTransitCount": 0,
   "insertedCount": 23
}

So a solution might include:

• creating a new device sensor profile (TEMP_RELHU) to match the actual logger output
• changing that device and its sensor labels to match TEMP_RELHU (i.e. with label #2:rH)

[nicholas512]

This file was one where the Delta Time didn't work:

Logger: #E50BB3 'PT1000TEMP' - USP_EXP2 - (CGI) Expander for GP5W - (V2.7, Jan 12 2016),
Delta Time: -64406 secs,,
No,Time,#1:oC
4,08.07.2021 00:08:28,21.7933

The regex match for a delta time line is "^Delta\\s+Time:\\s*(-?[0-9]+)\\s+secs\\s*$". Note that there are trailing commas in this file (is this new??)

Should be able to replace the regex with
"^Delta\\s+Time:\\s*(-?[0-9]+)\\s+secs[,\\s]*$"