CSV Downloader Exports Null/Empty Values

[aeratek]

The CSV download functionality is exporting empty entries, which become difficult to work with in a spreadsheet (and in my CSV importer code I'm working with). Example shown at bottom.

The empty values come from one or more of the following: speed, altitude, climb, track, epx, epy, epv.

There's a few ways of approach a fix to this: (1) input the string "NULL" in place of the emptiness between commas, just like the GPS dongle does when it provides raw data in CSV format to the NodeJS app (I think), (2) add a new button or otherwise replace this button's functionality in the dashboard with the functionality of only exporting measure_time, latitude and longitude, for the time being, or (3) do not fix and make it the customer's problem (whoever downloads the CSV has to reformat it, removing these empty areas and aligning columns/rows manually/automatically somehow).

Mon Mar 16 10:00:36 EDT 2015,40.71669957,-73.965549512,1.23,34.83,-0.255,71.7706,15.02,22.024,89.408
Mon Mar 16 10:00:41 EDT 2015,40.71676894,-73.965521658,0.172,25.879,-0.398,274.1072,33.364,64.557,320.055
Mon Mar 16 10:00:46 EDT 2015,40.716646362,-73.965659989,4.261,,,219.897,33.364,64.557,
Mon Mar 16 10:00:51 EDT 2015,40.716503709,-73.965838982,3.998,,,212.2336,,,
Mon Mar 16 10:00:56 EDT 2015,40.716502086,-73.965827608,0.245,,,258.3136,,,

[aohren]

Sure, we can put placeholder values in for empty data if that's better.
Most CSV data I've ever worked with uses this convention though (just an
empty field) to represent missing values, instead of putting in a
placeholder string like "NULL" in there.

FWIW, I imported this sample output in Google Spreadsheets and locally in
OpenOffice and the data looks fine (appropriate fields are blank, just like
the data).

On Fri, Apr 3, 2015 at 2:37 PM, aeratek [email protected] wrote:

The CSV download functionality is exporting empty entries, which become
difficult to work with in a spreadsheet (and in my CSV importer code I'm
working with). Example shown at bottom.

The empty values come from one or more of the following: speed, altitude,
climb, track, epx, epy, epv.

There's a few ways of approach a fix to this: (1) input the string "NULL"
in place of the emptiness between commas, just like the GPS dongle does
when it provides raw data in CSV format to the NodeJS app (I think), (2)
add a new button or otherwise replace this button's functionality in the
dashboard with the functionality of only exporting measure_time, latitude
and longitude, for the time being, or (3) do not fix and make it the
customer's problem (whoever downloads the CSV has to reformat it, removing
these empty areas and aligning columns/rows manually/automatically
somehow).

Mon Mar 16 10:00:36 EDT
2015,40.71669957,-73.965549512,1.23,34.83,-0.255,71.7706,15.02,22.024,89.408
Mon Mar 16 10:00:41 EDT
2015,40.71676894,-73.965521658,0.172,25.879,-0.398,274.1072,33.364,64.557,320.055
Mon Mar 16 10:00:46 EDT
2015,40.716646362,-73.965659989,4.261,,,219.897,33.364,64.557,
Mon Mar 16 10:00:51 EDT 2015,40.716503709,-73.965838982,3.998,,,212.2336,,,
Mon Mar 16 10:00:56 EDT 2015,40.716502086,-73.965827608,0.245,,,258.3136,,,

—
Reply to this email directly or view it on GitHub
#19.

[aohren]

Also, what clients / apps are having problems with this exactly (ie- what
CSV import lib / spreadsheet app are you using)? Might be worth figuring
out what they expect for empty fields. For example it might just be Excel
expecting quoting the fields (I think that's MS-style).

Inserting a placeholder might have it's own problems. Like if you try
importing the CSV into a database, with the empty fields, that's fine. But
if you have "NULL" or some other value in there, you'll have the word
"NULL" all over your dataset and that'll require it's own
cleaning/reformatting.

On Sat, Apr 4, 2015 at 9:54 AM, Adam Ohren [email protected] wrote:

Sure, we can put placeholder values in for empty data if that's better.
Most CSV data I've ever worked with uses this convention though (just an
empty field) to represent missing values, instead of putting in a
placeholder string like "NULL" in there.

FWIW, I imported this sample output in Google Spreadsheets and locally in
OpenOffice and the data looks fine (appropriate fields are blank, just like
the data).

On Fri, Apr 3, 2015 at 2:37 PM, aeratek [email protected] wrote:

The CSV download functionality is exporting empty entries, which become
difficult to work with in a spreadsheet (and in my CSV importer code I'm
working with). Example shown at bottom.

The empty values come from one or more of the following: speed, altitude,
climb, track, epx, epy, epv.

There's a few ways of approach a fix to this: (1) input the string "NULL"
in place of the emptiness between commas, just like the GPS dongle does
when it provides raw data in CSV format to the NodeJS app (I think), (2)
add a new button or otherwise replace this button's functionality in the
dashboard with the functionality of only exporting measure_time, latitude
and longitude, for the time being, or (3) do not fix and make it the
customer's problem (whoever downloads the CSV has to reformat it, removing
these empty areas and aligning columns/rows manually/automatically
somehow).

Mon Mar 16 10:00:36 EDT
2015,40.71669957,-73.965549512,1.23,34.83,-0.255,71.7706,15.02,22.024,89.408
Mon Mar 16 10:00:41 EDT
2015,40.71676894,-73.965521658,0.172,25.879,-0.398,274.1072,33.364,64.557,320.055
Mon Mar 16 10:00:46 EDT
2015,40.716646362,-73.965659989,4.261,,,219.897,33.364,64.557,
Mon Mar 16 10:00:51 EDT
2015,40.716503709,-73.965838982,3.998,,,212.2336,,,
Mon Mar 16 10:00:56 EDT
2015,40.716502086,-73.965827608,0.245,,,258.3136,,,

—
Reply to this email directly or view it on GitHub
#19.

[aeratek]

Gotcha. Ok, let's hold off on implementing this, then, given what you
mentioned. I will look at what happens when importing these CSV and KML
files into tools like CartoDB and Google Earth, when they have empty/null
values, just to double-check there's no issues. Both tools are usually
pretty good about this. In my case, I was attempting to import this data
into my own (Java) app and I really only needed measure_time, lat and lon,
so the other data wasn't doing much for me and was just causing CSV-parsing
complexity for the simple mapping and analysis I was doing.

On Sat, Apr 4, 2015 at 3:54 AM, Adam Ohren [email protected] wrote:

Sure, we can put placeholder values in for empty data if that's better.
Most CSV data I've ever worked with uses this convention though (just an
empty field) to represent missing values, instead of putting in a
placeholder string like "NULL" in there.

FWIW, I imported this sample output in Google Spreadsheets and locally in
OpenOffice and the data looks fine (appropriate fields are blank, just like
the data).

On Fri, Apr 3, 2015 at 2:37 PM, aeratek [email protected] wrote:

The CSV download functionality is exporting empty entries, which become
difficult to work with in a spreadsheet (and in my CSV importer code I'm
working with). Example shown at bottom.

The empty values come from one or more of the following: speed, altitude,
climb, track, epx, epy, epv.

There's a few ways of approach a fix to this: (1) input the string "NULL"
in place of the emptiness between commas, just like the GPS dongle does
when it provides raw data in CSV format to the NodeJS app (I think), (2)
add a new button or otherwise replace this button's functionality in the
dashboard with the functionality of only exporting measure_time, latitude
and longitude, for the time being, or (3) do not fix and make it the
customer's problem (whoever downloads the CSV has to reformat it,
removing
these empty areas and aligning columns/rows manually/automatically
somehow).

Mon Mar 16 10:00:36 EDT

2015,40.71669957,-73.965549512,1.23,34.83,-0.255,71.7706,15.02,22.024,89.408
Mon Mar 16 10:00:41 EDT

2015,40.71676894,-73.965521658,0.172,25.879,-0.398,274.1072,33.364,64.557,320.055
Mon Mar 16 10:00:46 EDT
2015,40.716646362,-73.965659989,4.261,,,219.897,33.364,64.557,
Mon Mar 16 10:00:51 EDT
2015,40.716503709,-73.965838982,3.998,,,212.2336,,,
Mon Mar 16 10:00:56 EDT
2015,40.716502086,-73.965827608,0.245,,,258.3136,,,

—
Reply to this email directly or view it on GitHub
#19.

—
Reply to this email directly or view it on GitHub
#19 (comment).

[aohren]

Cool. Another thing we can do is have a request specify which fields they
want. So like if you only want time, lat, and long, then it could exclude
all the other fields in the download.

On Saturday, April 4, 2015, aeratek [email protected] wrote:

Gotcha. Ok, let's hold off on implementing this, then, given what you
mentioned. I will look at what happens when importing these CSV and KML
files into tools like CartoDB and Google Earth, when they have empty/null
values, just to double-check there's no issues. Both tools are usually
pretty good about this. In my case, I was attempting to import this data
into my own (Java) app and I really only needed measure_time, lat and lon,
so the other data wasn't doing much for me and was just causing CSV-parsing
complexity for the simple mapping and analysis I was doing.

On Sat, Apr 4, 2015 at 3:54 AM, Adam Ohren <[email protected]
javascript:_e(%7B%7D,'cvml','[email protected]');> wrote:

Sure, we can put placeholder values in for empty data if that's better.
Most CSV data I've ever worked with uses this convention though (just an
empty field) to represent missing values, instead of putting in a
placeholder string like "NULL" in there.

FWIW, I imported this sample output in Google Spreadsheets and locally in
OpenOffice and the data looks fine (appropriate fields are blank, just
like
the data).

On Fri, Apr 3, 2015 at 2:37 PM, aeratek <[email protected]
javascript:_e(%7B%7D,'cvml','[email protected]');> wrote:

The CSV download functionality is exporting empty entries, which become
difficult to work with in a spreadsheet (and in my CSV importer code
I'm
working with). Example shown at bottom.

The empty values come from one or more of the following: speed,
altitude,
climb, track, epx, epy, epv.

There's a few ways of approach a fix to this: (1) input the string
"NULL"
in place of the emptiness between commas, just like the GPS dongle does
when it provides raw data in CSV format to the NodeJS app (I think),
(2)
add a new button or otherwise replace this button's functionality in
the
dashboard with the functionality of only exporting measure_time,
latitude
and longitude, for the time being, or (3) do not fix and make it the
customer's problem (whoever downloads the CSV has to reformat it,
removing
these empty areas and aligning columns/rows manually/automatically
somehow).

Mon Mar 16 10:00:36 EDT

2015,40.71669957,-73.965549512,1.23,34.83,-0.255,71.7706,15.02,22.024,89.408

Mon Mar 16 10:00:41 EDT

2015,40.71676894,-73.965521658,0.172,25.879,-0.398,274.1072,33.364,64.557,320.055

Mon Mar 16 10:00:46 EDT
2015,40.716646362,-73.965659989,4.261,,,219.897,33.364,64.557,
Mon Mar 16 10:00:51 EDT
2015,40.716503709,-73.965838982,3.998,,,212.2336,,,
Mon Mar 16 10:00:56 EDT
2015,40.716502086,-73.965827608,0.245,,,258.3136,,,

—
Reply to this email directly or view it on GitHub
#19.

—
Reply to this email directly or view it on GitHub
#19 (comment).

—
Reply to this email directly or view it on GitHub
#19 (comment).

from mobile; forgive brevity