Hey Matthias,
I agree. I've been talking with Phil Ashlock who is very interested in
ensuring there is a DataPackages-style approach. That's what we plan to
include in the wiki. CSV may have its limits but it's also one of the most
common formats for non-technologists. Publishers could just as easily use
JSON, XML, or some other format. My goal was to recommend data fields to
include and recommend standard field names based upon the existing Open311
convention. That being said, many governments (more than 90%) use
commercial open data solutions where CSV is readily available and built in
to the product.
Thanks!
-Andrew
On Mon, Jul 25, 2016 at 8:28 AM Matthias Meißer <<email obscured>>
wrote:
> Open311 Lists <http://lists.open311.org>
>
> [image: Photo of Matthias Meisser]
> <http://lists.open311.org/p/71GvPrmFOep5CReWb6yW0i>
> *Antw: [open311 discuss] Standardizing Bulk 311 Data*
> <http://lists.open311.org/r/topic/KVTEi9pbceqGI7U8JeGr1>
> by *Matthias Meisser* <http://lists.open311.org/p/71GvPrmFOep5CReWb6yW0i>
> in *Open311 Discuss* <http://lists.open311.org/groups/discuss>
> ------------------------------
> Hi Cliff,
>
> >>> "Cliff Ingham" <<email obscured>>
> <<email obscured>> schrieb am 15.07.2016
> um 20:04 in
> Nachricht <<email obscured>>:
> <<email obscured>>
> > Is there a special recommendation for the field and text delimiters
> to be
> > used?
> >
> > In particular, our request descriptions are the text that people have
> typed.
> > That text includes wacky characters, like commas and quotes. The
> CSV we're
> > preparing uses the double quote as the text delimiter; however, we're
>
> > concerned that users who download the data might need to know what
> delimiters
> > to set during the CSV import into their spreadsheet software.
> >
> > Is this something that should be in the spec?
>
> I tried to follow the thread losely and just want to point out, that
> IMHO are worth to consider for the format:
> - CSV is pretty poor file format (no metadata like encoding /
> lineending / delimiter, no structure, no binary payload ...)
> - JSON allows structures / hierachies and existing schemas for (tables,
> geodata, ...)
> - same for XML esp. with the benefit to mixin different subspecs which
> make it useful for semantic web etc.
>
> So maybe it`s worth to consider another file format?
> Please also keep in mind, that international standards runs into i18n
> problems (units, charcter sets, timezones, ....) that might need to be
> covered by file format and schema!
>
> regards,
> Matthias Meißer
> Rest of post… <http://lists.open311.org/r/post/KVTEi9pbceqGI7U8JeGr1>
> ------------------------------
> ⮪ Reply
>
<discuss@lists.open311.org?subject=Re%3A%20Antw%3A%20%5Bopen311%20discuss%5D%20Standardizing%20Bulk%20311%20Data>
> 🖂 New topic <discuss@lists.open311.org> View topic…
> <http://lists.open311.org/r/topic/KVTEi9pbceqGI7U8JeGr1>
> Unsubscribe
>
<discuss@lists.open311.org?subject=Unsubscribe&body=Hello%2C%0A%0APlease%20remove%20me%20from%20Open311%20Discuss%0A%3Chttp%3A//lists.open311.org/groups/discuss%3E%0A%0AThank%20you.>
> • Switch to a daily digest
>
<discuss@lists.open311.org?subject=Digest%20on&body=Hello%2C%0A%0APlease%20switch%20me%20from%20receiving%20one%20email%20per%20post%20to%20the%20daily%0Adigest%2C%20which%20summarises%20the%20all%20the%20posts%20made%20each%20day%20in%0AOpen311%20Discuss%0A%3Chttp%3A//lists.open311.org/groups/discuss%3E%0A%0AThank%20you.>
I agree. I've been talking with Phil Ashlock who is very interested in
ensuring there is a DataPackages-style approach. That's what we plan to
include in the wiki. CSV may have its limits but it's also one of the most
common formats for non-technologists. Publishers could just as easily use
JSON, XML, or some other format. My goal was to recommend data fields to
include and recommend standard field names based upon the existing Open311
convention. That being said, many governments (more than 90%) use
commercial open data solutions where CSV is readily available and built in
to the product.
Thanks!
-Andrew
On Mon, Jul 25, 2016 at 8:28 AM Matthias Meißer <<email obscured>>
wrote:
> Open311 Lists <http://lists.open311.org>
>
> [image: Photo of Matthias Meisser]
> <http://lists.open311.org/p/71GvPrmFOep5CReWb6yW0i>
> *Antw: [open311 discuss] Standardizing Bulk 311 Data*
> <http://lists.open311.org/r/topic/KVTEi9pbceqGI7U8JeGr1>
> by *Matthias Meisser* <http://lists.open311.org/p/71GvPrmFOep5CReWb6yW0i>
> in *Open311 Discuss* <http://lists.open311.org/groups/discuss>
> ------------------------------
> Hi Cliff,
>
> >>> "Cliff Ingham" <<email obscured>>
> <<email obscured>> schrieb am 15.07.2016
> um 20:04 in
> Nachricht <<email obscured>>:
> <<email obscured>>
> > Is there a special recommendation for the field and text delimiters
> to be
> > used?
> >
> > In particular, our request descriptions are the text that people have
> typed.
> > That text includes wacky characters, like commas and quotes. The
> CSV we're
> > preparing uses the double quote as the text delimiter; however, we're
>
> > concerned that users who download the data might need to know what
> delimiters
> > to set during the CSV import into their spreadsheet software.
> >
> > Is this something that should be in the spec?
>
> I tried to follow the thread losely and just want to point out, that
> IMHO are worth to consider for the format:
> - CSV is pretty poor file format (no metadata like encoding /
> lineending / delimiter, no structure, no binary payload ...)
> - JSON allows structures / hierachies and existing schemas for (tables,
> geodata, ...)
> - same for XML esp. with the benefit to mixin different subspecs which
> make it useful for semantic web etc.
>
> So maybe it`s worth to consider another file format?
> Please also keep in mind, that international standards runs into i18n
> problems (units, charcter sets, timezones, ....) that might need to be
> covered by file format and schema!
>
> regards,
> Matthias Meißer
> Rest of post… <http://lists.open311.org/r/post/KVTEi9pbceqGI7U8JeGr1>
> ------------------------------
> ⮪ Reply
>
<discuss@lists.open311.org?subject=Re%3A%20Antw%3A%20%5Bopen311%20discuss%5D%20Standardizing%20Bulk%20311%20Data>
> 🖂 New topic <discuss@lists.open311.org> View topic…
> <http://lists.open311.org/r/topic/KVTEi9pbceqGI7U8JeGr1>
> Unsubscribe
>
<discuss@lists.open311.org?subject=Unsubscribe&body=Hello%2C%0A%0APlease%20remove%20me%20from%20Open311%20Discuss%0A%3Chttp%3A//lists.open311.org/groups/discuss%3E%0A%0AThank%20you.>
> • Switch to a daily digest
>
<discuss@lists.open311.org?subject=Digest%20on&body=Hello%2C%0A%0APlease%20switch%20me%20from%20receiving%20one%20email%20per%20post%20to%20the%20daily%0Adigest%2C%20which%20summarises%20the%20all%20the%20posts%20made%20each%20day%20in%0AOpen311%20Discuss%0A%3Chttp%3A//lists.open311.org/groups/discuss%3E%0A%0AThank%20you.>