Open311

Discussion Lists

SeeClickFix API Feedback

Back to topic listing

  • From: Ian Bicking

    Subject: SeeClickFix API Feedback

    Date: Jun 24, 2009 05:18 PM 

    I looked at the SeeClickFix API a bit, and I wasn't sure where to send
feedback, but since Ben Berkowitz is on this list I figure I'll just send it
here.
Feedback reading from the TOP of the API document (
http://seeclickfix.com/simple_pages/23) to the bottom:

1. I think it would be valuable to focus more on the service request
document.  This is essentially the core document for the API, and focusing
on it is key to the RESTfulness of the API.  I mentioned in another email (
http://lists.open311.org/discuss/archive/2009/06/1245703774174/forum_view)
how Atom entries are a potential way of representing requests, but even if
the document is JSON that's fine.  You might even give it a mimetype, like
application/x-municipal-service-request+json... but I dunno, maybe not.

2. I don't think much is accomplished with a dual XML/JSON API. If the XML
and JSON aren't equivalent, of course you end up with all sorts of
nastiness.  If they are just two representations of the same thing, then you
have to deal with some invisible "real" record that has two views.  I think
it's easier for everyone if you just have a canonical representation.  JSON
and XML are also extensible in different ways, and it could get confusing.
 In JSON you typically add a new key to an object (aka hash table) where in
XML you use namespaces and tags.  There are some attempts to represent XML
style extensions in JSON, but the feel like a lot of trouble to go through
just to create something that feels terribly awkward.  From what I can tell,
XML is only proposed but not specified?  I assume extensibility will be
important, as mapping the API to existing systems or novel use cases will
require additional fields.

3. Locale seems useful, but also seems a bit tricky... I'm guessing some
fields are potentially based on some translatable underlying resource?  For
instance, "Phone: +1 (123) 456 - 7890" -- if that's based on some structured
record you could translate "Phone".  But if it's just a text field then it's
not translatable.  Seems like it would be tricky on the API level.  Though
one thing in the DC 311 API is the service request type definitions, which
has a bunch of fields with questions (like
"'Is there damage to the vehicle?") that clearly should be translatable.
 Essentially this aspect of the DC API is a description of their schema for
different kinds of requests.

4. Creating tickets is done with PUT /v1/issue -- it should be POST.  PUT
should go to the actual URL, like PUT /v1/issue/my-issue -- but when you
create an issue you don't know what URL it will go to.  Atompub goes over
this:

Request:
  POST /container
  Content-Type: application/atom+xml; type=entry

  <entry>...</entry>

Response:
  201 Created
  Location: /container/id-created-by-server
  Content-Type: application/atom+xml; type=entry

  <entry>...</entry>

The response contains the entry after any changes made by the server.  The
server might clean HTML for instance, or reject elements that it doesn't
understand.  The location where the entry ends up is given in the Location
field.  I'd recommend the same pattern.  You might want to use the pattern
of a self-URL embedded in the entry itself.  In Atom this is <link
rel="self" href="..." />, in JSON you could just use a "self" key.
The specification as it is doesn't make it that clear what it accepts.  It
mentions some variables, but it's not clear if the body of the request is a
JSON document with those keys?

5. It would probably be useful to give the locale of the request.  E.g., if
someone submits a request in Spanish, to say that somewhere (maybe so it can
be routed to a Spanish-speaking dispatcher).  It doesn't seem like locale
means quite this.

6. There should be ways to track who submitted the issue, in some way that
is essentially private to the software submitting the issue.  If there's
abuse through some frontend, the person managing that frontend might have
their own ways of identifying the problem.

7. Atom-style globally unique identifiers could be useful when routing
requests.  One use case we've considered is a service that accepts possibly
incomplete requests, and corrects and routes them (perhaps partially
crowdsourced).  For instance, we're thinking about a service for collecting
bike rack requests, which might have no direct interaction with the 311
database.  Anyway, I can imagine tickets going into systems and then coming
out (like if they are misfiled); having a globally unique and stable ID
makes it easier to see duplicates when this happens.  The only tricky part
then is that this ID has to be carried around with the request, instead of
using purely internal IDs.  Atom doesn't require that the globally unique ID
show up in any URL.

8. The callback URI stuff is confusing.  It shows up in several places, but
not entirely consistently.

9. Why does GET /v1/issue/<id> accept request parameters?

10. For GET /v1/tags, a description of tags would be helpful.  Also an
indication about whether freeform tags are allowed.  For instance, DC's
service request types seem mappable to tags.  I could imagine it returning a
structure like {"S0000": {"description": "Abandoned vehicle"}}.  It seems
like making the value an object leaves useful room for extension.  I would
assume that tags should not be translated, but the descriptions could be.

11. GData does queries with URLs like /v1/issues/-/tag1/tag2 (where all tags
are intersections).  The - avoids some potential conflicts.  I don't have
strong opinions about it, though, just noting what GData does.

12. Not sure why /v1/search and /v1/find are different.  Can't it just be
the intersection of the queries you perform?

13. Find should support polygons and the like as well.  Some cities are not
aligned with North (e.g., Manhattan), so a box isn't a good fit for any
normal selection.  Even a point and radius is probably a more natural
search.  (I wonder if OpenSearch has anything to suggest ways of describing
the available search parameters?)

14. Dates are useful for queries, lets you see updates.

15. Is there paging for large results?  This also applies to GET /v1/issue.

16. There's nothing feed-like.  Feeds would be handy.  An actual feed would
be handy.

Then the open questions:

    Q: How do we handle API keying?
    Q: Is it required or optional?

I'd imagine it could be optional at least for some parts of the API, that
only relate to reading.

    Q: Should there be an automated request/response mechanism in the API?

Not sure what this means.

    Q: Are providers allowed to rate limit clients? If so are there
guidelines?

They always could, of course -- clients should never rely on the server
always accepting the request.  I thought there was a good HTTP response code
for this, but I can't find one.

    Q: Should all messages return status 200?

Definitely no!

    Q: Should all error messages be human readable?

Not sure.  The Content-Type of the error response should at least indicate
the possibilities (i.e., text/html or text/plain are directly human
readable, and application/json is definitely not).  It might be useful to
define a JSON response format, similar to JSON-RPC's.

    Q: Should all error messages be localized to the requested or default
locale?

Mostly the error messages are for the developers, not for the users
interacting with some service.  So I think the requested locale isn't that
applicable.

    Q: Would supporting client based request identifiers that allow clients
to replay requests without repetitive effects be worth while?

You mean like stable IDs, like I mention in 7?  I think so.

    Q: How long is the provider expected to track these

If it is handled like Atom, indefinitely.  This doesn't seem too hard to do,
it's just another field for requests.

    Q: May providers refuse service at their discretion?

Of course they always can do that, whether we want them to or not, and it
can happen unintentionally at any time (e.g., server outage).

    Q: How does a service provider reject PUT/POST/DELETE? Do rejects
trigger callbacks?

400 Bad Request, or 405 Method Not Allowed.  Callbacks should only trigger
if the service request is actually changed, so no.

    Q: Is the initial description considered a comment on the issue?

I *think* this is how DC treats it, but I'm not sure.

    Q: Can anyone add comments to issues?

I think that would be useful to allow, though maybe optional.  I could
imagine it would be better in some cases to turn it off because it would
just encourage people to submit comments that no one is going to read.


--
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker

    Attachments

    Thread Outline:

  • From: Ben Berkowitz

    Subject: Re: SeeClickFix API Feedback

    Date: Jun 24, 2009 05:22 PM 

    Ian,
Thanks for all of this an appreciate the feedback.
Will take a closer look at your comments now.
-Ben

On Wed, Jun 24, 2009 at 5:18 PM, Ian Bicking <ianb@...> wrote:

> I looked at the SeeClickFix API a bit, and I wasn't sure where to send
> feedback, but since Ben Berkowitz is on this list I figure I'll just send it
> here.
> Feedback reading from the TOP of the API document (
> http://seeclickfix.com/simple_pages/23) to the bottom:
>
> 1. I think it would be valuable to focus more on the service request
> document.  This is essentially the core document for the API, and focusing
> on it is key to the RESTfulness of the API.  I mentioned in another email (
> http://lists.open311.org/discuss/archive/2009/06/1245703774174/forum_view)
> how Atom entries are a potential way of representing requests, but even if
> the document is JSON that's fine.  You might even give it a mimetype, like
> application/x-municipal-service-request+json... but I dunno, maybe not.
>
> 2. I don't think much is accomplished with a dual XML/JSON API. If the XML
> and JSON aren't equivalent, of course you end up with all sorts of
> nastiness.  If they are just two representations of the same thing, then you
> have to deal with some invisible "real" record that has two views.  I think
> it's easier for everyone if you just have a canonical representation.  JSON
> and XML are also extensible in different ways, and it could get confusing.
>  In JSON you typically add a new key to an object (aka hash table) where in
> XML you use namespaces and tags.  There are some attempts to represent XML
> style extensions in JSON, but the feel like a lot of trouble to go through
> just to create something that feels terribly awkward.  From what I can tell,
> XML is only proposed but not specified?  I assume extensibility will be
> important, as mapping the API to existing systems or novel use cases will
> require additional fields.
>
> 3. Locale seems useful, but also seems a bit tricky... I'm guessing some
> fields are potentially based on some translatable underlying resource?  For
> instance, "Phone: +1 (123) 456 - 7890" -- if that's based on some structured
> record you could translate "Phone".  But if it's just a text field then it's
> not translatable.  Seems like it would be tricky on the API level.  Though
> one thing in the DC 311 API is the service request type definitions, which
> has a bunch of fields with questions (like
> "'Is there damage to the vehicle?") that clearly should be translatable.
>  Essentially this aspect of the DC API is a description of their schema for
> different kinds of requests.
>
> 4. Creating tickets is done with PUT /v1/issue -- it should be POST.  PUT
> should go to the actual URL, like PUT /v1/issue/my-issue -- but when you
> create an issue you don't know what URL it will go to.  Atompub goes over
> this:
>
> Request:
>   POST /container
>   Content-Type: application/atom+xml; type=entry
>
>   <entry>...</entry>
>
> Response:
>   201 Created
>   Location: /container/id-created-by-server
>   Content-Type: application/atom+xml; type=entry
>
>   <entry>...</entry>
>
> The response contains the entry after any changes made by the server.  The
> server might clean HTML for instance, or reject elements that it doesn't
> understand.  The location where the entry ends up is given in the Location
> field.  I'd recommend the same pattern.  You might want to use the pattern
> of a self-URL embedded in the entry itself.  In Atom this is <link
> rel="self" href="..." />, in JSON you could just use a "self" key.
> The specification as it is doesn't make it that clear what it accepts.  It
> mentions some variables, but it's not clear if the body of the request is a
> JSON document with those keys?
>
> 5. It would probably be useful to give the locale of the request.  E.g., if
> someone submits a request in Spanish, to say that somewhere (maybe so it can
> be routed to a Spanish-speaking dispatcher).  It doesn't seem like locale
> means quite this.
>
> 6. There should be ways to track who submitted the issue, in some way that
> is essentially private to the software submitting the issue.  If there's
> abuse through some frontend, the person managing that frontend might have
> their own ways of identifying the problem.
>
> 7. Atom-style globally unique identifiers could be useful when routing
> requests.  One use case we've considered is a service that accepts possibly
> incomplete requests, and corrects and routes them (perhaps partially
> crowdsourced).  For instance, we're thinking about a service for collecting
> bike rack requests, which might have no direct interaction with the 311
> database.  Anyway, I can imagine tickets going into systems and then coming
> out (like if they are misfiled); having a globally unique and stable ID
> makes it easier to see duplicates when this happens.  The only tricky part
> then is that this ID has to be carried around with the request, instead of
> using purely internal IDs.  Atom doesn't require that the globally unique ID
> show up in any URL.
>
> 8. The callback URI stuff is confusing.  It shows up in several places, but
> not entirely consistently.
>
> 9. Why does GET /v1/issue/<id> accept request parameters?
>
> 10. For GET /v1/tags, a description of tags would be helpful.  Also an
> indication about whether freeform tags are allowed.  For instance, DC's
> service request types seem mappable to tags.  I could imagine it returning a
> structure like {"S0000": {"description": "Abandoned vehicle"}}.  It seems
> like making the value an object leaves useful room for extension.  I would
> assume that tags should not be translated, but the descriptions could be.
>
> 11. GData does queries with URLs like /v1/issues/-/tag1/tag2 (where all
> tags are intersections).  The - avoids some potential conflicts.  I don't
> have strong opinions about it, though, just noting what GData does.
>
> 12. Not sure why /v1/search and /v1/find are different.  Can't it just be
> the intersection of the queries you perform?
>
> 13. Find should support polygons and the like as well.  Some cities are not
> aligned with North (e.g., Manhattan), so a box isn't a good fit for any
> normal selection.  Even a point and radius is probably a more natural
> search.  (I wonder if OpenSearch has anything to suggest ways of describing
> the available search parameters?)
>
> 14. Dates are useful for queries, lets you see updates.
>
> 15. Is there paging for large results?  This also applies to GET /v1/issue.
>
> 16. There's nothing feed-like.  Feeds would be handy.  An actual feed would
> be handy.
>
> Then the open questions:
>
>     Q: How do we handle API keying?
>     Q: Is it required or optional?
>
> I'd imagine it could be optional at least for some parts of the API, that
> only relate to reading.
>
>     Q: Should there be an automated request/response mechanism in the API?
>
> Not sure what this means.
>
>     Q: Are providers allowed to rate limit clients? If so are there
> guidelines?
>
> They always could, of course -- clients should never rely on the server
> always accepting the request.  I thought there was a good HTTP response code
> for this, but I can't find one.
>
>     Q: Should all messages return status 200?
>
> Definitely no!
>
>     Q: Should all error messages be human readable?
>
> Not sure.  The Content-Type of the error response should at least indicate
> the possibilities (i.e., text/html or text/plain are directly human
> readable, and application/json is definitely not).  It might be useful to
> define a JSON response format, similar to JSON-RPC's.
>
>     Q: Should all error messages be localized to the requested or default
> locale?
>
> Mostly the error messages are for the developers, not for the users
> interacting with some service.  So I think the requested locale isn't that
> applicable.
>
>     Q: Would supporting client based request identifiers that allow clients
> to replay requests without repetitive effects be worth while?
>
> You mean like stable IDs, like I mention in 7?  I think so.
>
>     Q: How long is the provider expected to track these
>
> If it is handled like Atom, indefinitely.  This doesn't seem too hard to
> do, it's just another field for requests.
>
>     Q: May providers refuse service at their discretion?
>
> Of course they always can do that, whether we want them to or not, and it
> can happen unintentionally at any time (e.g., server outage).
>
>     Q: How does a service provider reject PUT/POST/DELETE? Do rejects
> trigger callbacks?
>
> 400 Bad Request, or 405 Method Not Allowed.  Callbacks should only trigger
> if the service request is actually changed, so no.
>
>     Q: Is the initial description considered a comment on the issue?
>
> I *think* this is how DC treats it, but I'm not sure.
>
>     Q: Can anyone add comments to issues?
>
> I think that would be useful to allow, though maybe optional.  I could
> imagine it would be better in some cases to turn it off because it would
> just encourage people to submit comments that no one is going to read.
>
>
> --
> Ian Bicking  |  http://blog.ianbicking.org  |
> http://topplabs.org/civichacker
>
>
> --
> Archive: http://lists.open311.org/[…]/1245878335482<http://lists.open311.org/discuss/archive/2009/06/1245878335482>
> To unsubscribe send an email with subject "unsubscribe" to
> discuss@.... Please contact
> discuss-manager@... for questions.
>



-- 
Have you seen my new startup for social good? SeeClickFix (
www.seeclickfix.com) Check it out and report an issue in your neighborhood.
Power to the Community.

    Attachments

  • From: Jeff Hammel

    Subject: Re: SeeClickFix API Feedback

    Date: Jul 06, 2009 05:07 PM 

    Feedback on feedback and some comments of my own:

On Wed, Jun 24, 2009 at 04:18:33PM -0500, Ian Bicking wrote:
>    I looked at the SeeClickFix API a bit, and I wasn't sure where to send
>    feedback, but since Ben Berkowitz is on this list I figure I'll just send
>    it here.
>    Feedback reading from
>    the�TOP�of�the�API�document�([1]http://seeclickfix.com/simple_pages/23) to
>    the bottom:
> 
>    1. I think it would be valuable to focus more on the service request
>    document. �This is essentially the core document for the API, and focusing
>    on it is key to the RESTfulness of the
>    API.��I�mentioned�in�another�email�([2]http://lists.open311.org/discuss/archive/2009/06/1245703774174/forum_view)
>    how Atom entries are a potential way of representing requests, but even if
>    the document is JSON that's fine. �You might even give it a mimetype, like
>    application/x-municipal-service-request+json... but I dunno, maybe not.
> 
>    2. I don't think much is accomplished with a dual XML/JSON API. If the XML
>    and JSON aren't equivalent, of course you end up with all sorts of
>    nastiness. �If they are just two representations of the same thing, then
>    you have to deal with some invisible "real" record that has two views. �I
>    think it's easier for everyone if you just have a canonical
>    representation. �JSON and XML are also extensible in different ways, and
>    it could get confusing. �In JSON you typically add a new key to an object
>    (aka hash table) where in XML you use namespaces and tags. �There are some
>    attempts to represent XML style extensions in JSON, but the feel like a
>    lot of trouble to go through just to create something that feels terribly
>    awkward. �From what I can tell, XML is only proposed but not specified? �I
>    assume extensibility will be important, as mapping the API to existing
>    systems or novel use cases will require additional fields.

Since the original format isn't in JSON anyway (assumedly it is data in some sort of database), and converting to JSON is a markup anyway (even if it is simplejson.dumps in python), I don't know how having two views is bad.  I think they are fairly interchangeable, given parameters for how you want them to be intercahnged.  For instance, I wrote a simple function that will do something like this (not for production, just for fun):

def json2xml(dictionary, buffer=None):
    if buffer is None:
        buffer = StringIO()
    for key, value in dictionary.items():
        if isinstance(value, basestring):
            print >> buffer, '<%s>%s</%s>' % (key, value, key)
            continue
        if value is None:
            print >> buffer, '<%s/>' % key
            continue
        if isinstance(value, dict):
            print >> buffer, '<%s>' % key
            json2xml(value, buffer)
            print >> buffer, '</%s>' % key
        else:
            # iterables                                                         
            for item in value:
                json2xml({key: item}, buffer)
    return buffer.getvalue()

I don't see any compelling reason to use XML over JSON (or vice versa for that matter), but there are good tools for either.  If clients want both, I say, why not?

>    3. Locale seems useful, but also seems a bit tricky... I'm guessing some
>    fields are potentially based on some translatable underlying resource?
>    �For instance, "Phone: +1 (123) 456 - 7890" -- if that's based on some
>    structured record you could translate "Phone". �But if it's just a text
>    field then it's not translatable. �Seems like it would be tricky on the
>    API level. �Though one thing in the DC 311 API is the service request type
>    definitions, which has a bunch of fields with questions (like
>    "'Is�there�damage�to�the�vehicle?") that clearly should be translatable.
>    �Essentially this aspect of the DC API is a description of their schema
>    for different kinds of requests.

Yeah, I think locale at this level would be very tricky.  Useful, but tricky.

>    4. Creating tickets is done with PUT /v1/issue -- it should be POST. �PUT
>    should go to the actual URL, like PUT /v1/issue/my-issue -- but when you
>    create an issue you don't know what URL it will go to. �Atompub goes over
>    this:
> 
>    Request:
>    ��POST /container
>    ��Content-Type: application/atom+xml; type=entry
> 
>    ��<entry>...</entry>
> 
>    Response:
>    ��201 Created
>    ��Location: /container/id-created-by-server
>    ��Content-Type: application/atom+xml; type=entry
> 
>    ��<entry>...</entry>
> 
>    The response contains the entry after any changes made by the server. �The
>    server might clean HTML for instance, or reject elements that it doesn't
>    understand. �The location where the entry ends up is given in the Location
>    field. �I'd recommend the same pattern. �You might want to use the pattern
>    of a self-URL embedded in the entry itself. �In Atom this is <link
>    rel="self" href="..." />, in JSON you could just use a "self" key.
>    The specification as it is doesn't make it that clear what it accepts. �It
>    mentions some variables, but it's not clear if the body of the request is
>    a JSON document with those keys?

Yeah, I noticed this too.  No point in using PUT here.

>    5. It would probably be useful to give the locale of the request. �E.g.,
>    if someone submits a request in Spanish, to say that somewhere (maybe so
>    it can be routed to a Spanish-speaking dispatcher). �It doesn't seem like
>    locale means quite this.
> 
>    6. There should be ways to track who submitted the issue, in some way that
>    is essentially private to the software submitting the issue. �If there's
>    abuse through some frontend, the person managing that frontend might have
>    their own ways of identifying the problem.
> 
>    7. Atom-style globally unique identifiers could be useful when routing
>    requests. �One use case we've considered is a service that accepts
>    possibly incomplete requests, and corrects and routes them (perhaps
>    partially crowdsourced). �For instance, we're thinking about a service for
>    collecting bike rack requests, which might have no direct interaction with
>    the 311 database. �Anyway, I can imagine tickets going into systems and
>    then coming out (like if they are misfiled); having a globally unique and
>    stable ID makes it easier to see duplicates when this happens. �The only
>    tricky part then is that this ID has to be carried around with the
>    request, instead of using purely internal IDs. �Atom doesn't require that
>    the globally unique ID show up in any URL.
> 
>    8. The callback URI stuff is confusing. �It shows up in several places,
>    but not entirely consistently.

Confusing, yes, and I do get the feeling that its not really thought out. On the other hand, I find it very compelling.

>    9. Why does GET /v1/issue/<id> accept request parameters?
> 
>    10. For GET /v1/tags, a description of tags would be helpful. �Also an
>    indication about whether freeform tags are allowed. �For instance, DC's
>    service request types seem mappable to tags. �I could imagine it returning
>    a structure like {"S0000": {"description": "Abandoned vehicle"}}. �It
>    seems like making the value an object leaves useful room for extension. �I
>    would assume that tags should not be translated, but the descriptions
>    could be.
> 
>    11. GData does queries with URLs like /v1/issues/-/tag1/tag2 (where all
>    tags are intersections). �The - avoids some potential conflicts. �I don't
>    have strong opinions about it, though, just noting what GData does.
> 
>    12. Not sure why /v1/search and /v1/find are different. �Can't it just be
>    the intersection of the queries you perform?

I dislilke this too.

>    13. Find should support polygons and the like as well. �Some cities are
>    not aligned with North (e.g., Manhattan), so a box isn't a good fit for
>    any normal selection. �Even a point and radius is probably a more natural
>    search. �(I wonder if OpenSearch has anything to suggest ways of
>    describing the available search parameters?)
> 
>    14. Dates are useful for queries, lets you see updates.
> 
>    15. Is there paging for large results? �This also applies to GET
>    /v1/issue.
> 
>    16. There's nothing feed-like. �Feeds would be handy. �An actual feed
>    would be handy.

An RSS view?  Maybe the XML is in fact RSS or at least optionally so?

>    Then the open questions:
> 
>    �� �Q: How do we handle API keying?
>    ��� Q: Is it required or optional?
> 
>    I'd imagine it could be optional at least for some parts of the API, that
>    only relate to reading.
 
IMHO, the API keyring is a poor excuse for auth.

>    �� �Q: Should there be an automated request/response mechanism in the API?
> 
>    Not sure what this means.

Nor I ;)

>    �� �Q: Are providers allowed to rate limit clients? If so are there
>    guidelines?
> 
>    They always could, of course -- clients should never rely on the server
>    always accepting the request. �I thought there was a good HTTP response
>    code for this, but I can't find one.
> 
>    �� �Q: Should all messages return status 200?
> 
>    Definitely no!

Hah!

>    �� �Q: Should all error messages be human readable?
> 
>    Not sure. �The Content-Type of the error response should at least indicate
>    the possibilities (i.e., text/html or text/plain are directly human
>    readable, and application/json is definitely not). �It might be useful to
>    define a JSON response format, similar to JSON-RPC's.
> 
>    �� �Q: Should all error messages be localized to the requested or default
>    locale?
> 
>    Mostly the error messages are for the developers, not for the users
>    interacting with some service. �So I think the requested locale isn't that
>    applicable.
> 
>    �� �Q: Would supporting client based request identifiers that allow
>    clients to replay requests without repetitive effects be worth while?
> 
>    You mean like stable IDs, like I mention in 7? �I think so.
> 
>    �� �Q: How long is the provider expected to track these
> 
>    If it is handled like Atom, indefinitely. �This doesn't seem too hard to
>    do, it's just another field for requests.
> 
>    �� �Q: May providers refuse service at their discretion?
> 
>    Of course they always can do that, whether we want them to or not, and it
>    can happen unintentionally at any time (e.g., server outage).
> 
>    �� �Q: How does a service provider reject PUT/POST/DELETE? Do rejects
>    trigger callbacks?
> 
>    400 Bad Request, or 405 Method Not Allowed. �Callbacks should only trigger
>    if the service request is actually changed, so no.
> 
>    �� �Q: Is the initial description considered a comment on the issue?
> 
>    I *think* this is how DC treats it, but I'm not sure.

>    �� �Q: Can anyone add comments to issues?
> 
>    I think that would be useful to allow, though maybe optional. �I could
>    imagine it would be better in some cases to turn it off because it would
>    just encourage people to submit comments that no one is going to read.

I think this is an auth/auth problem, which is a whole different scope, I feel, and deserves its own discussion.

I'll put up additional notes tomorrow.  Bye for now!

>    --
>    Ian Bicking �| �[3]http://blog.ianbicking.org �|
>    �[4]http://topplabs.org/civichacker
> 
>    --
>    Archive: [5]http://lists.open311.org/[...]/1245878335482
>    To unsubscribe send an email with subject "unsubscribe" to
>    [6]discuss@.... Please contact
>    [7]discuss-manager@... for questions.
> 
> References
> 
>    Visible links
>    1. http://seeclickfix.com/simple_pages/23
>    2. http://lists.open311.org/discuss/archive/2009/06/1245703774174/forum_view
>    3. http://blog.ianbicking.org/
>    4. http://topplabs.org/civichacker
>    5. http://lists.open311.org/discuss/archive/2009/06/1245878335482
>    6. mailto:discuss@...
>    7. mailto:discuss-manager@...

    • From: Ian Bicking

      Subject: Re: SeeClickFix API Feedback

      Date: Jul 06, 2009 05:22 PM 

      On Mon, Jul 6, 2009 at 4:07 PM, Jeff Hammel <jhammel@...> wrote:

> Since the original format isn't in JSON anyway (assumedly it is data in
> some sort of database), and converting to JSON is a markup anyway (even if
> it is simplejson.dumps in python), I don't know how having two views is bad.
>  I think they are fairly interchangeable, given parameters for how you want
> them to be intercahnged.  For instance, I wrote a simple function that will
> do something like this (not for production, just for fun):
>
> def json2xml(dictionary, buffer=None):
>    if buffer is None:
>        buffer = StringIO()
>    for key, value in dictionary.items():
>        if isinstance(value, basestring):
>            print >> buffer, '<%s>%s</%s>' % (key, value, key)
>            continue
>        if value is None:
>            print >> buffer, '<%s/>' % key
>            continue
>        if isinstance(value, dict):
>            print >> buffer, '<%s>' % key
>            json2xml(value, buffer)
>            print >> buffer, '</%s>' % key
>        else:
>            # iterables
>            for item in value:
>                json2xml({key: item}, buffer)
>    return buffer.getvalue()
>
> I don't see any compelling reason to use XML over JSON (or vice versa for
> that matter), but there are good tools for either.  If clients want both, I
> say, why not?
>

Well, extension still feels like the trickiest case.  A clear extension is
the DC 311 service requests, which has specific questions.  For instance, if
you report a dead animal, you are supposed to answer the question "What kind
of animal is it?".  They happen to give this question the code DEDANM-KIND.
 In JSON this might be:

{"summary": "There's a dead animal in the street and I swear I saw some rats
eating it",
 "DEDANM-KIND": "A dog, so sad",
 ...}

In XML this might be:

<servicerequest
 xmlns="http://open311-foundation.org/"
 xmlns:dc="http://api.dc.gov/311">
  <summary>
    There's a dead animal in the street and I swear I saw some rats eating
it
  </summary>
  <dc:DEDANM-KIND>A dog, so sad</dc:DEDANM-KIND>
</servicerequest>

They look similar, but you do have to add in that xmlns to match XML
conventions.  Also, if you use something other than strings in JSON it's not
entirely equivalent to specify that in XML.  Or worse, if you allow nested
structures in *either* one, they are not equivalent to each other (there's
nothing in JSON that clearly maps to XML attributes, for example).

>    16. There's nothing feed-like.  Feeds would be handy.  An actual feed
> >    would be handy.
>
> An RSS view?  Maybe the XML is in fact RSS or at least optionally so?
>

If using XML, Atom entries are a completely reasonable container for service
requests.  Feeds would fall out pretty easily.  Though you lose some of the
benefit when you create extension elements, as standard feed readers don't
pay any attention to those... so either you cram extra information into the
body of the feed item (like the location) or most people don't see or
realize that the extra information exists.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker

      Attachments

  • From: Jeff Hammel

    Subject: Re: SeeClickFix API Feedback

    Date: Jul 08, 2009 12:38 PM 

    Other notes:

   /versions
   Description

   This base resource is intended to be used to provide clients with
   information on which versions of the Open311 API the provider supports.
   GET
   Accepts:
   -no parameters-
   Returns:
       versions: List of supported API versions

   Example
   Request:
       http://api.provider.gov/open311/info.json

Um, I assume this is supposed to be "http://api.provider.gov/open311/versions.json" ?

Also, I dislike the . syntax.  Its actually less RESTful, IMHO, than http://api.provider.gov/open311/versions?format=json . Though some might disagree

I find the /v1/ syntax a bit of an over-design.  Why would you be running more
than one version on a server?  it seems like possibly useful for debugging....
never useful for production (notably, the DC API also has this....dunno why this is needed at the API level). Am I misunderstanding something?

/v1/issue
   PUT
   Returns:
       Success
           Same as GET to http://<root>/v1/issue/<id>

Should be a redirect.  Maybe that's what they mean?

   /v1/issue/<id>
   GET
   Accepts:
       [locale]
           summary
               description
       tags

       callback URI
           display name

Not sure what "display name" is.  Or callback URI?

So this is basically the same as /<trac project>/ticket/<id>

   /v1/tag
   Accepts:
       [locale]: The desired language locale for the response.

       [tag]: Some or no portion of a tag

Is '?tag=q' -> 'tags that match "q"'?

   Returns:
       Success
           Ordered list of suggested tags
       Failure
               Locale

           Error Description

Is this querying for resources tagged in a certain way?  I'm not sure from thisdescription.  So it returns URIs/URLs?

   /v1/callback/<source URI>/<resource>...

   This is the resource that will be called by a client if that client was
   a provider that had a callback registered with it for the provider URI
   that it is now calling.
   To register a call back the caller MUST provide a URI. This URI MUST
   refer to a Open311 compliant provider system. If the service provider
   accepts the registration of a callback URI on a service request, the
   provider MUST make calls to the URI when state on that service request
   changes.

I think this is really cool.  A hearty +1.

   /v1/search
   Accepts:
       [locale]

       query: The query string

Well, the query string is a bit vague.  A bit *real* vague.

   Authorization
   Q: How do we handle API keying?

Why bother?

    Q: Should there be an automated request/response mechanism in the
   API?

Not sure what this means.

I think Ian covered all my other real concerns.

Jeff

On Wed, Jun 24, 2009 at 04:18:33PM -0500, Ian Bicking wrote:
>    I looked at the SeeClickFix API a bit, and I wasn't sure where to send
>    feedback, but since Ben Berkowitz is on this list I figure I'll just send
>    it here.
>    Feedback reading from
>    the�TOP�of�the�API�document�([1]http://seeclickfix.com/simple_pages/23) to
>    the bottom:
> 
>    1. I think it would be valuable to focus more on the service request
>    document. �This is essentially the core document for the API, and focusing
>    on it is key to the RESTfulness of the
>    API.��I�mentioned�in�another�email�([2]http://lists.open311.org/discuss/archive/2009/06/1245703774174/forum_view)
>    how Atom entries are a potential way of representing requests, but even if
>    the document is JSON that's fine. �You might even give it a mimetype, like
>    application/x-municipal-service-request+json... but I dunno, maybe not.
> 
>    2. I don't think much is accomplished with a dual XML/JSON API. If the XML
>    and JSON aren't equivalent, of course you end up with all sorts of
>    nastiness. �If they are just two representations of the same thing, then
>    you have to deal with some invisible "real" record that has two views. �I
>    think it's easier for everyone if you just have a canonical
>    representation. �JSON and XML are also extensible in different ways, and
>    it could get confusing. �In JSON you typically add a new key to an object
>    (aka hash table) where in XML you use namespaces and tags. �There are some
>    attempts to represent XML style extensions in JSON, but the feel like a
>    lot of trouble to go through just to create something that feels terribly
>    awkward. �From what I can tell, XML is only proposed but not specified? �I
>    assume extensibility will be important, as mapping the API to existing
>    systems or novel use cases will require additional fields.
> 
>    3. Locale seems useful, but also seems a bit tricky... I'm guessing some
>    fields are potentially based on some translatable underlying resource?
>    �For instance, "Phone: +1 (123) 456 - 7890" -- if that's based on some
>    structured record you could translate "Phone". �But if it's just a text
>    field then it's not translatable. �Seems like it would be tricky on the
>    API level. �Though one thing in the DC 311 API is the service request type
>    definitions, which has a bunch of fields with questions (like
>    "'Is�there�damage�to�the�vehicle?") that clearly should be translatable.
>    �Essentially this aspect of the DC API is a description of their schema
>    for different kinds of requests.
> 
>    4. Creating tickets is done with PUT /v1/issue -- it should be POST. �PUT
>    should go to the actual URL, like PUT /v1/issue/my-issue -- but when you
>    create an issue you don't know what URL it will go to. �Atompub goes over
>    this:
> 
>    Request:
>    ��POST /container
>    ��Content-Type: application/atom+xml; type=entry
> 
>    ��<entry>...</entry>
> 
>    Response:
>    ��201 Created
>    ��Location: /container/id-created-by-server
>    ��Content-Type: application/atom+xml; type=entry
> 
>    ��<entry>...</entry>
> 
>    The response contains the entry after any changes made by the server. �The
>    server might clean HTML for instance, or reject elements that it doesn't
>    understand. �The location where the entry ends up is given in the Location
>    field. �I'd recommend the same pattern. �You might want to use the pattern
>    of a self-URL embedded in the entry itself. �In Atom this is <link
>    rel="self" href="..." />, in JSON you could just use a "self" key.
>    The specification as it is doesn't make it that clear what it accepts. �It
>    mentions some variables, but it's not clear if the body of the request is
>    a JSON document with those keys?
> 
>    5. It would probably be useful to give the locale of the request. �E.g.,
>    if someone submits a request in Spanish, to say that somewhere (maybe so
>    it can be routed to a Spanish-speaking dispatcher). �It doesn't seem like
>    locale means quite this.
> 
>    6. There should be ways to track who submitted the issue, in some way that
>    is essentially private to the software submitting the issue. �If there's
>    abuse through some frontend, the person managing that frontend might have
>    their own ways of identifying the problem.
> 
>    7. Atom-style globally unique identifiers could be useful when routing
>    requests. �One use case we've considered is a service that accepts
>    possibly incomplete requests, and corrects and routes them (perhaps
>    partially crowdsourced). �For instance, we're thinking about a service for
>    collecting bike rack requests, which might have no direct interaction with
>    the 311 database. �Anyway, I can imagine tickets going into systems and
>    then coming out (like if they are misfiled); having a globally unique and
>    stable ID makes it easier to see duplicates when this happens. �The only
>    tricky part then is that this ID has to be carried around with the
>    request, instead of using purely internal IDs. �Atom doesn't require that
>    the globally unique ID show up in any URL.
> 
>    8. The callback URI stuff is confusing. �It shows up in several places,
>    but not entirely consistently.
> 
>    9. Why does GET /v1/issue/<id> accept request parameters?
> 
>    10. For GET /v1/tags, a description of tags would be helpful. �Also an
>    indication about whether freeform tags are allowed. �For instance, DC's
>    service request types seem mappable to tags. �I could imagine it returning
>    a structure like {"S0000": {"description": "Abandoned vehicle"}}. �It
>    seems like making the value an object leaves useful room for extension. �I
>    would assume that tags should not be translated, but the descriptions
>    could be.
> 
>    11. GData does queries with URLs like /v1/issues/-/tag1/tag2 (where all
>    tags are intersections). �The - avoids some potential conflicts. �I don't
>    have strong opinions about it, though, just noting what GData does.
> 
>    12. Not sure why /v1/search and /v1/find are different. �Can't it just be
>    the intersection of the queries you perform?
> 
>    13. Find should support polygons and the like as well. �Some cities are
>    not aligned with North (e.g., Manhattan), so a box isn't a good fit for
>    any normal selection. �Even a point and radius is probably a more natural
>    search. �(I wonder if OpenSearch has anything to suggest ways of
>    describing the available search parameters?)
> 
>    14. Dates are useful for queries, lets you see updates.
> 
>    15. Is there paging for large results? �This also applies to GET
>    /v1/issue.
> 
>    16. There's nothing feed-like. �Feeds would be handy. �An actual feed
>    would be handy.
> 
>    Then the open questions:
> 
>    �� �Q: How do we handle API keying?
>    ��� Q: Is it required or optional?
> 
>    I'd imagine it could be optional at least for some parts of the API, that
>    only relate to reading.
> 
>    �� �Q: Should there be an automated request/response mechanism in the API?
> 
>    Not sure what this means.
> 
>    �� �Q: Are providers allowed to rate limit clients? If so are there
>    guidelines?
> 
>    They always could, of course -- clients should never rely on the server
>    always accepting the request. �I thought there was a good HTTP response
>    code for this, but I can't find one.
> 
>    �� �Q: Should all messages return status 200?
> 
>    Definitely no!
> 
>    �� �Q: Should all error messages be human readable?
> 
>    Not sure. �The Content-Type of the error response should at least indicate
>    the possibilities (i.e., text/html or text/plain are directly human
>    readable, and application/json is definitely not). �It might be useful to
>    define a JSON response format, similar to JSON-RPC's.
> 
>    �� �Q: Should all error messages be localized to the requested or default
>    locale?
> 
>    Mostly the error messages are for the developers, not for the users
>    interacting with some service. �So I think the requested locale isn't that
>    applicable.
> 
>    �� �Q: Would supporting client based request identifiers that allow
>    clients to replay requests without repetitive effects be worth while?
> 
>    You mean like stable IDs, like I mention in 7? �I think so.
> 
>    �� �Q: How long is the provider expected to track these
> 
>    If it is handled like Atom, indefinitely. �This doesn't seem too hard to
>    do, it's just another field for requests.
> 
>    �� �Q: May providers refuse service at their discretion?
> 
>    Of course they always can do that, whether we want them to or not, and it
>    can happen unintentionally at any time (e.g., server outage).
> 
>    �� �Q: How does a service provider reject PUT/POST/DELETE? Do rejects
>    trigger callbacks?
> 
>    400 Bad Request, or 405 Method Not Allowed. �Callbacks should only trigger
>    if the service request is actually changed, so no.
> 
>    �� �Q: Is the initial description considered a comment on the issue?
> 
>    I *think* this is how DC treats it, but I'm not sure.
> 
>    �� �Q: Can anyone add comments to issues?
> 
>    I think that would be useful to allow, though maybe optional. �I could
>    imagine it would be better in some cases to turn it off because it would
>    just encourage people to submit comments that no one is going to read.
>    --
>    Ian Bicking �| �[3]http://blog.ianbicking.org �|
>    �[4]http://topplabs.org/civichacker
> 
>    --
>    Archive: [5]http://lists.open311.org/[...]/1245878335482
>    To unsubscribe send an email with subject "unsubscribe" to
>    [6]discuss@.... Please contact
>    [7]discuss-manager@... for questions.
> 
> References
> 
>    Visible links
>    1. http://seeclickfix.com/simple_pages/23
>    2. http://lists.open311.org/discuss/archive/2009/06/1245703774174/forum_view
>    3. http://blog.ianbicking.org/
>    4. http://topplabs.org/civichacker
>    5. http://lists.open311.org/discuss/archive/2009/06/1245878335482
>    6. mailto:discuss@...
>    7. mailto:discuss-manager@...
Previous thread Next thread