Socrata API doesn't support full history of dataset modifications

[mshenfield]

The Socrata API only returns the date of last modification for each row, which is unreliable for a complete picture of the dataset. I've posed the question on Stack Overflow, which has more details about the options available and where they falls short. The endpoint we are using http://fillInSocrataPortal/data.json also only gives the latest modification date.

We can get for each row the date it was last modified. According to Socratathis won't provide an accurate history, different methods of updating may re-load the entire dataset even if only a few rows were modified/added/deleted. This may be a roadblock to the enhancements in #12 .

[jchapin]

@mshenfield, lets talk about this one. We may have to build the app to poll the Socrata API on a daily basis and store information in our app's database to get an accurate picture of the updates. The future release of a new data set will likely have to be provided by information completely outside of the API.

[mshenfield]

@jchapin looks like it. I heard back from Socrata's developer team and they confirmed it's not in the API http://stackoverflow.com/questions/31643449/does-the-socrata-soda-api-support-getting-a-list-of-dates-on-which-the-dataset-w/31686589#31686589. Doesn't seem too bad. We run a daily GET on the all the data.json, and just append the Last_Date_Updated to a list associated with each brigade if it is different.