Data model: store extra data on the Proof model (location, date, currency, Google Cloud Vision ?)

[raphodn]

Idea

Currently we ask users to provide data each time they upload a price, such as its location, date & currency.
But this data is common for all prices for a given proof (at least for PRICE_TAG & RECEIPT proofs).

If we start storing this data on the proof directly, we can think about speeding-up price addition for the users.

And have data quality checks to verify that the proof prices share common data.

Questions

• we'll have duplicate data between a proof and its price(s). good idea ? what to do in the long run ?

Related issues

• Proof: store an optional date #322
• Proof: store an optional currency #336
• Proof: store an optional location #323
• Proof: send to Google Cloud Vision, and store the matching JSON output #320
• Proof: add new "consumption-related" fields on receipts: total amount, items count #387
• Data quality: rules & ideas #319

[raphodn]

@monsieurtanuki answering your comment here

My initial point was at the database level - I come from the world of relational databases where redundant data are not welcome.

You're right, I think at some point we'll have to lock the "duplicate" data on the Price field, to avoid changing them. Easy in the UI, harder if via the API. And maybe simply delete this data at some point ?
We'll have a "data quality" discussion in the next few weeks as this is deployed and tested by users.

[monsieurtanuki]

It would be interesting to check how incoherent the database currently is: percentage of proofs whose prices are affected to different locations or currencies.
Fixing incoherences.

Then maybe a v2 of:

• uploadProof (mandatory location and currency)
• addPrice (no more location nor currency, or a warning/error if they don't match the proof's)

Eventually, should you duplicate proof data at the individual price level, or just retrieve it for getPrices? That's a matter of pros/cons about space and speed optimization.

[raphodn]

Yes the v2 is exactly what i've almost finished implementing, at least in the web UI :)

And ideally it'll require some checks and rules in the backend as well, but it can come a bit after

[raphodn]

the "v2" changes have been deployed !

there's still some minor issues, such as proofs without prices so without location/date/currency during the migration :

• i made some changes in the web frontend to allow editing proof date & currency
• i need to do the same for proof location
• and disable price addition if all three of these info is not filled on the proof

[raphodn]

I linked a new issue - #387 - to start asking users some extra metadata : how many products, total price.
Think of anything else ? cc @serpico