September 2015. Visualizing commodity price data.
- Partner: TBD
- Start date: Sep 2015
- Status: Converting .csv files into a .json. Building a prototype viz in
MetricsGraphics.js
.
The .dbf
files are in the following format:
SORTCOD | CTYP | DIV | DTE | WEK | ITM01 | PRA01 | PRB01 | ... | ITM35 | PRA35 | PRB35 |
---|---|---|---|---|---|---|---|---|---|---|---|
int | int | int | yyyy-mm-dd | int | int | int | int | ... | int | int | int |
where
- SORTCOD = ?
- CTYP = commodity type?
- DIV = geographic region/division?
- DTE = date
- WEK = ?
- ITM = the commodity
- PRA = min price
- PRB = max price
So we need to reshape
it from wide to long. Final, desired output would be a JSON array of commodities, where each commodity is an object with date-price data.
Currently:
{
"Bahawalpur": {
"(item 1)": [{"date": x, "price": y},...,{"date": x, "price": y}],
"(item 2)": [{"date": x, "price": y},...,{"date": x, "price": y}]
},
"D.G. Khan": { ... }
}
Goal:
[
{
"region": "Bahawalpur",
"items": [
{"(item 1)": [[{"date": x, "price": y},...,{"date": x, "price": y}]},
{"(item 2)": [[{"date": x, "price": y},...,{"date": x, "price": y}]},
]
}, {
"region": "D.G. Khan",
"items": [
{"(item 1)": [[{"date": x, "price": y},...,{"date": x, "price": y}]},
{"(item 2)": [[{"date": x, "price": y},...,{"date": x, "price": y}]},
]
}
]
Getgit
set up.ConvertSee.dbf
to.csv
.cleaning.py
.Does that nested JSON make sense?ConvertSee.csv
into nested.json
.json-prep.html
for the actual conversion.First go at MetricsGraphics.js.Seedemo.html
.Is the JSON in the right format for MetricsGraphics?alldata.json
: convert object into array of objects?alldata.json
:"item": d.itm
?Make pretty.Sort of pretty.- Standardize axes?
- Region filter functionality.
- Error/alert for missing data.