Add 2023 examination data from CBO, TSY, and JCT

[martinholmer]

Add FY data from CBO, TSY, and JCT, that can be converted to CY 2023 amounts, against which microdata estimates will be compared in the examination phase of the project.

[nikhilwoodruff]

Thanks Martin- but do we need to have these as .text and .awk files? It'd be more manageable if everything was a CSV (or even JSON/YAML).

[martinholmer]

@nikhilwoodruff asked in PR #10:

Thanks Martin- but do we need to have these as .text and .awk files? It'd be more manageable if everything was a CSV (or even JSON/YAML).

Why would it "be more manageable?

The .txt files are not rectangular, so CSV-format is not a good choice.
JSON and YAML files would need Python scripts to do the interpolation/extrapolation work, which is far more complex and time consuming.
So, the current setup is, from my point of view, quite manageable and economical for the task at hand.

[nikhilwoodruff]

@nikhilwoodruff asked in PR #10:

Thanks Martin- but do we need to have these as .text and .awk files? It'd be more manageable if everything was a CSV (or even JSON/YAML).

Why would it "be more manageable?

The .txt files are not rectangular, so CSV-format is not a good choice. JSON and YAML files would need Python scripts to do the interpolation/extrapolation work, which is far more complex and time consuming. So, the current setup is, from my point of view, quite manageable and economical for the task at hand.

Not sure I follow really. The data seems rectangular to me, or at least we could make it so. And there's a script already doing the interpolation in shell language- why is Python much more complicated?

[martinholmer]

@nikhilwoodruff said:

Not sure I follow really. The data seems rectangular to me, or at least we could make it so. And there's a script already doing the interpolation in shell language- why is Python much more complicated?

So, you have not answered my question:

Why would it "be more manageable"?

And @nikhilwoodruff also said:

there's a script already doing the interpolation in shell language

What shell language and which script?

[nikhilwoodruff]

So, you have not answered my question:
Why would it "be more manageable"?

Yes, well my thinking is that CSV files are easiest to manipulate/show in documentation/pull into programs across all languages in this project, and especially Python. Second to that, consistency probably helps us, so if everything is CSV/Python that'd be the easiest combination and avoid having to build logic to parse text files, or copy the values.

[nikhilwoodruff]

What shell language and which script?

Sorry- was referring to the AWK script.

[martinholmer]

@nikhilwoodruff, I changed the spaced-delimited text files to CSV files so that they "are easie[r] to manipulate/show in documentation/pull into programs across all languages in this project."