User extensibility of Taxbrain

[feenberg]

In the existing online TAXSIM calculator, users can submit datasets, and taxes
are calculated and returned. While the user can't change the tax law, she can
change the data. So, for example, if she wants to simulate a secondary earner
credit of 10%, she would multiply the earnings of the lessor earning spouse by
.9 before submitting the dataset. In practice this accomodates many but
certainly not all tax proposals. If users write to me and ask for something
different, I usually code it as an option they can turn on. I have done this
many times, but the modifications are usually simple and take no more than a few
minutes. You can see a list at

 http://www.nber.org/taxsim/plans.html [www.nber.org]

You can see that many of them are things that the OSPC calculator can already do
easily but some are not. TAXSIM can do a few things TB can not.

We want to allow users to do as much of this themselves as possible, perhaps
without installing and modifying the tax calculator. Most users will not be
strong programmers and the Python calculator is an intimidating system.

In the current context this would mean allowing the user to specify a
transformations of input variables before the tax calculator acts upon PlanY and
transformations of intermediate variables after they are calculated and before
they are used in subsequent calculations.We would have to supply hooks that the
user could replace at various points in the calculator.  Just how general the
allowed transformation would be depends on our skill at keeping things secure,
but hopefully something of the form:

     evalue = arithmetic_expression_involving_evalues_and_constants

would be the minimum, and perhaps much more. Max and min would have to be
allowed, perhaps some other built-in functions. It depends on what can be done
to ensure security is maintained. Once this facility is available, it could be
applied at several other points in the calculation, including transformations to

 c00100 agi
 c04470 itemized deductions
 c04800 taxable income
 c05800 income tax
 c62100 minimum taxable income

where in each case the native PlanY calculation of the value would be modified
by the application of the user supplied formula, for example:

 c00100 = c00100 - min(100000,.1*e00200)

would be an adjustment of 10% of wages up to 10,000 applied to AGI before the
taxable income was calculated and

 c05800 = c05800 + .05*max(0,(c04800-250000))

It would be marvelous if more general code could be allowed, especially "if"
conditions, as tax proposals are usually dense with if conditions. Sometimes
those can be translated to min or max, but not always easily.

Also some way to index dollar constant dollar values used as thresholds. I might
suggest that we define a function to index dollar values and suggest it be used:

 c00100 = c00100 - min(real(100000),.1*e00100)

where the base year is whatever is the first year of output in our calculator.

I am not familiar with how isolated a Python routine can be so I can't say how
much it is possible to do. I certainly don't suggest writing a parser for a new
language. But if the expression or code can be confirmed to have no side
effects, and only E-values of a single taxpayer can be input or output
parameters, we should be able to do a lot.

Where ever I have mentioned e-values above, I mean to include codes such as FDED
or XOCAH.

I image we would offer a text box for the code, and a selection box for which
hook it would attach to.

[MattHJensen]

This could be an extremely useful enhancement. A problem we are going to have in the future is that tax reform options will proliferate on the TaxBrain input page. We are in the process of adding yet child tax credit for instance. Yes, this will be an "additional additional child tax credit"...

If there were user hooks like what @feenberg is proposing, then we could have the TaxBrain input page hew pretty closely to current law and allow for parameter-value deviations from current law through the input fields. To make a more substantial reforms, a user would write a little script that would be entered at one of the hooks. Overtime, our users will develop a library of "reform scripts" that can be shared and reviewed by everyone.

For an assessment of how doable this all is, however, I defer to the expertise of Mr. @talumbau.

[martinholmer]

@MattHJensen, Have you considered the pros and cons of allowing TaxBrain users to simply upload a JSON policy reform file (like the ones that have been tested and used in the TAXSIM validation work)?

[zrisher]

@martinholmer said:

Have you considered the pros and cons of allowing TaxBrain users to simply upload a JSON policy reform file (like the ones that have been tested and used in the TAXSIM validation work)?

That would ease the problem of form field proliferation on webapp, but doesn't accommodate the custom logic that @feenberg proposes.

Obviously anyone using the tax calculator locally can just extend the existing calculation classes and methods. But for webapp users, providing fully customizable logic requires two things:

• hooks in each of the relevant areas (e.g. before and after each year's regular calculation)
• a fully secured sandbox environment in which to run the code. This is hard. From what I've read, it seems like running PyPy's sandbox in a separate VM might be the only fully secure way. Some good points of reference:
  • http://doc.pypy.org/en/latest/sandbox.html
  • https://mail.python.org/pipermail/python-dev/2013-November/130132.html
  • http://programmers.stackexchange.com/questions/191623/best-practices-for-execution-of-untrusted-code

If we decided the latter point was too difficult, you could come up with a custom pseudo script that we parse and turn into python code with a small list of operations, conditions, and variables as @feenberg suggests. Also nontrivial, though.

Another possibility is having a form for people to suggest changes to taxcalc that the existing contributors could code up and enable toggling through a reform parameter. That continues the unchecked growth of the reform file, but may lead to an improved understanding of reform logic over time that would help us transform it into a very generalized form.

It would be pretty cool to see a framework for specifying tax plans that could be freely edited, shared, and run from the web. But any way you approach it, it's a big project.

[MattHJensen]

That would ease the problem of form field proliferation on webapp, but doesn't accommodate the custom logic that @feenberg proposes.

This is my understanding as well.

[MattHJensen]

It would be pretty cool to see a framework for specifying tax plans that could be freely edited, shared, and run from the web. But any way you approach it, it's a big project.

I agree. John O'Hare has called this the "Holy Grail" of tax modeling user interfaces.

As @zrisher suggests above, and @talumbau has suggested before also, the way we would probably do this is two steps:

• hooks in each of the relevant areas
• a fully secured sandbox environment in which to run the code. This is hard.

I've been thinking that a good first step might be to start with the first step. Then we could see how we like it before doing the hard part.

I believe this would entail building hooks into the variables that @feenberg suggested ( c00100 agi, c04470 itemized deductions, c04800 taxable income, c05800 income tax, c62100 minimum taxable income) in calculate.py.

We would likely want to store the snippets in a new script in the taxcalc package, and the application of those snippets could be controlled by toggle parameters added to params.json.

We could then try moving existing reforms that are currently parameterized in Tax-Calculator but are nonparameteric w.r.t to the existing tax code from functions.py to the new "reform-snippet" script. I expect we could relatively easily move haircuts and various new credits like the "additional-additional child tax credit" out of functions.py to the new script. (There are reforms we couldn't move, like the BenefitSurtax.)

If we like this style of doing things and we think that writing similar snippets could be intuitive for TaxBrain users, then we could move to the next step.

• a fully secured sandbox environment in which to run the code. This is hard.

[martinholmer]

It is good to see some discussion of issue #429 after a hiatus of eight months.

@zrisher said on July 30, 2016:

@martinholmer said [on November 18, 2015]:

Have you considered the pros and cons of allowing TaxBrain users to simply upload a JSON policy reform file like the ones that have been tested and used in the [inctax.py script or the simtax.py script used in] TAXSIM validation work?

That would ease the problem of form field proliferation on TaxBrain, but doesn't accommodate the custom logic that @feenberg proposes [in issue #429].

Obviously anyone using the tax calculator locally can just extend the existing calculation classes and methods.

The above exchange highlights a number of issues that we should be keeping in mind.

First, policy reforms can be parametric reforms or structural reforms. Parametric reforms are those that can be simulated with the current-version of Tax-Calculator by specifying alternative values for existing policy parameters. Structural reforms are those that are not parametric reforms and require new Tax-Calculator logic to simulate. Note that the reforms able to be specified in the TaxBrain web application are now a subset of Tax-Calculator parametric reforms.

The discussion of issue #429 is focused on expanding the range of policy reforms that can be specified and simulated by TaxBrain users, but our conversation should be broader: it should focus on how to expand the range of reforms that can be simulated not only with TaxBrain but also by Tax-Calculator (to include some structural reforms).

One TaxBrain issue is the messy proliferation of checkboxes and reform parameters that are needed to specify particular parametric reforms. The problem with adding these to TaxBrain (as envisioned in issue #229) is not only the messiness, but also the fact that today's hot reform is tomorrow's stale reform. So, adding many parametric reform parameters to TaxBrain is not only a lot of work, but the work is never ending as new reform ideas replace old reform ideas.

Last November I suggested what I still consider to be an effective and relatively easy solution to this part of the overall problem: allowing TaxBrain users to simply upload a JSON policy reform file in the format used by the inctax.py and simtax.py interfaces to Tax-Calculator. We could even add to TaxBrain a library of such reform files for users to use as is or as starting points for specifying their own parametric reform. Note that these reform files are JSON, so no Python programming skills are required for a user to write them. But the biggest advantage of this approach is that it is much easier to implement than any way of handling structural reforms: Tax-Calculator can already handle these JSON reform files and it would seem to be only a modest effort to get TaxBrain to be able to handle them as an alternative way of specifying a reform.

This enhancement would not solve the problem of finding a way to handle structural reforms, but it would solve part of the problem being discussed here in issue #429. Why don't we walk before we try to run? Why not solve part of the problem with a relatively easy first step. Who knows, the team may even learn something in this first step that will be helpful in taking subsequent steps. Those subsequent steps will involve major changes to Tax-Calculator, while this first step requires no changes to Tax-Calculator.

@MattHJensen @feenberg @zrisher @talumbau @Amy-Xu @GoFroggyRun

[MattHJensen]

@martinholmer, would you mind opening an issue in the webapp-public repo describing the JSON reform uploading capability, including a description of how it could be included in the TaxBrain user interface?

I agree that the JSON reform capability doesn't solve the core issue raised by @feenberg in this issue -- the desire for users to be able to make structural reforms without needing to interact with the Tax-Calculator source code -- so I don't think we should close this issue yet.

But the JSON reform capability would solve other problems, some of which @martinholmer has already mentioned:

1. We could make new TC features available to TB users much more quickly because we wouldn't need to edit the TB user interface.
2. For many users, a JSON reform file might be a better format for archiving their past runs than a TB link.
3. We may decide some TC capabilities aren't worth the TB clutter, but we still want them available to power users w/o the puf.

[martinholmer]

This is a progress report on open issue #429, "User extensibility of TaxBrain":

During November 2016, TaxBrain began accepting an uploaded JSON reform file as an alternative way to specify a tax reform. This feature seems to work well and is now being used by at least one Congressional staff person to analyze reforms that are not able to be specified (yet?) on the main TaxBrain page. Currently, there is no way to upload a file containing behavioral-response parameters or economic growth parameters, so I'm going to work on that enhancement.

So far, there has been no visible progress on adding the ability to specify code (rather than parameter values) as a way to specify tax reforms. I stress "visible" progress because I know that several people are actively thinking about the remaining issues in #429.

@MattHJensen @feenberg @Amy-Xu @GoFroggyRun @andersonfrailey @codykallen @zrisher

[martinholmer]

Merge of pull request #1081 provides a template for resolving issue #429.
Suggestions about how to extend the #1081 approach should be put in a new issue.