Make Copies of Calculators Optional

[andersonfrailey]

Currently, Behavioral-Responses makes three copies of the calculator objects passed as arguments to the response function. The first two are made right when the function starts running, and the third occurs further along when adding the behavioral-response changes to income.

It's clear why we would want to make a copy before changing income, and I can see an argument for why we would want to make copies of the original copies from the beginning. However, could we add an argument to the responses function that makes creating these copies optional? From what I can see, there are no modifications being made to the calc1 and calc2 variables, only the copy of calc2. Of course I could be missing something obvious.

Making the copies optional would significantly reduce the amount of memory needed for partial equilibrium simulations, which would be especially helpful in Tax-Brain.

cc @hdoupe

[MattHJensen]

@andersonfrailey, thanks a lot for raising an issue about memory usage in Behavioral-Responses.

Right, we are making the copies to ensure this promise: neither calc_1 nor calc_2 are affected by this response function.

Specifically, the deep copies ensure that the response function doesn't call calc_all() on calc_1 and calc_2 and leave them in a calculated state when the user isn't expected it. (I suppose they also ensure against some other unintended side effects being added in the future).

So your suggestion makes sense in that it would allow users to explicitly choose whether to allow Behavioral-Responses to calculate their calc_1 and calc_2 or not. They get to ask themselves, do I want a permanent small memory effect or a temporary memory explosion?

On the other hand, adding this option will definitely add some complexity to the code and the user experience. Is it better to just accept and document that a side effect of behavior is calculating your calculators, and then get rid of those two copies for everyone? (I agree with your assessment that we should keep the third).

Interested in other perspectives here.

Also happy to review a PR demonstrating that giving the user the option doesn't add much complexity.

[andersonfrailey]

@MattHJensen, thanks for the response. I'll work on a PR that would make the copies optional to see how that affects complexity. For backwards compatibility purposes I'll make it so that the copies are still made by default and the user has to specify they don't want the copies made.

[hdoupe]

What do you all think about using a keyword argument, inplace, to indicate that the calculations should be performed on the calculators that are passed to response, instead of copies? Pandas and NumPy frequently use this keyword for the same purpose. So, it should be familiar to users. Here's an implementation for how this would work: master...hdoupe:inplace-flag

I benchmarked the aggregate memory usage using memory_profiler (version 0.55.0):

inplace	peak	increment
False	7.056 GB	6.88 GB
True	5.826 GB	5.652 GB

using this function:

def test_func(inplace):
    rec = tc.Records.cps_constructor()
    refyear = 2020
    assert refyear >= 2018
    reform = {'II_em': {refyear: 1500}}
    # ... construct pre-reform calculator
    pol = tc.Policy()
    calc1 = tc.Calculator(records=rec, policy=pol)
    calc1.advance_to_year(refyear)
    # ... construct two post-reform calculators
    pol.implement_reform(reform)
    calc2d = tc.Calculator(records=rec, policy=pol)  # for default behavior
    calc2d.advance_to_year(refyear)

    # Keep track of some of the variables that will be modifed
    # in response if inplace is True.
    df_before_1 = calc1.dataframe(tc.DIST_VARIABLES)
    df_before_2 = calc2d.dataframe(tc.DIST_VARIABLES)

    behresp.response(calc1, calc2d, elasticities={"inc": -0.2}, dump=True, inplace=inplace)

    # Grab the same variables after.
    df_after_1 = calc1.dataframe(tc.DIST_VARIABLES)
    df_after_2 = calc2d.dataframe(tc.DIST_VARIABLES)

    # If inplace is True, the variables should have been modified.
    # If inplace is False, response only modified a copy of calc_1
    # and calc_2.
    if inplace:
        assert not df_before_1.equals(df_after_1)
        assert not df_before_2.equals(df_after_2)
    else:
        assert df_before_1.equals(df_after_1)
        assert df_before_2.equals(df_after_2)

which I called like this:

# obtain high-level stats on memory usage
%memit test_func(inplace=False)

At the bottom of this comment, you can find a line-by-line breakdown to see how memory changed in each line of test_func. You can see that lines 102 and 103 increase the memory usage by about 1.38 GB when inplace is False. I'm curious to hear what people think about incorporating inplace into the API. Even saving 1 GB makes a difference to projects that run Tax-Calculator in parallel like Tax-Brain. For example, if Tax-Brain wants to run 5 years at once, this increases the memory requirement by 5GB.

You can replicate these results with this notebook. Note that the numbers will vary some with each run but the general memory usage stats should remain similar.

With inplace=False:

Filename: /home/hankdoupe/Documents/Behavioral-Responses/behresp/behavior.py

Line #    Mem usage    Increment   Line Contents
================================================
    13   2028.9 MiB   2028.9 MiB   def response(calc_1, calc_2, elasticities, dump=False, inplace=False):
    14                                 """
    15                                 Implements TaxBrain "Partial Equilibrium Simulation" dynamic analysis
    16                                 returning results as a tuple of Pandas DataFrame objects (df1, df2) where:
    17                                 df1 is extracted from a baseline-policy calc_1 copy, and
    18                                 df2 is extracted from a reform-policy calc_2 copy that incorporates the
    19                                     behavioral responses given by the nature of the baseline-to-reform
    20                                     change in policy and elasticities in the specified behavior dictionary.
    21                             
    22                                 Note: this function internally modifies a copy of calc_2 records to account
    23                                   for behavioral responses that arise from the policy reform that involves
    24                                   moving from calc1 policy to calc2 policy.  Neither calc_1 nor calc_2 need
    25                                   to have had calc_all() executed before calling the response function.
    26                                   By default, neither calc_1 nor calc_2 are affected by this response 
    27                                   function. To perform in-place calculations that affect calc_1 and calc_2,
    28                                   set inplace equal to True.
    29                             
    30                                 The elasticities argument is a dictionary containing the assumed response
    31                                 elasticities.  Omitting an elasticity key:value pair in the dictionary
    32                                 implies the omitted elasticity is assumed to be zero.  Here is the full
    33                                 dictionary content and each elasticity's internal name:
    34                             
    35                                  be_sub = elasticities['sub']
    36                                    Substitution elasticity of taxable income.
    37                                    Defined as proportional change in taxable income divided by
    38                                    proportional change in marginal net-of-tax rate (1-MTR) on taxpayer
    39                                    earnings caused by the reform.  Must be zero or positive.
    40                             
    41                                  be_inc = elasticities['inc']
    42                                    Income elasticity of taxable income.
    43                                    Defined as dollar change in taxable income divided by dollar change
    44                                    in after-tax income caused by the reform.  Must be zero or negative.
    45                             
    46                                  be_cg = elasticities['cg']
    47                                    Semi-elasticity of long-term capital gains.
    48                                    Defined as change in logarithm of long-term capital gains divided by
    49                                    change in marginal tax rate (MTR) on long-term capital gains caused by
    50                                    the reform.  Must be zero or negative.
    51                                    Read response function documentation (see below) for discussion of
    52                                    appropriate values.
    53                             
    54                                 The optional dump argument controls the number of variables included
    55                                 in the two returned DataFrame objects.  When dump=False (its default
    56                                 value), the variables in the two returned DataFrame objects include
    57                                 just the variables in the Tax-Calculator DIST_VARIABLES list, which
    58                                 is sufficient for constructing the standard Tax-Calculator tables.
    59                                 When dump=True, the variables in the two returned DataFrame objects
    60                                 include all the Tax-Calculator input and calculated output variables,
    61                                 which is the same output as produced by the Tax-Calculator tc --dump
    62                                 option except for one difference: the tc --dump option provides two
    63                                 calculated variables, mtr_inctax and mtr_paytax, that are replaced
    64                                 in the dump output of this response function by mtr_combined, which
    65                                 is the sum of mtr_inctax and mtr_paytax.
    66                             
    67                                 Note: the use here of a dollar-change income elasticity (rather than
    68                                   a proportional-change elasticity) is consistent with Feldstein and
    69                                   Feenberg, "The Taxation of Two Earner Families", NBER Working Paper
    70                                   No. 5155 (June 1995).  A proportional-change elasticity was used by
    71                                   Gruber and Saez, "The elasticity of taxable income: evidence and
    72                                   implications", Journal of Public Economics 84:1-32 (2002) [see
    73                                   equation 2 on page 10].
    74                             
    75                                 Note: the nature of the capital-gains elasticity used here is similar
    76                                   to that used in Joint Committee on Taxation, "New Evidence on the
    77                                   Tax Elasticity of Capital Gains: A Joint Working Paper of the Staff
    78                                   of the Joint Committee on Taxation and the Congressional Budget
    79                                   Office", (JCX-56-12), June 2012.  In particular, the elasticity
    80                                   use here is equivalent to the term inside the square brackets on
    81                                   the right-hand side of equation (4) on page 11 --- not the epsilon
    82                                   variable on the left-hand side of equation (4), which is equal to
    83                                   the elasticity used here times the weighted average marginal tax
    84                                   rate on long-term capital gains.  So, the JCT-CBO estimate of
    85                                   -0.792 for the epsilon elasticity (see JCT-CBO, Table 5) translates
    86                                   into a much larger absolute value for the be_cg semi-elasticity
    87                                   used by Tax-Calculator.
    88                                   To calculate the elasticity from a semi-elasticity, we multiply by
    89                                   MTRs from TC and weight by shares of taxable gains. To avoid those
    90                                   with zero MTRs, we restrict this to the top 40% of tax units by AGI.
    91                                   Using this function, a semi-elasticity of -3.45 corresponds to a tax
    92                                   rate elasticity of -0.792.
    93                             
    94                                 """
    95                                 # pylint: disable=too-many-locals,too-many-statements,too-many-branches
    96                             
    97                                 # Check function argument types and elasticity values
    98   2028.9 MiB      0.0 MiB       if inplace:
    99                                     calc1 = calc_1
   100                                     calc2 = calc_2
   101                                 else:
   102   2715.8 MiB    686.9 MiB           calc1 = copy.deepcopy(calc_1)
   103   3409.6 MiB    693.8 MiB           calc2 = copy.deepcopy(calc_2)
   104   3409.6 MiB      0.0 MiB       assert isinstance(calc1, tc.Calculator)
   105   3409.6 MiB      0.0 MiB       assert isinstance(calc2, tc.Calculator)
   106   3409.6 MiB      0.0 MiB       assert isinstance(elasticities, dict)
   107   3409.6 MiB      0.0 MiB       be_sub = elasticities['sub'] if 'sub' in elasticities else 0.0
   108   3409.6 MiB      0.0 MiB       be_inc = elasticities['inc'] if 'inc' in elasticities else 0.0
   109   3409.6 MiB      0.0 MiB       be_cg = elasticities['cg'] if 'cg' in elasticities else 0.0
   110   3409.6 MiB      0.0 MiB       assert be_sub >= 0.0
   111   3409.6 MiB      0.0 MiB       assert be_inc <= 0.0
   112   3409.6 MiB      0.0 MiB       assert be_cg <= 0.0
   113                             
   114                                 # Begin nested functions used only in this response function
   115   6039.7 MiB      0.0 MiB       def _update_ordinary_income(taxinc_change, calc):
   116                                     """
   117                                     Implement total taxable income change induced by behavioral response.
   118                                     """
   119                                     # compute AGI minus itemized deductions, agi_m_ided
   120   6039.7 MiB      0.0 MiB           agi = calc.array('c00100')
   121   6039.7 MiB      0.0 MiB           ided = np.where(calc.array('c04470') < calc.array('standard'),
   122   6039.7 MiB      0.0 MiB                           0., calc.array('c04470'))
   123   6039.7 MiB      0.0 MiB           agi_m_ided = agi - ided
   124                                     # assume behv response only for filing units with positive agi_m_ided
   125   6039.7 MiB      0.0 MiB           pos = np.array(agi_m_ided > 0., dtype=bool)
   126   6039.7 MiB      0.0 MiB           delta_income = np.where(pos, taxinc_change, 0.)
   127                                     # allocate delta_income into three parts
   128                                     # pylint: disable=unsupported-assignment-operation
   129   6039.7 MiB      0.0 MiB           winc = calc.array('e00200')
   130   6039.7 MiB      0.0 MiB           delta_winc = np.zeros_like(agi)
   131   6039.7 MiB      0.0 MiB           delta_winc[pos] = delta_income[pos] * winc[pos] / agi_m_ided[pos]
   132   6039.7 MiB      0.0 MiB           oinc = agi - winc
   133   6039.7 MiB      0.0 MiB           delta_oinc = np.zeros_like(agi)
   134   6039.7 MiB      0.0 MiB           delta_oinc[pos] = delta_income[pos] * oinc[pos] / agi_m_ided[pos]
   135   6039.7 MiB      0.0 MiB           delta_ided = np.zeros_like(agi)
   136   6039.7 MiB      0.0 MiB           delta_ided[pos] = delta_income[pos] * ided[pos] / agi_m_ided[pos]
   137                                     # confirm that the three parts are consistent with delta_income
   138   6039.7 MiB      0.0 MiB           assert np.allclose(delta_income, delta_winc + delta_oinc - delta_ided)
   139                                     # add the three parts to different records variables embedded in calc
   140   6039.7 MiB      0.0 MiB           calc.incarray('e00200', delta_winc)
   141   6039.7 MiB      0.0 MiB           calc.incarray('e00200p', delta_winc)
   142   6039.7 MiB      0.0 MiB           calc.incarray('e00300', delta_oinc)
   143   6039.7 MiB      0.0 MiB           calc.incarray('e19200', delta_ided)
   144   6039.7 MiB      0.0 MiB           return calc
   145                             
   146   6039.7 MiB      0.0 MiB       def _update_cap_gain_income(cap_gain_change, calc):
   147                                     """
   148                                     Implement capital gain change induced by behavioral responses.
   149                                     """
   150   6039.7 MiB      0.0 MiB           calc.incarray('p23250', cap_gain_change)
   151   6039.7 MiB      0.0 MiB           return calc
   152                             
   153   3618.8 MiB      0.0 MiB       def _mtr12(calc__1, calc__2, mtr_of='e00200p', tax_type='combined'):
   154                                     """
   155                                     Computes marginal tax rates for Calculator objects calc__1 and calc__2
   156                                     for specified mtr_of income type and specified tax_type.
   157                                     """
   158   3618.8 MiB      0.0 MiB           assert tax_type in ('combined', 'iitax')
   159   4914.5 MiB   1295.7 MiB           _, iitax1, combined1 = calc__1.mtr(mtr_of, wrt_full_compensation=True)
   160   5565.7 MiB    651.2 MiB           _, iitax2, combined2 = calc__2.mtr(mtr_of, wrt_full_compensation=True)
   161   5565.7 MiB      0.0 MiB           if tax_type == 'combined':
   162   5565.7 MiB      0.0 MiB               return (combined1, combined2)
   163                                     return (iitax1, iitax2)
   164                                 # End nested functions used only in this response function
   165                             
   166                                 # Begin main logic of response function
   167   3559.4 MiB    149.8 MiB       calc1.calc_all()
   168   3618.8 MiB     59.4 MiB       calc2.calc_all()
   169   3618.8 MiB      0.0 MiB       assert calc1.array_len == calc2.array_len
   170   3618.8 MiB      0.0 MiB       assert calc1.current_year == calc2.current_year
   171   3618.8 MiB      0.0 MiB       mtr_cap = 0.99
   172   3618.8 MiB      0.0 MiB       if dump:
   173   3618.8 MiB      0.0 MiB           recs_vinfo = tc.Records(data=None)  # contains records VARINFO only
   174   3618.8 MiB      0.0 MiB           dvars = list(recs_vinfo.USABLE_READ_VARS | recs_vinfo.CALCULATED_VARS)
   175                                 # Calculate sum of substitution and income effects
   176   3618.8 MiB      0.0 MiB       if be_sub == 0.0 and be_inc == 0.0:
   177                                     zero_sub_and_inc = True
   178                                     if dump:
   179                                         wage_mtr1 = np.zeros(calc1.array_len)
   180                                         wage_mtr2 = np.zeros(calc2.array_len)
   181                                 else:
   182   3618.8 MiB      0.0 MiB           zero_sub_and_inc = False
   183                                     # calculate marginal combined tax rates on taxpayer wages+salary
   184                                     # (e00200p is taxpayer's wages+salary)
   185   3618.8 MiB      0.0 MiB           wage_mtr1, wage_mtr2 = _mtr12(calc1, calc2,
   186   3618.8 MiB      0.0 MiB                                         mtr_of='e00200p',
   187   5565.7 MiB      0.0 MiB                                         tax_type='combined')
   188                                     # calculate magnitude of substitution effect
   189   5565.7 MiB      0.0 MiB           if be_sub == 0.0:
   190   5565.7 MiB      0.0 MiB               sub = np.zeros(calc1.array_len)
   191                                     else:
   192                                         # proportional change in marginal net-of-tax rates on earnings
   193                                         mtr1 = np.where(wage_mtr1 > mtr_cap, mtr_cap, wage_mtr1)
   194                                         mtr2 = np.where(wage_mtr2 > mtr_cap, mtr_cap, wage_mtr2)
   195                                         pch = ((1. - mtr2) / (1. - mtr1)) - 1.
   196                                         # Note: c04800 is filing unit's taxable income
   197                                         sub = be_sub * pch * calc1.array('c04800')
   198                                     # calculate magnitude of income effect
   199   5565.7 MiB      0.0 MiB           if be_inc == 0.0:
   200                                         inc = np.zeros(calc1.array_len)
   201                                     else:
   202                                         # dollar change in after-tax income
   203                                         # Note: combined is f.unit's income+payroll tax liability
   204   5565.7 MiB      0.0 MiB               dch = calc1.array('combined') - calc2.array('combined')
   205   5565.7 MiB      0.0 MiB               inc = be_inc * dch
   206                                     # calculate sum of substitution and income effects
   207   5565.7 MiB      0.0 MiB           si_chg = sub + inc
   208                                 # Calculate long-term capital-gains effect
   209   5565.7 MiB      0.0 MiB       if be_cg == 0.0:
   210   5565.7 MiB      0.0 MiB           ltcg_chg = np.zeros(calc1.array_len)
   211                                 else:
   212                                     # calculate marginal tax rates on long-term capital gains
   213                                     #  p23250 is filing units' long-term capital gains
   214                                     ltcg_mtr1, ltcg_mtr2 = _mtr12(calc1, calc2,
   215                                                                   mtr_of='p23250',
   216                                                                   tax_type='iitax')
   217                                     rch = ltcg_mtr2 - ltcg_mtr1
   218                                     exp_term = np.exp(be_cg * rch)
   219                                     new_ltcg = calc1.array('p23250') * exp_term
   220                                     ltcg_chg = new_ltcg - calc1.array('p23250')
   221                                 # Extract dataframe from calc1
   222   5565.7 MiB      0.0 MiB       if dump:
   223   6283.2 MiB    717.4 MiB           df1 = calc1.dataframe(dvars)
   224   6279.9 MiB      0.0 MiB           df1.drop('mtr_inctax', axis='columns', inplace=True)
   225   6276.4 MiB      0.0 MiB           df1.drop('mtr_paytax', axis='columns', inplace=True)
   226   6276.4 MiB      0.0 MiB           df1['mtr_combined'] = wage_mtr1 * 100
   227                                 else:
   228                                     df1 = calc1.dataframe(tc.DIST_VARIABLES)
   229   6276.4 MiB      0.0 MiB       if not inplace:
   230   6276.4 MiB      0.0 MiB           del calc1
   231                                 # Add behavioral-response changes to income sources
   232   6276.4 MiB      0.0 MiB       if inplace:
   233                                     calc2_behv = calc2
   234                                 else:
   235   6276.4 MiB      0.0 MiB           calc2_behv = copy.deepcopy(calc2)
   236   6039.7 MiB      0.0 MiB           del calc2
   237   6039.7 MiB      0.0 MiB       if not zero_sub_and_inc:
   238   6039.7 MiB      0.0 MiB           calc2_behv = _update_ordinary_income(si_chg, calc2_behv)
   239   6039.7 MiB      0.0 MiB       calc2_behv = _update_cap_gain_income(ltcg_chg, calc2_behv)
   240                                 # Recalculate post-reform taxes incorporating behavioral responses
   241   6039.7 MiB      0.0 MiB       calc2_behv.calc_all()
   242                                 # Extract dataframe from calc2_behv
   243   6039.7 MiB      0.0 MiB       if dump:
   244   6756.9 MiB    717.2 MiB           df2 = calc2_behv.dataframe(dvars)
   245   6753.6 MiB      0.0 MiB           df2.drop('mtr_inctax', axis='columns', inplace=True)
   246   6750.1 MiB      0.0 MiB           df2.drop('mtr_paytax', axis='columns', inplace=True)
   247   6750.1 MiB      0.0 MiB           df2['mtr_combined'] = wage_mtr2 * 100
   248                                 else:
   249                                     df2 = calc2_behv.dataframe(tc.DIST_VARIABLES)
   250   6750.1 MiB      0.0 MiB       if not inplace:
   251   6750.1 MiB      0.0 MiB           del calc2_behv
   252                                 # Return the two dataframes
   253   6750.1 MiB      0.0 MiB       return (df1, df2)

With inplace=True:

Filename: /home/hankdoupe/Documents/Behavioral-Responses/behresp/behavior.py

Line #    Mem usage    Increment   Line Contents
================================================
    13   2032.4 MiB   2032.4 MiB   def response(calc_1, calc_2, elasticities, dump=False, inplace=False):
    14                                 """
    15                                 Implements TaxBrain "Partial Equilibrium Simulation" dynamic analysis
    16                                 returning results as a tuple of Pandas DataFrame objects (df1, df2) where:
    17                                 df1 is extracted from a baseline-policy calc_1 copy, and
    18                                 df2 is extracted from a reform-policy calc_2 copy that incorporates the
    19                                     behavioral responses given by the nature of the baseline-to-reform
    20                                     change in policy and elasticities in the specified behavior dictionary.
    21                             
    22                                 Note: this function internally modifies a copy of calc_2 records to account
    23                                   for behavioral responses that arise from the policy reform that involves
    24                                   moving from calc1 policy to calc2 policy.  Neither calc_1 nor calc_2 need
    25                                   to have had calc_all() executed before calling the response function.
    26                                   By default, neither calc_1 nor calc_2 are affected by this response 
    27                                   function. To perform in-place calculations that affect calc_1 and calc_2,
    28                                   set inplace equal to True.
    29                             
    30                                 The elasticities argument is a dictionary containing the assumed response
    31                                 elasticities.  Omitting an elasticity key:value pair in the dictionary
    32                                 implies the omitted elasticity is assumed to be zero.  Here is the full
    33                                 dictionary content and each elasticity's internal name:
    34                             
    35                                  be_sub = elasticities['sub']
    36                                    Substitution elasticity of taxable income.
    37                                    Defined as proportional change in taxable income divided by
    38                                    proportional change in marginal net-of-tax rate (1-MTR) on taxpayer
    39                                    earnings caused by the reform.  Must be zero or positive.
    40                             
    41                                  be_inc = elasticities['inc']
    42                                    Income elasticity of taxable income.
    43                                    Defined as dollar change in taxable income divided by dollar change
    44                                    in after-tax income caused by the reform.  Must be zero or negative.
    45                             
    46                                  be_cg = elasticities['cg']
    47                                    Semi-elasticity of long-term capital gains.
    48                                    Defined as change in logarithm of long-term capital gains divided by
    49                                    change in marginal tax rate (MTR) on long-term capital gains caused by
    50                                    the reform.  Must be zero or negative.
    51                                    Read response function documentation (see below) for discussion of
    52                                    appropriate values.
    53                             
    54                                 The optional dump argument controls the number of variables included
    55                                 in the two returned DataFrame objects.  When dump=False (its default
    56                                 value), the variables in the two returned DataFrame objects include
    57                                 just the variables in the Tax-Calculator DIST_VARIABLES list, which
    58                                 is sufficient for constructing the standard Tax-Calculator tables.
    59                                 When dump=True, the variables in the two returned DataFrame objects
    60                                 include all the Tax-Calculator input and calculated output variables,
    61                                 which is the same output as produced by the Tax-Calculator tc --dump
    62                                 option except for one difference: the tc --dump option provides two
    63                                 calculated variables, mtr_inctax and mtr_paytax, that are replaced
    64                                 in the dump output of this response function by mtr_combined, which
    65                                 is the sum of mtr_inctax and mtr_paytax.
    66                             
    67                                 Note: the use here of a dollar-change income elasticity (rather than
    68                                   a proportional-change elasticity) is consistent with Feldstein and
    69                                   Feenberg, "The Taxation of Two Earner Families", NBER Working Paper
    70                                   No. 5155 (June 1995).  A proportional-change elasticity was used by
    71                                   Gruber and Saez, "The elasticity of taxable income: evidence and
    72                                   implications", Journal of Public Economics 84:1-32 (2002) [see
    73                                   equation 2 on page 10].
    74                             
    75                                 Note: the nature of the capital-gains elasticity used here is similar
    76                                   to that used in Joint Committee on Taxation, "New Evidence on the
    77                                   Tax Elasticity of Capital Gains: A Joint Working Paper of the Staff
    78                                   of the Joint Committee on Taxation and the Congressional Budget
    79                                   Office", (JCX-56-12), June 2012.  In particular, the elasticity
    80                                   use here is equivalent to the term inside the square brackets on
    81                                   the right-hand side of equation (4) on page 11 --- not the epsilon
    82                                   variable on the left-hand side of equation (4), which is equal to
    83                                   the elasticity used here times the weighted average marginal tax
    84                                   rate on long-term capital gains.  So, the JCT-CBO estimate of
    85                                   -0.792 for the epsilon elasticity (see JCT-CBO, Table 5) translates
    86                                   into a much larger absolute value for the be_cg semi-elasticity
    87                                   used by Tax-Calculator.
    88                                   To calculate the elasticity from a semi-elasticity, we multiply by
    89                                   MTRs from TC and weight by shares of taxable gains. To avoid those
    90                                   with zero MTRs, we restrict this to the top 40% of tax units by AGI.
    91                                   Using this function, a semi-elasticity of -3.45 corresponds to a tax
    92                                   rate elasticity of -0.792.
    93                             
    94                                 """
    95                                 # pylint: disable=too-many-locals,too-many-statements,too-many-branches
    96                             
    97                                 # Check function argument types and elasticity values
    98   2032.4 MiB      0.0 MiB       if inplace:
    99   2032.4 MiB      0.0 MiB           calc1 = calc_1
   100   2032.4 MiB      0.0 MiB           calc2 = calc_2
   101                                 else:
   102                                     calc1 = copy.deepcopy(calc_1)
   103                                     calc2 = copy.deepcopy(calc_2)
   104   2032.4 MiB      0.0 MiB       assert isinstance(calc1, tc.Calculator)
   105   2032.4 MiB      0.0 MiB       assert isinstance(calc2, tc.Calculator)
   106   2032.4 MiB      0.0 MiB       assert isinstance(elasticities, dict)
   107   2032.4 MiB      0.0 MiB       be_sub = elasticities['sub'] if 'sub' in elasticities else 0.0
   108   2032.4 MiB      0.0 MiB       be_inc = elasticities['inc'] if 'inc' in elasticities else 0.0
   109   2032.4 MiB      0.0 MiB       be_cg = elasticities['cg'] if 'cg' in elasticities else 0.0
   110   2032.4 MiB      0.0 MiB       assert be_sub >= 0.0
   111   2032.4 MiB      0.0 MiB       assert be_inc <= 0.0
   112   2032.4 MiB      0.0 MiB       assert be_cg <= 0.0
   113                             
   114                                 # Begin nested functions used only in this response function
   115   4910.2 MiB      0.0 MiB       def _update_ordinary_income(taxinc_change, calc):
   116                                     """
   117                                     Implement total taxable income change induced by behavioral response.
   118                                     """
   119                                     # compute AGI minus itemized deductions, agi_m_ided
   120   4910.2 MiB      0.0 MiB           agi = calc.array('c00100')
   121   4910.2 MiB      0.0 MiB           ided = np.where(calc.array('c04470') < calc.array('standard'),
   122   4910.2 MiB      0.0 MiB                           0., calc.array('c04470'))
   123   4910.2 MiB      0.0 MiB           agi_m_ided = agi - ided
   124                                     # assume behv response only for filing units with positive agi_m_ided
   125   4910.2 MiB      0.0 MiB           pos = np.array(agi_m_ided > 0., dtype=bool)
   126   4910.2 MiB      0.0 MiB           delta_income = np.where(pos, taxinc_change, 0.)
   127                                     # allocate delta_income into three parts
   128                                     # pylint: disable=unsupported-assignment-operation
   129   4910.2 MiB      0.0 MiB           winc = calc.array('e00200')
   130   4910.2 MiB      0.0 MiB           delta_winc = np.zeros_like(agi)
   131   4910.2 MiB      0.0 MiB           delta_winc[pos] = delta_income[pos] * winc[pos] / agi_m_ided[pos]
   132   4910.2 MiB      0.0 MiB           oinc = agi - winc
   133   4910.2 MiB      0.0 MiB           delta_oinc = np.zeros_like(agi)
   134   4910.2 MiB      0.0 MiB           delta_oinc[pos] = delta_income[pos] * oinc[pos] / agi_m_ided[pos]
   135   4910.2 MiB      0.0 MiB           delta_ided = np.zeros_like(agi)
   136   4910.2 MiB      0.0 MiB           delta_ided[pos] = delta_income[pos] * ided[pos] / agi_m_ided[pos]
   137                                     # confirm that the three parts are consistent with delta_income
   138   4910.2 MiB      0.0 MiB           assert np.allclose(delta_income, delta_winc + delta_oinc - delta_ided)
   139                                     # add the three parts to different records variables embedded in calc
   140   4910.2 MiB      0.0 MiB           calc.incarray('e00200', delta_winc)
   141   4910.2 MiB      0.0 MiB           calc.incarray('e00200p', delta_winc)
   142   4910.2 MiB      0.0 MiB           calc.incarray('e00300', delta_oinc)
   143   4910.2 MiB      0.0 MiB           calc.incarray('e19200', delta_ided)
   144   4910.2 MiB      0.0 MiB           return calc
   145                             
   146   4910.2 MiB      0.0 MiB       def _update_cap_gain_income(cap_gain_change, calc):
   147                                     """
   148                                     Implement capital gain change induced by behavioral responses.
   149                                     """
   150   4910.2 MiB      0.0 MiB           calc.incarray('p23250', cap_gain_change)
   151   4910.2 MiB      0.0 MiB           return calc
   152                             
   153   2229.9 MiB      0.0 MiB       def _mtr12(calc__1, calc__2, mtr_of='e00200p', tax_type='combined'):
   154                                     """
   155                                     Computes marginal tax rates for Calculator objects calc__1 and calc__2
   156                                     for specified mtr_of income type and specified tax_type.
   157                                     """
   158   2229.9 MiB      0.0 MiB           assert tax_type in ('combined', 'iitax')
   159   3527.8 MiB   1297.8 MiB           _, iitax1, combined1 = calc__1.mtr(mtr_of, wrt_full_compensation=True)
   160   4199.6 MiB    671.8 MiB           _, iitax2, combined2 = calc__2.mtr(mtr_of, wrt_full_compensation=True)
   161   4199.6 MiB      0.0 MiB           if tax_type == 'combined':
   162   4199.6 MiB      0.0 MiB               return (combined1, combined2)
   163                                     return (iitax1, iitax2)
   164                                 # End nested functions used only in this response function
   165                             
   166                                 # Begin main logic of response function
   167   2170.5 MiB    138.0 MiB       calc1.calc_all()
   168   2229.9 MiB     59.5 MiB       calc2.calc_all()
   169   2229.9 MiB      0.0 MiB       assert calc1.array_len == calc2.array_len
   170   2229.9 MiB      0.0 MiB       assert calc1.current_year == calc2.current_year
   171   2229.9 MiB      0.0 MiB       mtr_cap = 0.99
   172   2229.9 MiB      0.0 MiB       if dump:
   173   2229.9 MiB      0.0 MiB           recs_vinfo = tc.Records(data=None)  # contains records VARINFO only
   174   2229.9 MiB      0.0 MiB           dvars = list(recs_vinfo.USABLE_READ_VARS | recs_vinfo.CALCULATED_VARS)
   175                                 # Calculate sum of substitution and income effects
   176   2229.9 MiB      0.0 MiB       if be_sub == 0.0 and be_inc == 0.0:
   177                                     zero_sub_and_inc = True
   178                                     if dump:
   179                                         wage_mtr1 = np.zeros(calc1.array_len)
   180                                         wage_mtr2 = np.zeros(calc2.array_len)
   181                                 else:
   182   2229.9 MiB      0.0 MiB           zero_sub_and_inc = False
   183                                     # calculate marginal combined tax rates on taxpayer wages+salary
   184                                     # (e00200p is taxpayer's wages+salary)
   185   2229.9 MiB      0.0 MiB           wage_mtr1, wage_mtr2 = _mtr12(calc1, calc2,
   186   2229.9 MiB      0.0 MiB                                         mtr_of='e00200p',
   187   4199.6 MiB      0.0 MiB                                         tax_type='combined')
   188                                     # calculate magnitude of substitution effect
   189   4199.6 MiB      0.0 MiB           if be_sub == 0.0:
   190   4199.6 MiB      0.0 MiB               sub = np.zeros(calc1.array_len)
   191                                     else:
   192                                         # proportional change in marginal net-of-tax rates on earnings
   193                                         mtr1 = np.where(wage_mtr1 > mtr_cap, mtr_cap, wage_mtr1)
   194                                         mtr2 = np.where(wage_mtr2 > mtr_cap, mtr_cap, wage_mtr2)
   195                                         pch = ((1. - mtr2) / (1. - mtr1)) - 1.
   196                                         # Note: c04800 is filing unit's taxable income
   197                                         sub = be_sub * pch * calc1.array('c04800')
   198                                     # calculate magnitude of income effect
   199   4199.6 MiB      0.0 MiB           if be_inc == 0.0:
   200                                         inc = np.zeros(calc1.array_len)
   201                                     else:
   202                                         # dollar change in after-tax income
   203                                         # Note: combined is f.unit's income+payroll tax liability
   204   4199.6 MiB      0.0 MiB               dch = calc1.array('combined') - calc2.array('combined')
   205   4199.6 MiB      0.0 MiB               inc = be_inc * dch
   206                                     # calculate sum of substitution and income effects
   207   4199.6 MiB      0.0 MiB           si_chg = sub + inc
   208                                 # Calculate long-term capital-gains effect
   209   4199.6 MiB      0.0 MiB       if be_cg == 0.0:
   210   4199.6 MiB      0.0 MiB           ltcg_chg = np.zeros(calc1.array_len)
   211                                 else:
   212                                     # calculate marginal tax rates on long-term capital gains
   213                                     #  p23250 is filing units' long-term capital gains
   214                                     ltcg_mtr1, ltcg_mtr2 = _mtr12(calc1, calc2,
   215                                                                   mtr_of='p23250',
   216                                                                   tax_type='iitax')
   217                                     rch = ltcg_mtr2 - ltcg_mtr1
   218                                     exp_term = np.exp(be_cg * rch)
   219                                     new_ltcg = calc1.array('p23250') * exp_term
   220                                     ltcg_chg = new_ltcg - calc1.array('p23250')
   221                                 # Extract dataframe from calc1
   222   4199.6 MiB      0.0 MiB       if dump:
   223   4917.0 MiB    717.4 MiB           df1 = calc1.dataframe(dvars)
   224   4913.7 MiB      0.0 MiB           df1.drop('mtr_inctax', axis='columns', inplace=True)
   225   4910.2 MiB      0.0 MiB           df1.drop('mtr_paytax', axis='columns', inplace=True)
   226   4910.2 MiB      0.0 MiB           df1['mtr_combined'] = wage_mtr1 * 100
   227                                 else:
   228                                     df1 = calc1.dataframe(tc.DIST_VARIABLES)
   229   4910.2 MiB      0.0 MiB       if not inplace:
   230                                     del calc1
   231                                 # Add behavioral-response changes to income sources
   232   4910.2 MiB      0.0 MiB       if inplace:
   233   4910.2 MiB      0.0 MiB           calc2_behv = calc2
   234                                 else:
   235                                     calc2_behv = copy.deepcopy(calc2)
   236                                     del calc2
   237   4910.2 MiB      0.0 MiB       if not zero_sub_and_inc:
   238   4910.2 MiB      0.0 MiB           calc2_behv = _update_ordinary_income(si_chg, calc2_behv)
   239   4910.2 MiB      0.0 MiB       calc2_behv = _update_cap_gain_income(ltcg_chg, calc2_behv)
   240                                 # Recalculate post-reform taxes incorporating behavioral responses
   241   4910.2 MiB      0.0 MiB       calc2_behv.calc_all()
   242                                 # Extract dataframe from calc2_behv
   243   4910.2 MiB      0.0 MiB       if dump:
   244   5627.4 MiB    717.2 MiB           df2 = calc2_behv.dataframe(dvars)
   245   5624.1 MiB      0.0 MiB           df2.drop('mtr_inctax', axis='columns', inplace=True)
   246   5620.6 MiB      0.0 MiB           df2.drop('mtr_paytax', axis='columns', inplace=True)
   247   5620.6 MiB      0.0 MiB           df2['mtr_combined'] = wage_mtr2 * 100
   248                                 else:
   249                                     df2 = calc2_behv.dataframe(tc.DIST_VARIABLES)
   250   5620.6 MiB      0.0 MiB       if not inplace:
   251                                     del calc2_behv
   252                                 # Return the two dataframes
   253   5620.6 MiB      0.0 MiB       return (df1, df2)

[hdoupe]

@MattHJensen I'm trying to run Tax-Brain with behresp on my 16GB RAM limited laptop and it's reminding me of this issue. I still think it would be helpful to be able to run behresp without making so many copies of Calculator objects.