Feature/Fix: Add simulated transaction fee to calculations.csv and fix compute logic

.gitignore

@@ -1,6 +1,7 @@
 logs/
 reports/
 .vscode/
+.env
 */*/__pycache__/
 */__pycache__/
 email.ini*

docs/configuration.rst

@@ -89,17 +89,17 @@ Available configuration parameters are:
 
   Example::
 
-    Current Baker Balance: 17,400 XTZ
-    Total Delegations: 69,520 XTZ
-    Total Staked: 86,920 XTZ
+    Current Baker Balance: 17,400 tez
+    Total Delegations: 69,520 tez
+    Total Staked: 86,920 tez
 
     service_fee: 9
     owners_map:
       {'tz1PV5g16m9hHMAVJ4Hx6NzzUHgksDnTLFcK' : 0.4,
        'tz1PirboZKFVqkfE45hVLpkpXaZtLk3mqC17' : 0.4,
        'tz1VxS7ff4YnZRs8b4mMP4WaMVpoQjuo1rjf' : 0.2}
 
-  Charlie, and Dave, have each transfered 6,960 Tez to the baker address. Edwin has transfered 3,480 Tez. They are each partial owners of the baking balance. When rewards are delivered at the end of each cycle, 9% is taken as the bakery fee (ie: *service_fee*). That 9% is dispersed to any *founders*. If there are no founders, that 9% remains in the baker's balance.
+  Charlie, and Dave, have each transfered 6,960 tez to the baker address. Edwin has transfered 3,480 tez. They are each partial owners of the baking balance. When rewards are delivered at the end of each cycle, 9% is taken as the bakery fee (ie: *service_fee*). That 9% is dispersed to any *founders*. If there are no founders, that 9% remains in the baker's balance.
   The baker address is technically a delegator to itself. Its share of rewards are part of the overall cycle rewards. Charlie, Dave, and Edwin divide the "baker address rewards" as per the ratios in *owners_map*. Additionally, owners are *not* subject to the *service_fee*.
 
 **specials_map**
@@ -119,7 +119,7 @@ Available configuration parameters are:
                     'tz1PirboZKFVqkfE45hVLpkpXaZtLk3mqC17'}
 
 **min_delegation_amt**
-  A minimum delegation amount can be set here. If this value is set to 10, 10 XTZ are required as minimum. It is important to define what happens to the rewards of excluded delegates that are below the minimum delegation balance in rules_map.
+  A minimum delegation amount can be set here. If this value is set to 10, 10 tez are required as minimum. It is important to define what happens to the rewards of excluded delegates that are below the minimum delegation balance in rules_map.
 
   Example::
 
@@ -146,6 +146,8 @@ Available configuration parameters are:
 
     delegator_pays_ra_fee : False
 
+    Note: This option does also apply to the burn fee needed to payout to kt accounts.
+
 **pay_denunciation_rewards**
   True/False - Baker may get rewarded for denunciating another baker's equivocation (double baking or double endorsing). The protocol rewards the baker including the denunciation. When True, these rewards will be distributed. When False, they will remain in the baker's account, allowing the baker to reimburse the party at fault if they desire. Must be set to True when using RPC backend as RPC is not able to itemize rewards.
 

docs/contributors.rst

@@ -1,6 +1,9 @@
 For Developers
 =====================================================
 
+General:
+---------
+
 If you are interested in improving our project, this is just great! Our community looks forward to your contribution. Please follow this process:
 
 1. Start a bug/feature issue on GitHub about your proposed change. Continue discussion until a solution is decided.
@@ -40,4 +43,35 @@ Guidelines for rewarding PRs in the context of the TRD grant:
 4. The PR can only be approved if an agreement about the work effort (in hrs) is made.
 5. The PR should be merged using a squash merge, the work effort of the contributor and reviewer should be documented in the squash merge commit message.
 6. The work efforts are paid out in a regular basis. The payments are documented in a dedicated file named `contributor_payments.csv`.
-7. For each payment, the reason for payment should be documented (commit hash, review efforts, etc.). Additionally, the paid amount in USD, the VWAP XTZ-USD on the payment day, the payed amount in XTZ and the transaction hash are documented.
+7. For each payment, the reason for payment should be documented (commit hash, review efforts, etc.). Additionally, the paid amount in USD, the VWAP XTZ-USD on the payment day, the payed amount in XTZ and the transaction hash are documented.
+
+Setup:
+-------
+
+you can contribute to TRD using your the developer tool of your choice. The follwing setup helps with VS Code.
+
+1. Creat a `.env` file in the workspace directory with the following content:
+
+    Windows: PYTHONPATH=${PYTHONPATH};./src
+    Mac/Linux: PYTHONPATH=${PYTHONPATH}:./src
+
+2. Create a `settings.json` file in the folder `.vscode` with the following content:
+
+.. code-block:: JSON
+
+    {
+    "terminal.integrated.env.osx": {
+      "PYTHONPATH": "${env:PYTHONPATH}:${workspaceFolder}/src",
+    },
+    "terminal.integrated.env.linux": {
+      "PYTHONPATH": "${env:PYTHONPATH}:${workspaceFolder}/src",
+    },
+    "terminal.integrated.env.windows": {
+      "PYTHONPATH": "${env:PYTHONPATH};${workspaceFolder}/src",
+    },
+    "python.testing.pytestArgs": [
+      "tests"
+    ],
+    "python.testing.unittestEnabled": false,
+    "python.testing.pytestEnabled": true
+  }

docs/plugins.md

@@ -125,7 +125,7 @@ plugins:
       - 827384777
     bot_api_key: 988877766:SKDJFLSJDFJLJSKDFJLKSDJFLKJDF
     telegram_text: >
-      &#x2728; Reward for cycle %CYCLE% <b>complete</b>! We had %NDELEGATORS% <i>delegators</i> in the cycle and paid out %TREWARDS% XTZ in rewards!
+      &#x2728; Reward for cycle %CYCLE% <b>complete</b>! We had %NDELEGATORS% <i>delegators</i> in the cycle and paid out %TREWARDS% tez in rewards!
 ```
 
 ### Example Result
@@ -166,14 +166,14 @@ plugins:
     access_token: YYYYYYYY
     access_secret: WWWWWWWW
     tweet_text: >
-      Reward for cycle %CYCLE% complete! We had %NDELEGATORS% delegators in the cycle and paid out %TREWARDS% XTZ in rewards. #ourbakery #rewards #tezos
+      Reward for cycle %CYCLE% complete! We had %NDELEGATORS% delegators in the cycle and paid out %TREWARDS% tez in rewards. #ourbakery #rewards #tezos
 ```
 
 The above example configuration will produce a tweet that looks like this:
 
 ```
 Reward for cycle 290 complete! We had 133 delegators
-in the cycle and paid out 1234.98 XTZ in rewards.
+in the cycle and paid out 1234.98 tez in rewards.
 #ourbakery #rewards #tezos
 ```
 
@@ -200,7 +200,7 @@ plugins:
     send_admin: False
     discord_text: >
       Rewards for cycle %CYCLE% are completed.
-      We paid out %TREWARDS% XTZ in rewards to %NDELEGATORS% delegators.        
+      We paid out %TREWARDS% tez in rewards to %NDELEGATORS% delegators.        
 ```
 
 ### Example Result

examples/tz1boot1pK9h2BVGXdyvfQSv8kd1LQM6H889.yaml

@@ -49,7 +49,7 @@ plugins:
     bot_api_key: 988877766:SKDJFLSJDFJLJSKDFJLKSDJFLKJDF
     telegram_text: >
       Rewards for cycle %CYCLE% are completed.
-      We paid out %TREWARDS% XTZ in rewards to %NDELEGATORS% delegators.
+      We paid out %TREWARDS% tez in rewards to %NDELEGATORS% delegators.
   twitter:
     api_key: XXXXXXXX
     api_secret: ZZZZZZZZ
@@ -67,4 +67,4 @@ plugins:
     send_admin: False
     discord_text: >
       Rewards for cycle %CYCLE% are completed.
-      We paid out %TREWARDS% XTZ in rewards to %NDELEGATORS% delegators.
+      We paid out %TREWARDS% tez in rewards to %NDELEGATORS% delegators.

requirements.txt

@@ -5,3 +5,7 @@ fysom==2.1.6
 parse
 transitions
 tweepy
+# developer
+parameterized
+Pygments
+graphviz

src/Constants.py

@@ -75,7 +75,9 @@
 }
 
 
-MUTEZ = 1e6
+MUTEZ_PER_TEZ = 1e6
+MAXIMUM_ROUNDING_ERROR = 10  # mutez
+ALMOST_ZERO = 1e-6
 
 VERSION = 10.0
 

src/calc/calculate_phase1.py

@@ -67,10 +67,8 @@ def calculate(self, reward_data0, total_amount):
 
         # total reward amount needs to be diminished at the same rate total balance diminishes
         new_total_amnt_multiplier = new_total_balance / total_balance
-        new_total_amount = int(
-            Decimal(total_amount * new_total_amnt_multiplier).to_integral_value(
-                rounding=ROUND_HALF_DOWN
-            )
-        )
+        new_total_amount = Decimal(
+            total_amount * new_total_amnt_multiplier
+        ).to_integral_value(rounding=ROUND_HALF_DOWN)
 
-        return rewards, new_total_amount
+        return rewards, int(new_total_amount)

src/calc/calculate_phase2.py

@@ -66,4 +66,4 @@ def calculate(self, reward_data1, total_amount):
             rl2.ratio2 = rl2.ratio
 
         # total reward amount remains the same
-        return rewards, total_amount
+        return rewards, int(total_amount)

src/calc/calculate_phase3.py

@@ -5,6 +5,7 @@
 )
 from model.baking_conf import MIN_DELEGATION_KEY
 from model.reward_log import RewardLog, TYPE_FOUNDERS_PARENT
+from Constants import ALMOST_ZERO
 
 
 class CalculatePhase3(CalculatePhaseBase):
@@ -63,7 +64,7 @@ def calculate(self, reward_data2, total_amount):
             total_service_fee_ratio += rl.service_fee_ratio
 
         # create founders parent record
-        if total_service_fee_ratio > 1e-6:  # >0
+        if total_service_fee_ratio > ALMOST_ZERO:
             rl = RewardLog(
                 address=TYPE_FOUNDERS_PARENT,
                 type=TYPE_FOUNDERS_PARENT,
@@ -77,4 +78,4 @@ def calculate(self, reward_data2, total_amount):
 
             new_rewards.append(rl)
 
-        return new_rewards, total_amount
+        return new_rewards, int(total_amount)

src/calc/calculate_phase4.py

@@ -66,4 +66,4 @@ def calculate(self, reward_data3, total_amount):
                 rl3.ratio4 = rl3.ratio
                 new_rewards.append(rl3)
 
-        return new_rewards, total_amount
+        return new_rewards, int(total_amount)

src/calc/calculate_phase_base.py

@@ -2,9 +2,9 @@
 
 import itertools
 
-BY_CONFIGURATION = "Excluded by configuration"
-BY_MIN_DELEGATION = "Excluded by min delegation"
-BY_ZERO_BALANCE = "Excluded by zero balance"
+BY_CONFIGURATION = "Excluded by configuration. "
+BY_MIN_DELEGATION = "Excluded by min delegation. "
+BY_ZERO_BALANCE = "Excluded by zero balance. "
 
 
 class CalculatePhaseBase(ABC):

src/calc/calculate_phase_final.py

@@ -38,10 +38,7 @@ def calculate(self, reward_data5, total_amount, adjustments={}):
                     rounding=ROUND_HALF_DOWN
                 )
             )
-            rl.payable = (
-                rl.type in [TYPE_FOUNDER, TYPE_OWNER, TYPE_DELEGATOR]
-                and rl.adjusted_amount > 0
-            )
+            rl.payable = rl.type in [TYPE_FOUNDER, TYPE_OWNER, TYPE_DELEGATOR]
             rl.service_fee_amount = int(
                 Decimal(rl.service_fee_ratio * total_amount).to_integral_value(
                     rounding=ROUND_HALF_DOWN

src/calc/phased_payment_calculator.py

@@ -1,5 +1,6 @@
 import functools
 
+from Constants import MAXIMUM_ROUNDING_ERROR, ALMOST_ZERO
 from calc.calculate_phase0 import CalculatePhase0
 from calc.calculate_phase1 import CalculatePhase1
 from calc.calculate_phase2 import CalculatePhase2
@@ -15,9 +16,6 @@
 
 logger = main_logger.getChild("phased_calculator")
 
-MINOR_DIFF = 10
-MINOR_RATIO_DIFF = 1e-6
-
 
 class PhasedPaymentCalculator:
     """
@@ -58,9 +56,9 @@ def calculate(self, reward_provider_model, adjustments=None):
         phase0 = CalculatePhase0(reward_provider_model)
         rwrd_logs = phase0.calculate()
 
-        total_rwrd_amnt = reward_provider_model.computed_reward_amount
+        total_rwrd_amnt = int(reward_provider_model.computed_reward_amount)
         logger.info(
-            "Total rewards before processing is {:,} mutez.".format(total_rwrd_amnt)
+            "Total rewards before processing is {:<,d} mutez.".format(total_rwrd_amnt)
         )
         if total_rwrd_amnt == 0:
             logger.debug("NO REWARDS to process!")
@@ -133,41 +131,44 @@ def calculate(self, reward_provider_model, adjustments=None):
         rwrd_logs.sort(key=functools.cmp_to_key(cmp_by_type_balance))
 
         # check if there is difference between sum of calculated amounts and total_rewards
-        total_delegator_amounts = sum(
-            [rl.adjusted_amount for rl in rwrd_logs if not rl.skipped]
+        total_delegator_amounts = int(
+            sum([rl.adjusted_amount for rl in rwrd_logs if not rl.skipped])
+        )
+        total_adjustments = int(
+            sum([rl.adjustment for rl in rwrd_logs if not rl.skipped])
         )
-        total_adjustments = sum([rl.adjustment for rl in rwrd_logs if not rl.skipped])
-        amnt_pay_diff = abs(
-            total_rwrd_amnt + total_adjustments - total_delegator_amounts
+        amnt_pay_diff = int(
+            abs(total_rwrd_amnt + total_adjustments - total_delegator_amounts)
         )
 
         logger.info(
-            "Total rewards after processing is {:,} mutez.".format(total_rwrd_amnt)
+            "Total rewards after processing is {:<,d} mutez.".format(total_rwrd_amnt)
         )
+
         if total_adjustments < 0:
             logger.info(
-                "Total adjustment for past early payout is {:,} mutez.".format(
+                "Total adjustment for past early payout is {:<,d} mutez.".format(
                     total_adjustments
                 )
             )
             logger.info(
-                "Adjusted total rewards is {:,} mutez.".format(
+                "Adjusted total rewards is {:<,d} mutez.".format(
                     total_rwrd_amnt + total_adjustments
                 )
             )
         logger.info(
-            "Sum of amounts allocated to delegators is {:,} mutez".format(
+            "Sum of amounts allocated to delegators is {:<,d} mutez".format(
                 total_delegator_amounts
             )
         )
         logger.info(
-            "Difference between total rewards and sum of amounts allocated to delegators is {:,} mutez. "
-            "This is due to floating point arithmetic. (max allowed diff is {:,})".format(
-                amnt_pay_diff, MINOR_DIFF
+            "Difference between total rewards and sum of amounts allocated to delegators is {:<,d} mutez. "
+            "This is due to floating point arithmetic. (max allowed diff is {:<,d} mutez)".format(
+                amnt_pay_diff, int(MAXIMUM_ROUNDING_ERROR)
             )
         )
 
-        return rwrd_logs, total_rwrd_amnt
+        return rwrd_logs, int(total_rwrd_amnt)
 
     def almost_equal(self, double1, double2):
-        return abs(double1 - double2) < MINOR_RATIO_DIFF
+        return abs(double1 - double2) < ALMOST_ZERO

src/config/yaml_baking_conf_parser.py

@@ -2,7 +2,7 @@
 
 from config.addr_type import AddrType
 from config.yaml_conf_parser import YamlConfParser
-from Constants import RewardsType
+from Constants import RewardsType, ALMOST_ZERO
 from exception.configuration import ConfigurationException
 from log_config import main_logger
 from model.baking_conf import (
@@ -147,7 +147,9 @@ def validate_share_map(self, conf_obj, map_name):
 
         if len(share_map) > 0:
             try:
-                if abs(1 - sum(share_map.values()) > 1e-4):  # a zero check actually
+                if abs(
+                    1 - sum(share_map.values()) > ALMOST_ZERO
+                ):  # a zero check actually
                     raise ConfigurationException(
                         "Map '{}' shares does not sum up to 1!".format(map_name)
                     )
@@ -312,13 +314,13 @@ def validate_plugins(self, conf_obj):
     def validate_rewards_type(self, conf_obj):
 
         if REWARDS_TYPE not in conf_obj or conf_obj[REWARDS_TYPE] is None:
-            conf_obj[REWARDS_TYPE] = "actual"
+            conf_obj[REWARDS_TYPE] = RewardsType.ACTUAL
             logger.warning(
                 "[config_parser] Parameter '{:s}' is missing or incorrectly configured. "
                 "Defaults to 'actual' rewards payout type.".format(REWARDS_TYPE)
             )
 
-        if conf_obj[REWARDS_TYPE] == "estimated":
+        if conf_obj[REWARDS_TYPE] == RewardsType.ESTIMATED:
             raise ConfigurationException(
                 "Setting 'rewards_type' to 'estimated' is no longer supported.\n"
                 "Please see https://tezos-reward-distributor-organization.github.io/tezos-reward-distributor/payouttiming.html\n"