Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persist the dynamic changes to the batch size #156

Merged
merged 3 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions packages/packages.json
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
{
"dev": {
"skill/valory/market_manager_abci/0.1.0": "bafybeic7o4pclkhnugyn7js5g3asxuqhkxpvunlp3mpup7aovhg2fto22i",
"skill/valory/decision_maker_abci/0.1.0": "bafybeiezxeb4vwsuzbfbka5emvzqlzs4zghcqyt6ilatdakokiuwqkejoy",
"skill/valory/trader_abci/0.1.0": "bafybeiaxrpftik5wf6x3avulcc5n4wvpis3kmroggjoet2bs4q7lbizr5e",
"skill/valory/decision_maker_abci/0.1.0": "bafybeigdp4pgm4ons5pzptmclyglyxyqut266s4zutpmqj2e22wdpdkt5y",
"skill/valory/trader_abci/0.1.0": "bafybeib3ennterz5yrqflnsu6asy2yat4s35n72pvqsm2wv37l7434oyoa",
"contract/valory/market_maker/0.1.0": "bafybeid5jnetbsusgxkpqzwyiqlwut55xktfbeloipxcemcuu5oopmqtl4",
"agent/valory/trader/0.1.0": "bafybeibjajgtln64kas6o2nsxn4k5vf2akqyvlak3ugcpl2mbuvbserytq",
"service/valory/trader/0.1.0": "bafybeifux2exrevxmvwfxy6iazce7hvr3juvjitsx3nrlwsbtjy5utium4",
"agent/valory/trader/0.1.0": "bafybeife3upqzbesfm2kpokglvsmqjvsjqyno4pookr74eidq7nqtymz2y",
"service/valory/trader/0.1.0": "bafybeidvn5cx5mqzey3ygcaxdaiy2zj565d3p7lhsz74imi5dwyikqxfey",
"contract/valory/erc20/0.1.0": "bafybeiepg5ymxhtylgijs3mf3lezz6cwcsougbrv2gr4xcq3bp5wxusi64",
"skill/valory/tx_settlement_multiplexer_abci/0.1.0": "bafybeig6pwr7zcou53qehlytced5rdltue7phfttt4ms55hk2bk3brfpve",
"skill/valory/tx_settlement_multiplexer_abci/0.1.0": "bafybeic6bveoijtjzvquphekmset7sondro4iknljf5qnylkadeeusiwp4",
"contract/valory/mech/0.1.0": "bafybeieo65rtidzgrx22la7z7azoqbisd35hnoqfw7l3xxctotvctjlkju",
"contract/valory/realitio/0.1.0": "bafybeiejybwx4d3iwaxumxuox7vbub2syyzycqnxaflnjakrgcf4z5gcle",
"contract/valory/realitio/0.1.0": "bafybeieoily22pasgkzzcyawhqyrsko7b52lghnvwuskhomavb7tlb6pxa",
"contract/valory/realitio_proxy/0.1.0": "bafybeidx37xzjjmapwacedgzhum6grfzhp5vhouz4zu3pvpgdy5pgb2fr4",
"contract/valory/conditional_tokens/0.1.0": "bafybeihnhs4ttjttlba2vqwyufsundne4oazblbvwwlq5x7isklnslczju",
"contract/valory/conditional_tokens/0.1.0": "bafybeifu5axib5ifzq6bomfscs7nnx5qknkzymlz6gfn7ohjsb2shghrei",
"contract/valory/agent_registry/0.1.0": "bafybeibezt4xtzgf25eidmwev6moki74eufmb4hx7cpvd6odbyoxo42hxi",
"contract/valory/service_staking_token/0.1.0": "bafybeig2pnb3f4y65hlhhva52fsrwb6k3dqkqttq43sxd5kb4drgob65oy",
"skill/valory/staking_abci/0.1.0": "bafybeia3acxumkjg566dvgagv3swz7htf5xpvwq6ovmecvp5j2zdxdoabe"
Expand Down
10 changes: 5 additions & 5 deletions packages/valory/agents/trader/aea-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ contracts:
- valory/erc20:0.1.0:bafybeiepg5ymxhtylgijs3mf3lezz6cwcsougbrv2gr4xcq3bp5wxusi64
- valory/multisend:0.1.0:bafybeig5byt5urg2d2bsecufxe5ql7f4mezg3mekfleeh32nmuusx66p4y
- valory/mech:0.1.0:bafybeieo65rtidzgrx22la7z7azoqbisd35hnoqfw7l3xxctotvctjlkju
- valory/conditional_tokens:0.1.0:bafybeihnhs4ttjttlba2vqwyufsundne4oazblbvwwlq5x7isklnslczju
- valory/realitio:0.1.0:bafybeiejybwx4d3iwaxumxuox7vbub2syyzycqnxaflnjakrgcf4z5gcle
- valory/conditional_tokens:0.1.0:bafybeifu5axib5ifzq6bomfscs7nnx5qknkzymlz6gfn7ohjsb2shghrei
- valory/realitio:0.1.0:bafybeieoily22pasgkzzcyawhqyrsko7b52lghnvwuskhomavb7tlb6pxa
- valory/realitio_proxy:0.1.0:bafybeidx37xzjjmapwacedgzhum6grfzhp5vhouz4zu3pvpgdy5pgb2fr4
- valory/agent_registry:0.1.0:bafybeibezt4xtzgf25eidmwev6moki74eufmb4hx7cpvd6odbyoxo42hxi
- valory/service_staking_token:0.1.0:bafybeig2pnb3f4y65hlhhva52fsrwb6k3dqkqttq43sxd5kb4drgob65oy
Expand All @@ -43,10 +43,10 @@ skills:
- valory/reset_pause_abci:0.1.0:bafybeiencz2uytz6fqj5wg7mcunevxjh4xg5gd6pqcgkshgqywhcimweyi
- valory/termination_abci:0.1.0:bafybeigainmpy37gb33ogbrpggjbkuasodhthh5oz4vbzjdfjozerpf224
- valory/transaction_settlement_abci:0.1.0:bafybeigk3debp6dswutqsuls2lqfvyj4ghe6kwjc2zfinnsvj6hujynxtq
- valory/tx_settlement_multiplexer_abci:0.1.0:bafybeig6pwr7zcou53qehlytced5rdltue7phfttt4ms55hk2bk3brfpve
- valory/tx_settlement_multiplexer_abci:0.1.0:bafybeic6bveoijtjzvquphekmset7sondro4iknljf5qnylkadeeusiwp4
- valory/market_manager_abci:0.1.0:bafybeic7o4pclkhnugyn7js5g3asxuqhkxpvunlp3mpup7aovhg2fto22i
- valory/decision_maker_abci:0.1.0:bafybeiezxeb4vwsuzbfbka5emvzqlzs4zghcqyt6ilatdakokiuwqkejoy
- valory/trader_abci:0.1.0:bafybeiaxrpftik5wf6x3avulcc5n4wvpis3kmroggjoet2bs4q7lbizr5e
- valory/decision_maker_abci:0.1.0:bafybeigdp4pgm4ons5pzptmclyglyxyqut266s4zutpmqj2e22wdpdkt5y
- valory/trader_abci:0.1.0:bafybeib3ennterz5yrqflnsu6asy2yat4s35n72pvqsm2wv37l7434oyoa
- valory/staking_abci:0.1.0:bafybeia3acxumkjg566dvgagv3swz7htf5xpvwq6ovmecvp5j2zdxdoabe
default_ledger: ethereum
required_ledgers:
Expand Down
2 changes: 1 addition & 1 deletion packages/valory/contracts/conditional_tokens/contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def check_redeemed(
except (Urllib3ReadTimeoutError, RequestsReadTimeoutError):
msg = (
"The RPC timed out! This usually happens if the filtering is too wide. "
f"The service tried to filter from block {from_block} to {to_block}."
f"The service tried to filter from block {from_block} to {to_block}. "
f"If this issue persists, please try lowering the `EVENT_FILTERING_BATCH_SIZE`!"
)
return dict(error=msg)
Expand Down
2 changes: 1 addition & 1 deletion packages/valory/contracts/conditional_tokens/contract.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ aea_version: '>=1.0.0, <2.0.0'
fingerprint:
__init__.py: bafybeidhdxio3oq5gqdnxmngumvt3fcd6zyiyrpk5f2k4dwhflbg4e5iky
build/ConditionalTokens.json: bafybeia2ahis7zx2yhhf23kpkcxu56hto6fwg6ptjg5ld46lp4dgz7cz3e
contract.py: bafybeiau5r7uwpuykumfrr6qzsfgyk23n2lbieemndr7x45dlvnwo7mxgy
contract.py: bafybeic34mnydr5m7xncg42tgg5qrjxkltsxxudnvog4jjcs6elg64z5ty
fingerprint_ignore_patterns: []
class_name: ConditionalTokensContract
contract_interface_paths:
Expand Down
2 changes: 1 addition & 1 deletion packages/valory/contracts/realitio/contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def get_claim_params(
except (Urllib3ReadTimeoutError, RequestsReadTimeoutError):
msg = (
"The RPC timed out! This usually happens if the filtering is too wide. "
f"The service tried to filter from block {from_block} to {to_block}."
f"The service tried to filter from block {from_block} to {to_block}. "
f"If this issue persists, please try lowering the `EVENT_FILTERING_BATCH_SIZE`!"
)
return dict(error=msg)
Expand Down
2 changes: 1 addition & 1 deletion packages/valory/contracts/realitio/contract.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ aea_version: '>=1.0.0, <2.0.0'
fingerprint:
__init__.py: bafybeictahkgfmlqv5kksvj6klmxtmjdpeq4sp3x7dp2yr5x4kmzbcihse
build/Realitio.json: bafybeiagi7zoeoy5s7duhg4oeuekj2s6z5mad2z6g2pn3n5elsvze25qiu
contract.py: bafybeie4qmxeb3wdok62lsn5eyct25us4qgbgq3qzswnx7a52xfvxo7c6q
contract.py: bafybeicre6tumch2ng7rpgx2qle3cqg65rra6s2otl5agdp7jzweiffxby
fingerprint_ignore_patterns: []
class_name: RealitioContract
contract_interface_paths:
Expand Down
2 changes: 1 addition & 1 deletion packages/valory/services/trader/service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ license: Apache-2.0
fingerprint:
README.md: bafybeigtuothskwyvrhfosps2bu6suauycolj67dpuxqvnicdrdu7yhtvq
fingerprint_ignore_patterns: []
agent: valory/trader:0.1.0:bafybeibjajgtln64kas6o2nsxn4k5vf2akqyvlak3ugcpl2mbuvbserytq
agent: valory/trader:0.1.0:bafybeife3upqzbesfm2kpokglvsmqjvsjqyno4pookr74eidq7nqtymz2y
number_of_agents: 4
deployment: {}
---
Expand Down
20 changes: 10 additions & 10 deletions packages/valory/skills/decision_maker_abci/behaviours/reedem.py
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,7 @@ def _check_already_redeemed(self) -> WaitableConditionType:

n_retries = 0
from_block = self.redeeming_progress.check_from_block
batch_size = self.params.event_filtering_batch_size
batch_size = self.redeeming_progress.event_filtering_batch_size
while from_block < self.redeeming_progress.check_to_block:
max_to_block = from_block + batch_size
to_block = min(max_to_block, self.redeeming_progress.check_to_block)
Expand All @@ -447,12 +447,12 @@ def _check_already_redeemed(self) -> WaitableConditionType:

if not result:
n_retries += 1
keep_fraction = (1 - self.params.reduce_factor) ** n_retries
batch_size = int(self.params.event_filtering_batch_size * keep_fraction)
msg = (
keep_fraction = 1 - self.params.reduce_factor
batch_size = int(batch_size * keep_fraction)
Adamantios marked this conversation as resolved.
Show resolved Hide resolved
self.redeeming_progress.event_filtering_batch_size = batch_size
self.context.logger.warning(
f"Repeating this call with a decreased batch size of {batch_size}."
)
self.context.logger.warning(msg)
continue

self.redeeming_progress.payouts.update(self.payouts_batch)
Expand Down Expand Up @@ -565,7 +565,7 @@ def get_claim_params(self) -> WaitableConditionType:

n_retries = 0
from_block = self.redeeming_progress.claim_from_block
batch_size = self.params.event_filtering_batch_size
batch_size = self.redeeming_progress.event_filtering_batch_size
while from_block < self.redeeming_progress.claim_to_block:
max_to_block = from_block + batch_size
to_block = min(max_to_block, self.redeeming_progress.claim_to_block)
Expand All @@ -585,12 +585,12 @@ def get_claim_params(self) -> WaitableConditionType:

if not result:
n_retries += 1
keep_fraction = (1 - self.params.reduce_factor) ** n_retries
batch_size = int(self.params.event_filtering_batch_size * keep_fraction)
msg = (
keep_fraction = 1 - self.params.reduce_factor
batch_size = int(batch_size * keep_fraction)
Adamantios marked this conversation as resolved.
Show resolved Hide resolved
self.redeeming_progress.event_filtering_batch_size = batch_size
self.context.logger.warning(
f"Repeating this call with a decreased batch size of {batch_size}."
)
self.context.logger.warning(msg)
continue

self.redeeming_progress.answered.extend(self.claim_params_batch)
Expand Down
8 changes: 8 additions & 0 deletions packages/valory/skills/decision_maker_abci/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ class RedeemingProgress:
policy: Optional[EGreedyPolicy] = None
claimable_amounts: Dict[HexBytes, int] = field(default_factory=lambda: {})
earliest_block_number: int = 0
event_filtering_batch_size: int = 0
check_started: bool = False
check_from_block: BlockIdentifier = "earliest"
check_to_block: BlockIdentifier = "latest"
Expand Down Expand Up @@ -139,6 +140,13 @@ def __init__(self, *args: Any, skill_context: SkillContext, **kwargs: Any) -> No
super().__init__(*args, skill_context=skill_context, **kwargs)
self.redeeming_progress: RedeemingProgress = RedeemingProgress()

def setup(self) -> None:
"""Set up the model."""
super().setup()
self.redeeming_progress.event_filtering_batch_size = (
self.context.params.event_filtering_batch_size
)


def extract_keys_from_template(delimiter: str, template: str) -> Set[str]:
"""Extract the keys from a string template, given the delimiter."""
Expand Down
8 changes: 4 additions & 4 deletions packages/valory/skills/decision_maker_abci/skill.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@ fingerprint:
behaviours/decision_receive.py: bafybeifacce2ke7oltnwnpdjdqfd74eaaw5wxnjfzk6c5tqdsxsmbzjj3m
behaviours/decision_request.py: bafybeievr7vae43e7jr4eqqhwe3emvgiih7ysa66jcb5g2oz5lbxua232q
behaviours/handle_failed_tx.py: bafybeidxpc6u575ymct5tdwutvzov6zqfdoio5irgldn3fw7q3lg36mmxm
behaviours/reedem.py: bafybeidlyt2bie2luguptagq3u7xytdeftckdnmpql4gzqrzdkqjspuyp4
behaviours/reedem.py: bafybeiekq77eatvf7xczbnyd4s6met4ci57ethwwoyxf5mhrcq7vuwiaxu
behaviours/round_behaviour.py: bafybeig4tdktyu6hapoqymnxh2bgpds547st6a44heue657wkctwe4gjvm
behaviours/sampling.py: bafybeifzhm4sspdvt227ksl5hjn26offgqpwempgbcwbr6dq7gyi2a46sm
behaviours/tool_selection.py: bafybeigfr2frkljrxyfxs5p3j42equzehgaqtkyuxk6eiujyudr6ajqakm
dialogues.py: bafybeigpwuzku3we7axmxeamg7vn656maww6emuztau5pg3ebsoquyfdqm
fsm_specification.yaml: bafybeifnob3ceim2mj7lqagtnpwqjqqxs5eg3oiwc73gwm6x5i2dvvlcya
handlers.py: bafybeihj33szgrcxnpd73s4nvluyxwwsvhjum2cuq3ilhhe6vfola3k7vy
models.py: bafybeiathvz2s7bhc4w4gggjoxomzwdxqtdnziq7afc335k5z32kpswgle
models.py: bafybeihmqhxixmklpp656o7i27fas2afabusmar2heyfnx5e3kgzdcvkji
payloads.py: bafybeigcic4vewdglakzpyqevhfwsolh4ywnbvxo5bgned7gl5uo3jif7m
policy.py: bafybeidpmx4ek3qze63zpuwixyf6t7bdv62ewgkzt3ljrzadiwdw64cueq
redeem_info.py: bafybeibddfxwp3577c3dl2utaowwltquu5fg6crezpumoebw563wxpbfrm
Expand All @@ -52,8 +52,8 @@ contracts:
- valory/erc20:0.1.0:bafybeiepg5ymxhtylgijs3mf3lezz6cwcsougbrv2gr4xcq3bp5wxusi64
- valory/multisend:0.1.0:bafybeig5byt5urg2d2bsecufxe5ql7f4mezg3mekfleeh32nmuusx66p4y
- valory/mech:0.1.0:bafybeieo65rtidzgrx22la7z7azoqbisd35hnoqfw7l3xxctotvctjlkju
- valory/conditional_tokens:0.1.0:bafybeihnhs4ttjttlba2vqwyufsundne4oazblbvwwlq5x7isklnslczju
- valory/realitio:0.1.0:bafybeiejybwx4d3iwaxumxuox7vbub2syyzycqnxaflnjakrgcf4z5gcle
- valory/conditional_tokens:0.1.0:bafybeifu5axib5ifzq6bomfscs7nnx5qknkzymlz6gfn7ohjsb2shghrei
- valory/realitio:0.1.0:bafybeieoily22pasgkzzcyawhqyrsko7b52lghnvwuskhomavb7tlb6pxa
- valory/realitio_proxy:0.1.0:bafybeidx37xzjjmapwacedgzhum6grfzhp5vhouz4zu3pvpgdy5pgb2fr4
- valory/agent_registry:0.1.0:bafybeibezt4xtzgf25eidmwev6moki74eufmb4hx7cpvd6odbyoxo42hxi
protocols:
Expand Down
4 changes: 2 additions & 2 deletions packages/valory/skills/trader_abci/skill.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ skills:
- valory/transaction_settlement_abci:0.1.0:bafybeigk3debp6dswutqsuls2lqfvyj4ghe6kwjc2zfinnsvj6hujynxtq
- valory/termination_abci:0.1.0:bafybeigainmpy37gb33ogbrpggjbkuasodhthh5oz4vbzjdfjozerpf224
- valory/market_manager_abci:0.1.0:bafybeic7o4pclkhnugyn7js5g3asxuqhkxpvunlp3mpup7aovhg2fto22i
- valory/decision_maker_abci:0.1.0:bafybeiezxeb4vwsuzbfbka5emvzqlzs4zghcqyt6ilatdakokiuwqkejoy
- valory/tx_settlement_multiplexer_abci:0.1.0:bafybeig6pwr7zcou53qehlytced5rdltue7phfttt4ms55hk2bk3brfpve
- valory/decision_maker_abci:0.1.0:bafybeigdp4pgm4ons5pzptmclyglyxyqut266s4zutpmqj2e22wdpdkt5y
- valory/tx_settlement_multiplexer_abci:0.1.0:bafybeic6bveoijtjzvquphekmset7sondro4iknljf5qnylkadeeusiwp4
- valory/staking_abci:0.1.0:bafybeia3acxumkjg566dvgagv3swz7htf5xpvwq6ovmecvp5j2zdxdoabe
behaviours:
main:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ protocols:
- valory/ledger_api:1.0.0:bafybeige5agrztgzfevyglf7mb4o7pzfttmq4f6zi765y4g2zvftbyowru
skills:
- valory/abstract_round_abci:0.1.0:bafybeidau7loztcfy3mxvoqrv7otbpciemd2wf3lsxyjraq4dcvuvib25e
- valory/decision_maker_abci:0.1.0:bafybeiezxeb4vwsuzbfbka5emvzqlzs4zghcqyt6ilatdakokiuwqkejoy
- valory/decision_maker_abci:0.1.0:bafybeigdp4pgm4ons5pzptmclyglyxyqut266s4zutpmqj2e22wdpdkt5y
- valory/staking_abci:0.1.0:bafybeia3acxumkjg566dvgagv3swz7htf5xpvwq6ovmecvp5j2zdxdoabe
behaviours:
main:
Expand Down