Help with developing a HGF model for foraging task

[kshtjkumar]

Hi, I'm new to HGF and wondering if it can work for my task.

I have an 8-armed bandit setup:

4 arms are rewarding, 4 are non-rewarding (participants don't know this).
Rewards are deterministic, not probabilistic.
Reward reduction: current_reward - 5 (on continuous harvest).
Replenishment: current_reward + 5 (when switching to another bandit).

Data includes:

Bandit choices.
Received reward.
Rewards available in other bandits at the time of choice.
Can HGF handle deterministic rewards like this? Any guidance on setting it up would be super helpful!

Thanks!

[LegrandNico]

Hi @kshtjkumar , also cc @PTWaade and @SylvainEstebe here,

Thank you for raising this interesting question. I think it actually fit well with the current direction we want to take regarding the development of the toolbox so I would be interested in trying to implement that. I would say it is definitely possible, we just need to think a bit about what would be the best API going forward.

Could you please describe what is happening exactly at each trial? If you already have code to simulate this task or can point to a specific paper that can be immensely helpful as well.

Looking forward to hearing from you :)

[kshtjkumar]

Hi @LegrandNico,

Thank you for your response, and apologies for the delayed reply. We are in the process of publishing the task, so I don’t have a paper to share yet. However, I’ll do my best to explain the trial structure in simple steps:

Participants interact with a set of 8 bandits (some rewarding some are non rewarding or "zero rewarding"), arranged in a way that choosing a farther bandit requires more time. They are given a total of 5 minutes to play.

When a participant selects a bandit, they receive a reward (e.g., 70). Repeated selections of the same bandit result in diminishing rewards (e.g., 60, then 50, and so on). Meanwhile, other rewarding bandits increase in their potential reward values, but this increase only happens when a rewarding bandit is harvested, and it does not apply to the bandit currently being used.

Let me know if you have further questions!

[PTWaade]

Hi @kshtjkumar - just chiming in here to say that this sounds exciting :)

So the normal way to model a standard multi-armed bandit task is to just have a classic HGF for each node. This means the rewards of each bandit is modeled as a value that might change. Then the expected rewards for each bandit can be used for some response function.
Since there is a steady decline, it would probably be appropriate to add a value parent (a "drift parent") to each of the HGF's - so that it'll learn for each HGF how it changes with each button press. This should let it learn the decrease or increase in value that happens each time a bandit is selected (I hope I understand this correctly: the "bad" bandits reduce in value over time, and the "good" bandits increase).
Finally, one could add a shared value parent if it is hypothesized that participants think some changes generalize between bandits, or similar.

Is this helpful? If I have misunderstood anything about the task, please do elaborate :)

All the best,
Peter

[kshtjkumar]

Hi @PTWaade,

Sorry for the delayed response! No, not all bandits provide reduced rewards when exploited continuously. Think of it like a tree—if you keep picking its fruit one after another, the available fruit decreases with each step. Meanwhile, other trees that you haven’t picked from continue to grow more fruit, increasing their potential yield. However, if you switch to a different tree, the previous one starts replenishing, eventually offering a higher reward again.

Does this explanation make sense? Let me know if anything needs further clarification!