Questions about the KAQN implementation

[moripiri]

Hello. As someone who are interested in RL and KAN, it's great to see that someone is already working on combining KAN and RL.

It's really exciting to find out that KAQN improves the performance and efficiency of RL, and now I'm planning to extend it to other RL algorithms(ex. TD3, SAC, etc). Yet before starting it I want to ask few questions about your implementation.

1. offline training

• KAQN seems to train after an episode ends, which is different to normal DQN implementation(training every step), and config.train_steps is set to 5, which seems to be very small in my opinion. Does training KAQN every step leads to training failure?

2. number of episode-based training schedule

• KAQN seems to schedule warming up(random action), copying q-network to target q-network, update_grid_from_samples. Yet doing it by number of episodes will not guarantee equal training setting by each run. Is it done on purpose?