Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unstable output for the state aka observation #232

Open
Abdelkader-gnichi opened this issue Feb 17, 2023 · 0 comments
Open

unstable output for the state aka observation #232

Abdelkader-gnichi opened this issue Feb 17, 2023 · 0 comments

Comments

@Abdelkader-gnichi
Copy link

Abdelkader-gnichi commented Feb 17, 2023

i run my code and every time i get a different out put so just i want an explanation why i have different out put
#########################################################################################
this is my code :

import numpy as np
import gym
import random
import matplotlib.pyplot as plt
from random import choice
from tqdm.notebook import tqdm
from kaggle_environments import evaluate, make

class ConnectX(gym.Env):
def init(self, switch_prob=0.5):
self.env = make('connectx', debug=True)
self.pair = [None, 'negamax']
self.trainer = self.env.train(self.pair)
self.switch_prob = switch_prob

    # Define required gym fields (examples):
    config = self.env.configuration
    self.action_space = gym.spaces.Discrete(config.columns)
    self.observation_space = gym.spaces.Discrete(config.columns * config.rows)

def switch_trainer(self):
    self.pair = self.pair[::-1]
    self.trainer = self.env.train(self.pair)

def step(self, action):
    return self.trainer.step(action)

def reset(self):
    if random.uniform(0, 1) < self.switch_prob:
        self.switch_trainer()
    return self.trainer.reset()

def render(self, **kwargs):
    return self.env.render(**kwargs)

class QTable:
def init(self, action_space):
self.table = dict()
self.action_space = action_space

def add_item(self, state_key):
    self.table[state_key] = list(np.zeros(self.action_space.n))
    
def __call__(self, state):
    board = state['board'][:] # Get a copy
    board.append(state.mark)
    state_key = np.array(board).astype(str)
    state_key = hex(int(''.join(state_key), 3))[2:]
    if state_key not in self.table.keys():
        self.add_item(state_key)
    
    return self.table[state_key]

env = ConnectX()

alpha = 0.1
gamma = 0.6
epsilon = 0.99
min_epsilon = 0.1

episodes = 10000

alpha_decay_step = 1000
alpha_decay_rate = 0.9
epsilon_decay_rate = 0.9999

q_table = QTable(env.action_space)

all_epochs = []
all_total_rewards = []
all_avg_rewards = [] # Last 100 steps
all_qtable_rows = []
all_epsilons = []

state = env.reset()
print(state.board) #this is the tricky
###########################################################################################
this is the different output:

/bin/python /home/gadour/Music/beginig.py
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
/bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
Traceback (most recent call last):
File "/home/gadour/Music/beginig.py", line 78, in
print(state.board)
^^^^^^^^^^^
AttributeError: 'Struct' object has no attribute 'board'
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
/bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
/bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
/bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
/bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
/bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
Traceback (most recent call last):
File "/home/gadour/Music/beginig.py", line 78, in
print(state.board)
^^^^^^^^^^^
AttributeError: 'Struct' object has no attribute 'board'
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
Traceback (most recent call last):
File "/home/gadour/Music/beginig.py", line 78, in
print(state.board)
^^^^^^^^^^^
AttributeError: 'Struct' object has no attribute 'board'
[gadour@fedora ns-3.37]$ /bin/python /home/gadour/Music/beginig.py
Loading environment lux_ai_s2 failed: No module named 'vec_noise'
Traceback (most recent call last):
File "/home/gadour/Music/beginig.py", line 78, in
print(state.board)
^^^^^^^^^^^
###########################################################################################
So guys why the hell some times return to me the list and some times just give me an error saying 'Struct' object has no attribute 'board' why that happen please i want logical explanation and thnx

Note: I know if i use state['board'] it work better, but i want to know why the unstable output it's 04:48pm now i can't sleep thinking, so please i'm so excited to know why it goes like that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant