Discrete state/observation with one_hot encoding.

[Elfits]

Here is the issue about the discrete state or observation with one_hot encoding.
Discrete state/observation utilized the gym.env.Discrete.

For discrete tasks.

[Elfits]

FYI.
Open AI does have envs with discrete observation space, such as the blackjack.
Inside blackjack.py, self.observation_space is defined by
self.observation_space = spaces.Tuple(( spaces.Discrete(32), spaces.Discrete(11), spaces.Discrete(2)))

So action_dim=env.action_space.n, may need to be modified.

[Elfits]

FYI.
If using spaces.Tuple for the observation spaces, data format will be different. Please see gym/spaces/tuple.py
#TODO:
Initialization of any network, such as dqn.py
get_replay_buffer.py

def get_space_size(space):
    if isinstance(space, Box):
        return space.shape
    elif isinstance(space, Discrete):
        return [1, ]  # space.n
    else:
        raise NotImplementedError("Assuming to use Box or Discrete")

[ymd-h]

Hi,

I checked and found Open AI gym provides utility functions for non-simple spaces
https://github.com/openai/gym/blob/master/gym/spaces/utils.py

• flatdim
• flatten
• unflatten

They are accessible from gym.spaces
https://github.com/openai/gym/blob/master/gym/spaces/__init__.py

For Discrete, one hot vector encoding is applied, so that just use these utility functions can solve the problems.

[ymd-h]

Hi, there are additional information.

https://openai.com/blog/ingredients-for-robotics-research/
OpenAI official blog recommend to use gym.wrappers.FlattenDictWrapper for dictionary base observation such as "robotics".

However, the wrapper does not exist any more, and probably it had been generalized to gym.wrappers.FlattenObservation
https://github.com/openai/gym/blob/master/gym/wrappers/__init__.py#L7

The wrapper uses the flatten function mentioned above.

So, in principle, Tuple, Dict, Discrete and so on can be used with this wrapper.

Note: That flatten function makes one hot vector for Discrete environment, so it is not memory efficient to convert before being stored into replay buffer.