observation_buffer.py has a copy error for a small percentage

[luoye2333]

Hello authors in ETH, great thanks for your excellent works! This repo has helped me very very much in my study and work. Recently I found a small error when i try to write a different implementation of the observation_buffer.py.

We should clone first before writing into the buffer, or we have overwrite risks for a small part (less than 1% in my test) of the data at the end of the buffer.

    def insert(self, new_obs):
        # Shift observations back.
        self.obs_buf[:, : self.num_obs * (self.include_history_steps - 1)] = self.obs_buf[:,self.num_obs : self.num_obs * self.include_history_steps].clone() # add this .clone()!!

        # Add new observation.
        self.obs_buf[:, -self.num_obs:] = new_obs

The percentage of the error data is very small so it does not affect the RL training at all. But the added .clone() will increase the time consumption to 2 times of the original way, so i recommend using a circular implementation. We just move the pointer when write a new observation so that we do not need to move the whole buffer. Refer to the test script in observation_buffer_3d_circular.py for testing.
The test results are as follows:

<class 'observation_buffer.ObservationBuffer'> insert 10000 using: 1.1655 seconds
<class 'observation_buffer_3d.ObservationBuffer3D'> insert 10000 using: 1.2878 seconds
<class '__main__.ObservationBuffer3D_circular'> insert 10000 using: 0.2963 seconds
0.0 0.0
larget batch error test: ok

Hoping for your reply!

[luoye2333]

Found that it is already fixed in Isaaclab circular_buffer.py. Ignore this PR.