flexs.utils.replay_buffers¶

Defines replay buffers used by some explorers.

class flexs.utils.replay_buffers.MinSegmentTree(capacity)[source]¶

Bases: flexs.utils.replay_buffers.SegmentTree

Create SegmentTree. Taken from OpenAI baselines Github repository: https://github.com/openai/baselines/blob/master/baselines/common/segment_tree.py

min(start=0, end=0)[source]¶

Return min(arr[start], …, arr[end]).

Return type: float

class flexs.utils.replay_buffers.PrioritizedReplayBuffer(obs_dim, size, batch_size=32, alpha=0.6)[source]¶

Bases: flexs.utils.replay_buffers.ReplayBuffer

Prioritized Replay buffer.

max_priority[source]¶

max priority

Type: float

tree_ptr[source]¶

next index of tree

Type: int

alpha[source]¶

alpha parameter for prioritized replay buffer

Type: float

sum_tree[source]¶

sum tree for prior

Type: SumSegmentTree

min_tree[source]¶

min tree for min prior to get max weight

Type: MinSegmentTree

sample_batch(beta=0.4)[source]¶

Sample a batch of experiences.

Return type: Dict[str, ndarray]

store(obs, act, rew, next_obs)[source]¶: Store experience and priority.

update_priorities(indices, priorities)[source]¶: Update priorities of sampled transitions.

class flexs.utils.replay_buffers.ReplayBuffer(obs_dim, size, batch_size=128)[source]¶

Bases: object

A simple numpy replay buffer.

sample_batch()[source]¶

Sample batch of timesteps from replay buffer.

Return type: Dict[str, ndarray]

store(obs, act, rew, next_obs)[source]¶: Store timestep in replay buffer.

class flexs.utils.replay_buffers.SegmentTree(capacity, operation, init_value)[source]¶

Bases: object

Create SegmentTree. Taken from OpenAI baselines Github repository: https://github.com/openai/baselines/blob/master/baselines/common/segment_tree.py

capacity[source]¶

Type: int

tree[source]¶

Type: list

operation[source]¶

Type: function

operate(start=0, end=0)[source]¶

Return result of applying self.operation.

Return type: float

class flexs.utils.replay_buffers.SumSegmentTree(capacity)[source]¶

Bases: flexs.utils.replay_buffers.SegmentTree

Create SumSegmentTree. Taken from OpenAI baselines github repository: https://github.com/openai/baselines/blob/master/baselines/common/segment_tree.py

retrieve(upperbound)[source]¶

Find the highest index i about upper bound in the tree

Return type: int

sum(start=0, end=0)[source]¶

Return arr[start] + … + arr[end].

Return type: float