Scikit-decide

# core

Domain specification

Domain

# Space

A space representing a finite or infinite set.

This class (or any of its descendant) is typically used to specify action, observation or goal spaces.

# contains Space

contains(
  self,
  x: T
) -> bool

Check whether x is a valid member of this space.

# Parameters

  • x: The member to consider.

# Returns

True if x is a valid member of this space (False otherwise).

# ImplicitSpace

A space formalized implicitly, i.e. by a black-box contains() function.

# Constructor ImplicitSpace

ImplicitSpace(
  contains_function: Callable[[T], bool]
) -> None

Initialize ImplicitSpace.

# Parameters

  • contains_function: The contains() function to use.

# Example

my_space = ImplicitSpace(lambda x: 10 > x['position'] > 5)

# contains Space

contains(
  self,
  x: T
) -> bool

Check whether x is a valid member of this space.

# Parameters

  • x: The member to consider.

# Returns

True if x is a valid member of this space (False otherwise).

# EnumerableSpace

A space which elements can be enumerated.

# contains Space

contains(
  self,
  x: T
) -> bool

Check whether x is a valid member of this space.

# Parameters

  • x: The member to consider.

# Returns

True if x is a valid member of this space (False otherwise).

# get_elements EnumerableSpace

get_elements(
  self
) -> Iterable[T]

Get the elements of this space.

# Returns

The elements of this space.

# EmptySpace

An (enumerable) empty space.

# contains Space

contains(
  self,
  x: T
) -> bool

Check whether x is a valid member of this space.

# Parameters

  • x: The member to consider.

# Returns

True if x is a valid member of this space (False otherwise).

# get_elements EnumerableSpace

get_elements(
  self
) -> Iterable[T]

Get the elements of this space.

# Returns

The elements of this space.

# SamplableSpace

A space which can be sampled (uniformly randomly).

# contains Space

contains(
  self,
  x: T
) -> bool

Check whether x is a valid member of this space.

# Parameters

  • x: The member to consider.

# Returns

True if x is a valid member of this space (False otherwise).

# sample SamplableSpace

sample(
  self
) -> T

Uniformly randomly sample a random element of this space.

# Returns

The sampled element.

# SerializableSpace

A space which can be serialized (to/from JSON).

# contains Space

contains(
  self,
  x: T
) -> bool

Check whether x is a valid member of this space.

# Parameters

  • x: The member to consider.

# Returns

True if x is a valid member of this space (False otherwise).

# from_jsonable SerializableSpace

from_jsonable(
  self,
  sample_n: Sequence
) -> Iterable[T]

Convert a JSONable data type to a batch of samples from this space.

# Parameters

  • sample_n: The JSONable data type to convert.

# Returns

The resulting batch of samples.

# to_jsonable SerializableSpace

to_jsonable(
  self,
  sample_n: Iterable[T]
) -> Sequence

Convert a batch of samples from this space to a JSONable data type.

# Parameters

  • sample_n: The batch of samples to convert.

# Returns

The resulting JSONable data type.

# Distribution

A probability distribution.

# sample Distribution

sample(
  self
) -> T

Sample from this distribution.

# Returns

The sampled element.

# ImplicitDistribution

A probability distribution formalized implicitly, i.e. by a black-box sample() function.

# Constructor ImplicitDistribution

ImplicitDistribution(
  sample_function: Callable[[], T]
) -> None

Initialize ImplicitDistribution.

# Parameters

  • sample_function: The sample() function to use.

# Example

import random

dice = ImplicitDistribution(lambda: random.randint(1, 6))
roll = dice.sample()

# sample Distribution

sample(
  self
) -> T

Sample from this distribution.

# Returns

The sampled element.

# DiscreteDistribution

A discrete probability distribution.

# Constructor DiscreteDistribution

DiscreteDistribution(
  values: list[tuple[T, float]]
) -> None

Initialize DiscreteDistribution.

TIP

If the given probabilities do not sum to 1, they are implicitly normalized as such for sampling.

# Parameters

  • values: The list of (element, probability) pairs.

# Example

game_strategy = DiscreteDistribution([('rock', 0.7), ('paper', 0.1), ('scissors', 0.2)])
move = game_strategy.sample()

# get_values DiscreteDistribution

get_values(
  self
) -> list[tuple[T, float]]

Get the list of (element, probability) pairs.

# Returns

The (element, probability) pairs.

# sample Distribution

sample(
  self
) -> T

Sample from this distribution.

# Returns

The sampled element.

# SingleValueDistribution

A single value distribution (i.e. Dirac distribution).

# Constructor SingleValueDistribution

SingleValueDistribution(
  value: T
) -> None

Initialize SingleValueDistribution.

# Parameters

  • value: The single value of this distribution.

# get_value SingleValueDistribution

get_value(
  self
) -> T

Get the single value of this distribution.

# Returns

The single value of this distribution.

# get_values DiscreteDistribution

get_values(
  self
) -> list[tuple[T, float]]

Get the list of (element, probability) pairs.

# Returns

The (element, probability) pairs.

# sample Distribution

sample(
  self
) -> T

Sample from this distribution.

# Returns

The sampled element.

# Value

A value (reward or cost).

WARNING

It is recommended to use either the reward or the cost parameter. If no one is used, a reward/cost of 0 is assumed. If both are used, reward will be considered and cost ignored. In any case, both reward and cost attributes will be defined after initialization.

# Parameters

  • reward: The optional reward.
  • cost: The optional cost.

# Example

# These two lines are equivalent, use the one you prefer
value_1 = Value(reward=-5)
value_2 = Value(cost=5)

assert value_1.reward == value_2.reward == -5  # True
assert value_1.cost == value_2.cost == 5  # True

# EnvironmentOutcome

An environment outcome for an internal transition.

# Parameters

  • observation: The agent's observation of the current environment.
  • value: The value (reward or cost) returned after previous action.
  • termination: Whether the episode has ended, in which case further step() calls will return undefined results.
  • info: Optional auxiliary diagnostic information (helpful for debugging, and sometimes learning).

# asdict ExtendedDataclass

asdict(
  self
)

Return the fields of the instance as a new dictionary mapping field names to field values.

# astuple ExtendedDataclass

astuple(
  self
)

Return the fields of the instance as a new tuple of field values.

# replace ExtendedDataclass

replace(
  self,
  **changes
)

Return a new object replacing specified fields with new values.

# TransitionOutcome

A transition outcome.

# Parameters

  • state: The new state after the transition.
  • value: The value (reward or cost) returned after previous action.
  • termination: Whether the episode has ended, in which case further step() calls will return undefined results.
  • info: Optional auxiliary diagnostic information (helpful for debugging, and sometimes learning).

# asdict ExtendedDataclass

asdict(
  self
)

Return the fields of the instance as a new dictionary mapping field names to field values.

# astuple ExtendedDataclass

astuple(
  self
)

Return the fields of the instance as a new tuple of field values.

# replace ExtendedDataclass

replace(
  self,
  **changes
)

Return a new object replacing specified fields with new values.

# StrDict

A dictionary with String keys (e.g. agent names).

# Constraint

A constraint.

# check Constraint

check(
  self,
  memory: Memory,
  action: D.T_event,
  next_state: Optional[D.T_state] = None
) -> bool

Check this constraint.

TIP

If this function never depends on the next_state parameter for its computation, it is recommended to indicate it by overriding Constraint._is_constraint_dependent_on_next_state_() to return False. This information can then be exploited by solvers to avoid computing next state to evaluate the constraint (more efficient).

# Parameters

  • memory: The source memory (state or history) of the transition.
  • action: The action taken in the given memory (state or history) triggering the transition.
  • next_state: The next state in which the transition ends (if needed for the computation).

# Returns

True if the constraint is checked (False otherwise).

# is_constraint_dependent_on_next_state Constraint

is_constraint_dependent_on_next_state(
  self
) -> bool

Indicate whether this constraint requires the next_state parameter for its computation (cached).

By default, Constraint.is_constraint_dependent_on_next_state() internally calls Constraint._is_constraint_dependent_on_next_state_() the first time and automatically caches its value to make future calls more efficient (since the returned value is assumed to be constant).

# Returns

True if the constraint computation depends on next_state (False otherwise).

# _is_constraint_dependent_on_next_state_ Constraint

_is_constraint_dependent_on_next_state_(
  self
) -> bool

Indicate whether this constraint requires the next_state parameter for its computation.

This is a helper function called by default from Constraint.is_constraint_dependent_on_next_state(), the difference being that the result is not cached here.

TIP

The underscore at the end of this function's name is a convention to remind that its result should be constant.

# Returns

True if the constraint computation depends on next_state (False otherwise).

# ImplicitConstraint

A constraint formalized implicitly, i.e. by a black-box check() function.

# Constructor ImplicitConstraint

ImplicitConstraint(
  check_function: Callable[[Memory, D.T_event, Optional[D.T_state]], bool],
  depends_on_next_state: bool = True
) -> None

Initialize ImplicitConstraint.

# Parameters

  • check_function: The check() function to use.
  • depends_on_next_state: Whether the check() function requires the next_state parameter for its computation.

# Example

constraint = ImplicitConstraint(lambda memory, action, next_state: next_state.x % 2 == 0)

# check Constraint

check(
  self,
  memory: Memory,
  action: D.T_event,
  next_state: Optional[D.T_state] = None
) -> bool

Check this constraint.

TIP

If this function never depends on the next_state parameter for its computation, it is recommended to indicate it by overriding Constraint._is_constraint_dependent_on_next_state_() to return False. This information can then be exploited by solvers to avoid computing next state to evaluate the constraint (more efficient).

# Parameters

  • memory: The source memory (state or history) of the transition.
  • action: The action taken in the given memory (state or history) triggering the transition.
  • next_state: The next state in which the transition ends (if needed for the computation).

# Returns

True if the constraint is checked (False otherwise).

# is_constraint_dependent_on_next_state Constraint

is_constraint_dependent_on_next_state(
  self
) -> bool

Indicate whether this constraint requires the next_state parameter for its computation (cached).

By default, Constraint.is_constraint_dependent_on_next_state() internally calls Constraint._is_constraint_dependent_on_next_state_() the first time and automatically caches its value to make future calls more efficient (since the returned value is assumed to be constant).

# Returns

True if the constraint computation depends on next_state (False otherwise).

# _is_constraint_dependent_on_next_state_ Constraint

_is_constraint_dependent_on_next_state_(
  self
) -> bool

Indicate whether this constraint requires the next_state parameter for its computation.

This is a helper function called by default from Constraint.is_constraint_dependent_on_next_state(), the difference being that the result is not cached here.

TIP

The underscore at the end of this function's name is a convention to remind that its result should be constant.

# Returns

True if the constraint computation depends on next_state (False otherwise).

# BoundConstraint

A constraint characterized by an evaluation function, an inequality and a bound.

# Example

A BoundConstraint with inequality '>=' is checked if (and only if) its BoundConstraint.evaluate() function returns a float greater than or equal to its bound.

# Constructor BoundConstraint

BoundConstraint(
  evaluate_function: Callable[[Memory, D.T_event, Optional[D.T_state]], float],
  inequality: str,
  bound: float,
  depends_on_next_state: bool = True
) -> None

Initialize BoundConstraint.

# Parameters

  • evaluate_function: The evaluate() function to use.
  • inequality: A string ('<', '<=', '>' or '>=') describing the constraint inequality.
  • bound: The bound of the constraint.
  • depends_on_next_state: Whether the evaluate() function requires the next_state parameter for its computation.

# Example

constraint = BoundConstraint((lambda memory, action, next_state: next_state.x), '>', 5.)

# check Constraint

check(
  self,
  memory: Memory,
  action: D.T_event,
  next_state: Optional[D.T_state] = None
) -> bool

Check this constraint.

TIP

If this function never depends on the next_state parameter for its computation, it is recommended to indicate it by overriding Constraint._is_constraint_dependent_on_next_state_() to return False. This information can then be exploited by solvers to avoid computing next state to evaluate the constraint (more efficient).

# Parameters

  • memory: The source memory (state or history) of the transition.
  • action: The action taken in the given memory (state or history) triggering the transition.
  • next_state: The next state in which the transition ends (if needed for the computation).

# Returns

True if the constraint is checked (False otherwise).

# evaluate BoundConstraint

evaluate(
  self,
  memory: Memory,
  action: D.T_event,
  next_state: Optional[D.T_state] = None
) -> float

Evaluate the left side of this BoundConstraint.

TIP

If this function never depends on the next_state parameter for its computation, it is recommended to indicate it by overriding Constraint._is_constraint_dependent_on_next_state_() to return False. This information can then be exploited by solvers to avoid computing next state to evaluate the constraint (more efficient).

# Parameters

  • memory: The source memory (state or history) of the transition.
  • action: The action taken in the given memory (state or history) triggering the transition.
  • next_state: The next state in which the transition ends (if needed for the computation).

# Returns

The float value resulting from the evaluation.

# get_bound BoundConstraint

get_bound(
  self
) -> float

Get the bound of the constraint.

# Returns

The constraint bound.

# get_inequality BoundConstraint

get_inequality(
  self
) -> str

Get the string ('<', '<=', '>' or '>=') describing the constraint inequality.

# Returns

The string describing the inequality.

# is_constraint_dependent_on_next_state Constraint

is_constraint_dependent_on_next_state(
  self
) -> bool

Indicate whether this constraint requires the next_state parameter for its computation (cached).

By default, Constraint.is_constraint_dependent_on_next_state() internally calls Constraint._is_constraint_dependent_on_next_state_() the first time and automatically caches its value to make future calls more efficient (since the returned value is assumed to be constant).

# Returns

True if the constraint computation depends on next_state (False otherwise).

# _is_constraint_dependent_on_next_state_ Constraint

_is_constraint_dependent_on_next_state_(
  self
) -> bool

Indicate whether this constraint requires the next_state parameter for its computation.

This is a helper function called by default from Constraint.is_constraint_dependent_on_next_state(), the difference being that the result is not cached here.

TIP

The underscore at the end of this function's name is a convention to remind that its result should be constant.

# Returns

True if the constraint computation depends on next_state (False otherwise).