Optimization Module

Optimization algorithm and reinforcement learning integration interfaces.

Optimization Interface

Base interfaces for simulation-optimization.

class simcraft.optimization.base.ObjectiveType[source]

Bases: Enum

Types of optimization objectives.

MINIMIZE = 1

MAXIMIZE = 2

class simcraft.optimization.base.Parameter[source]

Bases: object

A simulation parameter for optimization.

name

Parameter name

Type:: str

lower_bound

Minimum value

Type:: float

upper_bound

Maximum value

Type:: float

initial_value

Starting value

Type:: float

is_integer

Whether parameter must be integer

Type:: bool

description

Parameter description

Type:: str

name: str

lower_bound: float

upper_bound: float

initial_value: float | None = None

is_integer: bool = False

description: str = ''

validate(value)[source]

Validate and clip value to bounds.

Parameters:: value (float) – Value to validate
Returns:: Validated value
Return type:: float

__init__(name, lower_bound, upper_bound, initial_value=None, is_integer=False, description='')

Parameters:

name (str)
lower_bound (float)
upper_bound (float)
initial_value (float | None)
is_integer (bool)
description (str)

Return type:

None

class simcraft.optimization.base.SimulationObjective[source]

Bases: object

An objective function for optimization.

name

Objective name

Type:: str

direction

Minimize or maximize

Type:: ObjectiveType

weight

Weight for multi-objective (default 1.0)

Type:: float

target

Target value for satisfaction-based objectives

Type:: Optional[float]

name: str

direction: ObjectiveType = 1

weight: float = 1.0

target: float | None = None

property is_minimization: bool: Check if this is a minimization objective.

normalize(value, worst, best)[source]

Normalize objective value to [0, 1].

Parameters:

value (float) – Raw objective value
worst (float) – Worst observed value
best (float) – Best observed value

Returns:

Normalized value (0 = worst, 1 = best)

Return type:

float

__init__(name, direction=ObjectiveType.MINIMIZE, weight=1.0, target=None)

Parameters:

name (str)
direction (ObjectiveType)
weight (float)
target (float | None)

Return type:

None

class simcraft.optimization.base.EvaluationResult[source]

Bases: object

Result of a simulation evaluation.

parameters

Parameter values used

Type:: Dict[str, float]

objectives

Objective values obtained

Type:: Dict[str, float]

constraints

Constraint satisfaction

Type:: Dict[str, bool]

simulation_time

Final simulation time

Type:: float

replications

Number of replications

Type:: int

metadata

Additional evaluation data

Type:: Dict[str, Any]

parameters: Dict[str, float]

objectives: Dict[str, float]

constraints: Dict[str, bool]

simulation_time: float = 0.0

replications: int = 1

metadata: Dict[str, Any]

property is_feasible: bool: Check if all constraints are satisfied.

to_dict()[source]

Convert to dictionary.

Return type:: dict

__init__(parameters, objectives, constraints=<factory>, simulation_time=0.0, replications=1, metadata=<factory>)

Parameters:

parameters (Dict[str, float])
objectives (Dict[str, float])
constraints (Dict[str, bool])
simulation_time (float)
replications (int)
metadata (Dict[str, Any])

Return type:

None

class simcraft.optimization.base.OptimizationInterface[source]

Bases: ABC

Abstract interface for simulation-optimization integration.

Subclass this to create an optimizable simulation model.

Examples

>>> class MyOptModel(OptimizationInterface):
...     def get_parameters(self):
...         return [Parameter("capacity", 1, 10, is_integer=True)]
...
...     def get_objectives(self):
...         return [SimulationObjective("cost", ObjectiveType.MINIMIZE)]
...
...     def evaluate(self, params):
...         sim = MySimulation(capacity=params["capacity"])
...         sim.run(until=100)
...         return {"cost": sim.total_cost}

abstractmethod get_parameters()[source]

Get list of optimization parameters.

Returns:: Optimization parameters
Return type:: List[Parameter]

abstractmethod get_objectives()[source]

Get list of optimization objectives.

Returns:: Optimization objectives
Return type:: List[SimulationObjective]

abstractmethod evaluate(parameters, replications=1)[source]

Evaluate objective(s) for given parameters.

Parameters:

parameters (Dict[str, float]) – Parameter values
replications (int) – Number of replications

Returns:

Objective values

Return type:

Dict[str, float]

get_constraints()[source]

Get constraint functions.

Returns:: Constraint functions returning True if satisfied
Return type:: List[Callable]

evaluate_full(parameters, replications=1)[source]

Full evaluation with constraints.

Parameters:

parameters (Dict[str, float]) – Parameter values
replications (int) – Number of replications

Returns:

Complete evaluation result

Return type:

EvaluationResult

get_parameter_bounds()[source]

Get parameter bounds as lists.

Returns:: (lower_bounds, upper_bounds)
Return type:: Tuple[List[float], List[float]]

get_initial_point()[source]

Get initial parameter values.

Returns:: Initial parameter values
Return type:: Dict[str, float]

class simcraft.optimization.base.SimulationExperiment[source]

Bases: object

Manages simulation experiments for optimization.

Handles parameter sampling, replication, and result collection.

Parameters:: interface (OptimizationInterface) – Optimization interface

__init__(interface)[source]

Initialize experiment.

Parameters:: interface (OptimizationInterface)
Return type:: None

property results: List[EvaluationResult]: Get all evaluation results.

property best_result: EvaluationResult | None: Get best result found.

run_evaluation(parameters, replications=1)[source]

Run a single evaluation.

Parameters:

parameters (Dict[str, float]) – Parameter values
replications (int) – Number of replications

Returns:

Evaluation result

Return type:

EvaluationResult

run_grid_search(levels, replications=1)[source]

Run full factorial grid search.

Parameters:

levels (Dict[str, List[float]]) – Parameter levels to test
replications (int) – Replications per point

Returns:

All results

Return type:

List[EvaluationResult]

run_random_search(n_evaluations, replications=1, seed=None)[source]

Run random search.

Parameters:

n_evaluations (int) – Number of evaluations
replications (int) – Replications per point
seed (Optional[int]) – Random seed

Returns:

All results

Return type:

List[EvaluationResult]

export_results(filename)[source]

Export results to JSON file.

Parameters:: filename (str) – Output filename
Return type:: None

clear()[source]

Clear all results.

Return type:: None

Reinforcement Learning Interface

Gym-compatible environment wrapper for RL integration.

class simcraft.optimization.rl_interface.ActionSpace[source]

Bases: object

Definition of action space.

type

‘discrete’, ‘continuous’, or ‘multi_discrete’

Type:: str

n

Number of discrete actions

Type:: Optional[int]

shape

Shape for continuous actions

Type:: Optional[Tuple[int, …]]

low

Lower bounds for continuous

Type:: Optional[np.ndarray]

high

Upper bounds for continuous

Type:: Optional[np.ndarray]

nvec

Action counts for multi-discrete

Type:: Optional[List[int]]

type: str

n: int | None = None

shape: Tuple[int, ...] | None = None

low: ndarray | None = None

high: ndarray | None = None

nvec: List[int] | None = None

classmethod discrete(n)[source]

Create discrete action space.

Parameters:: n (int)
Return type:: ActionSpace

classmethod continuous(shape, low=-1.0, high=1.0)[source]

Create continuous action space.

Parameters:

shape (Tuple[int, ...])
low (float | ndarray)
high (float | ndarray)

Return type:

ActionSpace

classmethod multi_discrete(nvec)[source]

Create multi-discrete action space.

Parameters:: nvec (List[int])
Return type:: ActionSpace

__init__(type, n=None, shape=None, low=None, high=None, nvec=None)

Parameters:

type (str)
n (int | None)
shape (Tuple[int, ...] | None)
low (ndarray | None)
high (ndarray | None)
nvec (List[int] | None)

Return type:

None

class simcraft.optimization.rl_interface.StateSpace[source]

Bases: object

Definition of state space.

shape

State shape

Type:: Tuple[int, …]

low

Lower bounds (for bounded spaces)

Type:: Optional[np.ndarray]

high

Upper bounds (for bounded spaces)

Type:: Optional[np.ndarray]

dtype

Data type

Type:: type

shape: Tuple[int, ...]

low: ndarray | None = None

high: ndarray | None = None

dtype: alias of float32

classmethod box(shape, low=-inf, high=inf)[source]

Create box (continuous) state space.

Parameters:

shape (Tuple[int, ...])
low (float | ndarray)
high (float | ndarray)

Return type:

StateSpace

__init__(shape, low=None, high=None, dtype=<class 'numpy.float32'>)

Parameters:

shape (Tuple[int, ...])
low (ndarray | None)
high (ndarray | None)
dtype (type)

Return type:

None

class simcraft.optimization.rl_interface.Transition[source]

Bases: object

A single RL transition.

state

State before action

Type:: State

action

Action taken

Type:: Action

reward

Reward received

Type:: Reward

next_state

State after action

Type:: State

done

Whether episode ended

Type:: bool

info

Additional information

Type:: Dict

state: ndarray | Dict[str, Any]

action: int | ndarray | Dict[str, Any]

reward: float

next_state: ndarray | Dict[str, Any]

done: bool = False

info: Dict[str, Any]

__init__(state, action, reward, next_state, done=False, info=<factory>)

Parameters:

state (ndarray | Dict[str, Any])
action (int | ndarray | Dict[str, Any])
reward (float)
next_state (ndarray | Dict[str, Any])
done (bool)
info (Dict[str, Any])

Return type:

None

class simcraft.optimization.rl_interface.RLInterface[source]

Bases: ABC

Abstract interface for RL-simulation integration.

Subclass this to make a simulation model compatible with reinforcement learning agents.

Examples

>>> class PortRLInterface(RLInterface):
...     def __init__(self, sim):
...         self.sim = sim
...
...     def get_state(self):
...         return np.array([
...             self.sim.queue_length,
...             self.sim.utilization,
...         ])
...
...     def get_action_space(self):
...         return ActionSpace.discrete(4)  # 4 berths
...
...     def apply_action(self, action):
...         self.sim.allocate_berth(action)
...
...     def get_reward(self):
...         return -self.sim.waiting_time

abstractmethod get_state()[source]

Get current state observation.

Returns:: Current state
Return type:: State

abstractmethod get_action_space()[source]

Get action space definition.

Returns:: Action space
Return type:: ActionSpace

abstractmethod get_state_space()[source]

Get state space definition.

Returns:: State space
Return type:: StateSpace

abstractmethod apply_action(action)[source]

Apply an action to the simulation.

Parameters:: action (Action) – Action to apply
Return type:: None

abstractmethod get_reward()[source]

Get reward for current state/action.

Returns:: Reward value
Return type:: Reward

is_done()[source]

Check if episode is done.

Returns:: True if episode ended
Return type:: bool

get_info()[source]

Get additional information.

Returns:: Info dictionary
Return type:: Dict[str, Any]

reset()[source]

Reset environment and return initial state.

Returns:: Initial state
Return type:: State

class simcraft.optimization.rl_interface.RLEnvironment[source]

Bases: object

Gym-compatible wrapper for simulation-based RL.

Wraps an RLInterface to provide a standard RL environment API.

Parameters:

interface (RLInterface) – RL interface implementation
simulation (Simulation) – Simulation instance
max_steps (int) – Maximum steps per episode

Examples

>>> env = RLEnvironment(interface, sim, max_steps=1000)
>>> state = env.reset()
>>> for _ in range(100):
...     action = agent.select_action(state)
...     state, reward, done, info = env.step(action)
...     if done:
...         break

__init__(interface, simulation, max_steps=10000)[source]

Initialize environment.

Parameters:

interface (RLInterface)
simulation (Simulation)
max_steps (int)

Return type:

None

property action_space: ActionSpace: Get action space.

property observation_space: StateSpace: Get observation (state) space.

property current_step: int: Get current step in episode.

property episode: int: Get current episode number.

reset()[source]

Reset environment for new episode.

Returns:: Initial state
Return type:: State

step(action)[source]

Take a step in the environment.

Parameters:: action (Action) – Action to take
Returns:: (next_state, reward, done, info)
Return type:: Tuple[State, Reward, bool, Dict]

get_history()[source]

Get transition history.

Return type:: List[Transition]

clear_history()[source]

Clear transition history.

Return type:: None

class simcraft.optimization.rl_interface.DecisionPoint[source]

Bases: object

Represents a decision point in the simulation.

Used for event-driven RL where actions are taken at specific simulation events.

Parameters:

name (str) – Decision point name
state_fn (Callable) – Function returning current state
action_space (ActionSpace) – Available actions
apply_fn (Callable) – Function to apply action
reward_fn (Callable) – Function returning reward

__init__(name, state_fn, action_space, apply_fn, reward_fn)[source]

Initialize decision point.

Parameters:

name (str)
state_fn (Callable[[], ndarray | Dict[str, Any]])
action_space (ActionSpace)
apply_fn (Callable[[int | ndarray | Dict[str, Any]], None])
reward_fn (Callable[[], float])

Return type:

None

property action_space: ActionSpace: Get action space.

get_state()[source]

Get current state.

Return type:: ndarray | Dict[str, Any]

apply_action(action)[source]

Apply action.

Parameters:: action (int | ndarray | Dict[str, Any])
Return type:: None

get_reward()[source]

Get reward.

Return type:: float

class simcraft.optimization.rl_interface.ReplayBuffer[source]

Bases: object

Experience replay buffer for RL training.

Stores transitions and supports random sampling for off-policy algorithms.

Parameters:: capacity (int) – Maximum buffer size

__init__(capacity=10000)[source]

Initialize buffer.

Parameters:: capacity (int)
Return type:: None

property size: int: Get current buffer size.

push(transition)[source]

Add transition to buffer.

Parameters:: transition (Transition) – Transition to add
Return type:: None

sample(batch_size)[source]

Sample random batch of transitions.

Parameters:: batch_size (int) – Number of transitions to sample
Returns:: Sampled transitions
Return type:: List[Transition]

sample_batch(batch_size)[source]

Sample batch as numpy arrays.

Parameters:: batch_size (int) – Batch size
Returns:: (states, actions, rewards, next_states, dones)
Return type:: Tuple

clear()[source]

Clear buffer.

Return type:: None

class simcraft.optimization.rl_interface.MultiAgentInterface[source]

Bases: object

Interface for multi-agent reinforcement learning.

Supports multiple agents with potentially different action spaces and rewards.

Parameters:: n_agents (int) – Number of agents

Examples

>>> interface = MultiAgentInterface(n_agents=3)
>>> interface.add_agent("berth_allocator", berth_space, berth_reward)
>>> interface.add_agent("agv_dispatcher", agv_space, agv_reward)

__init__(n_agents=1)[source]

Initialize multi-agent interface.

Parameters:: n_agents (int)
Return type:: None

property n_agents: int: Get number of agents.

property agent_names: List[str]: Get agent names.

add_agent(name, action_space, reward_fn, state_fn=None, apply_fn=None)[source]

Add an agent.

Parameters:

name (str) – Agent name
action_space (ActionSpace) – Agent’s action space
reward_fn (Callable) – Agent’s reward function
state_fn (Optional[Callable]) – Agent’s state function (uses shared if None)
apply_fn (Optional[Callable]) – Action application function

Return type:

None

set_shared_state(state_fn)[source]

Set shared state function for all agents.

Parameters:: state_fn (Callable[[], ndarray | Dict[str, Any]])
Return type:: None

get_agent(name)[source]

Get agent by name.

Parameters:: name (str)
Return type:: DecisionPoint | None

get_states()[source]

Get states for all agents.

Return type:: Dict[str, ndarray | Dict[str, Any]]

apply_actions(actions)[source]

Apply actions for all agents.

Parameters:: actions (Dict[str, int | ndarray | Dict[str, Any]])
Return type:: None

get_rewards()[source]

Get rewards for all agents.

Return type:: Dict[str, float]