d3m.runtime module

class d3m.runtime.MultiResult(*args, **kwds)[source]

Bases: List[d3m.runtime.Result]

Results of running a pipeline multiple times.

check_success()[source]

Throws an exception if pipeline has not successfully finished in any of the runs.

Return type

None

has_error()[source]

Returns True if any of pipelines has not successfully finished.

Return type

bool

property pipeline_runs[source]
Return type

Sequence[PipelineRun]

class d3m.runtime.Result(pipeline_run, values, error=None)[source]

Bases: object

Results from running a pipeline.

Parameters

  • pipeline_run (PipelineRun) – A pipeline run description.

  • values (Dict[str, Any]) – A map between data references and their values computed during pipeline run.

  • error (Optional[Exception]) – If during a run an exception occurred, then it is available here.

check_success()[source]

Throws an exception if pipeline has not successfully finished.

Return type

None

has_error()[source]

Returns True if pipeline has not successfully finished.

Return type

bool

class d3m.runtime.Runtime(pipeline, hyperparams=None, *, problem_description=None, context, random_seed=0, volumes_dir=None, scratch_dir=None, is_standard_pipeline=False, environment=None, users=None)[source]

Bases: object

Reference runtime to fit and produce a pipeline.

Parameters

  • pipeline (d3m.metadata.pipeline.Pipeline) – A pipeline to run.

  • hyperparams (Sequence) – Values for free hyper-parameters of the pipeline. It should be a list, where each element corresponds to free hyper-parameters of the corresponding pipeline step. Not all free hyper-parameters have to be specified. Default values are used for those which are not. Optional.

  • problem_description (d3m.metadata.problem.Problem) – A parsed problem description in standard problem description schema.

  • context (d3m.metadata.base.Context) – In which context to run pipelines, default is TESTING.

  • random_seed (int) – A random seed to use for every run. This control all randomness during the run.

  • volumes_dir (str) – Path to a directory with static files required by primitives. In the standard directory structure (as obtained running python3 -m d3m index download).

  • scratch_dir (str) – Path to a directory to store any temporary files needed during execution.

  • is_standard_pipeline (bool) – Is the pipeline a standard pipeline?

  • environment (d3m.metadata.pipeline_run.RuntimeEnvironment) – A description of the runtime environment, including engine versions, Docker images, compute resources, and benchmarks. If not provided, an attempt is made to determine it automatically.

  • users (Sequence[d3m.metadata.pipeline_run.User]) – Users associated with running the pipeline.

pipeline[source]

A pipeline to run.

Type

d3m.metadata.pipeline.Pipeline

Type

Pipeline

hyperparams[source]

Values for free hyper-parameters of the pipeline. It should be a list, where each element corresponds to free hyper-parameters of the corresponding pipeline step. Not all free hyper-parameters have to be specified. Default values are used for those which are not. Optional.

Type

Sequence

Type

Sequence

problem_description[source]

A parsed problem description in standard problem description schema.

Type

d3m.metadata.problem.Problem

Type

Problem

context[source]

In which context to run pipelines, default is TESTING.

Type

d3m.metadata.base.Context

Type

Context

random_seed[source]

A random seed to use for every run. This control all randomness during the run.

Type

int

Type

int

volumes_dir[source]

Path to a directory with static files required by primitives. In the standard directory structure (as obtained running python3 -m d3m index download).

Type

str

Type

str

scratch_dir[source]

Path to a directory to store any temporary files needed during execution.

Type

str

Type

str

is_standard_pipeline[source]

Is the pipeline a standard pipeline?

Type

bool

Type

bool

environment[source]

A description of the runtime environment, including engine versions, Docker images, compute resources, and benchmarks. If not provided, an attempt is made to determine it automatically.

Type

d3m.metadata.pipeline_run.RuntimeEnvironment

Type

RuntimeEnvironment

users[source]

Users associated with running the pipeline.

Type

Sequence[d3m.metadata.pipeline_run.User]

Type

Sequence[User]

current_step[source]

Which step is currently being ran.

Type

int

Type

int

phase[source]

Which phase are we currently running.

Type

d3m.metadata.base.PipelineRunPhase

Type

PipelineRunPhase

pipeline_run[source]

A current instance of pipeline run.

Type

d3m.metadata.pipeline_run.PipelineRun

Type

PipelineRun

return_values[source]

Which values should the runtime keep during a pipeline run, even after they are necessary.

Type

Sequence[str]

Type

Sequence[str]

data_values[source]

Map between available data references and their values during the run.

Type

Dict[str, Any]

Type

Dict[str, Any]

steps_state[source]

Fitted state for each step of the pipeline.

Type

List[Union[Any, List]]

Type

List[Union[Any, List]]

context: d3m.metadata.base.Context[source]
current_step: int[source]
data_values: Dict[str, Any][source]
environment: d3m.metadata.pipeline_run.RuntimeEnvironment[source]
fit(inputs, *, return_values=None)[source]

Does a “fit” phase of the pipeline.

Parameters

  • inputs (Sequence[Any]) – A list of inputs to the pipeline.

  • return_values (Optional[Sequence[str]]) – A list of data references of all output values of all steps to return. If None, the output values of the whole pipeline are returned.

Returns

A result object with kept values, pipeline run description, and any exception.

Return type

Result

get_params()[source]
Return type

List[Union[Any, List]]

hyperparams: Sequence[source]
is_standard_pipeline: bool[source]
phase: d3m.metadata.base.PipelineRunPhase[source]
pipeline: d3m.metadata.pipeline.Pipeline[source]
pipeline_run: d3m.metadata.pipeline_run.PipelineRun[source]
problem_description: d3m.metadata.problem.Problem[source]
produce(inputs, *, return_values=None)[source]

Does a “produce” phase of the pipeline and returns outputs.

Parameters

  • inputs (Sequence[Any]) – A list of inputs to the pipeline.

  • return_values (Optional[Sequence[str]]) – A list of data references of all output values of all steps to return. If None, the output values of the whole pipeline are returned.

Returns

A result object with kept values, pipeline run description, and any exception.

Return type

Result

random_seed: int[source]
return_values: Sequence[str][source]
scratch_dir: str[source]
set_params(params)[source]
Return type

None

steps_state: List[Union[Any, List]][source]
users: Sequence[d3m.metadata.pipeline_run.User][source]
volumes_dir: str[source]
d3m.runtime.combine_folds(scores_list)[source]
Return type

DataFrame

d3m.runtime.combine_pipeline_runs(standard_pipeline_run, *, data_pipeline_run=None, scoring_pipeline_run=None, score_inputs=None, metrics=None, scores=None, fold_group_uuid=None, fold_index=None)[source]
Return type

None

d3m.runtime.combine_random_seed(scores, random_seed)[source]
Return type

DataFrame

d3m.runtime.evaluate(pipeline, inputs, *, data_pipeline, scoring_pipeline, problem_description, data_params, metrics, context, scoring_params=None, hyperparams=None, random_seed=0, data_random_seed=0, scoring_random_seed=0, volumes_dir=None, scratch_dir=None, runtime_environment=None)[source]

Values in data_params should be serialized as JSON, as obtained by JSON-serializing the output of hyper-parameter’s value_to_json_structure method call.

Return type

Tuple[List[DataFrame], MultiResult]

d3m.runtime.evaluate_handler(arguments, *, pipeline_resolver=None, pipeline_run_parser=None, dataset_resolver=None, problem_resolver=None)[source]
Return type

None

d3m.runtime.export_dataframe(dataframe, output_file=None)[source]
Return type

Optional[str]

d3m.runtime.fit(pipeline, inputs, *, problem_description, context, hyperparams=None, random_seed=0, volumes_dir=None, scratch_dir=None, runtime_environment=None, is_standard_pipeline=True, expose_produced_outputs=False)[source]
Return type

Tuple[Optional[Runtime], Optional[DataFrame], Result]

d3m.runtime.fit_handler(arguments, *, pipeline_resolver=None, pipeline_run_parser=None, dataset_resolver=None, problem_resolver=None)[source]
Return type

None

d3m.runtime.fit_produce_handler(arguments, *, pipeline_resolver=None, pipeline_run_parser=None, dataset_resolver=None, problem_resolver=None)[source]
Return type

None

d3m.runtime.fit_score_handler(arguments, *, pipeline_resolver=None, pipeline_run_parser=None, dataset_resolver=None, problem_resolver=None)[source]
Return type

None

d3m.runtime.get_metrics_from_list(metrics)[source]
Return type

Sequence[Dict]

d3m.runtime.get_metrics_from_problem_description(problem_description)[source]
Return type

Sequence[Dict]

d3m.runtime.get_singleton_value(value)[source]

A helper to extract a value from a singleton value (extracting a sole element of a container of length 1).

Return type

Any

d3m.runtime.main(argv)[source]
Return type

None

d3m.runtime.parse_pipeline_run(pipeline_run_file, pipeline_search_paths, datasets_dir, *, pipeline_resolver=None, dataset_resolver=None, problem_resolver=None, strict_resolving=False, compute_digest=<ComputeDigest.ONLY_IF_MISSING: 'ONLY_IF_MISSING'>, strict_digest=False, handle_score_split=True)[source]
Return type

Sequence[Dict[str, Any]]

d3m.runtime.prepare_data(inputs, *, data_pipeline, problem_description, data_params, context, random_seed=0, volumes_dir=None, scratch_dir=None, runtime_environment=None)[source]

Values in data_params should be serialized as JSON, as obtained by JSON-serializing the output of hyper-parameter’s value_to_json_structure method call.

Return type

Tuple[List, Result]

d3m.runtime.produce(fitted_pipeline, test_inputs, *, expose_produced_outputs=False)[source]
Return type

Tuple[Optional[DataFrame], Result]

d3m.runtime.produce_handler(arguments, *, pipeline_resolver=None, pipeline_run_parser=None, dataset_resolver=None, problem_resolver=None)[source]
Return type

None

d3m.runtime.save_steps_outputs(results, output_dir)[source]
Return type

None

d3m.runtime.score(predictions, score_inputs, *, scoring_pipeline, problem_description, metrics, predictions_random_seed=None, context, scoring_params=None, random_seed=0, volumes_dir=None, scratch_dir=None, runtime_environment=None)[source]
Return type

Tuple[Optional[DataFrame], Result]

d3m.runtime.score_handler(arguments, *, pipeline_resolver=None, pipeline_run_parser=None, dataset_resolver=None, problem_resolver=None)[source]
Return type

None

d3m.runtime.score_predictions_handler(arguments, *, pipeline_resolver=None, pipeline_run_parser=None, dataset_resolver=None, problem_resolver=None)[source]
Return type

None