zensols.deeplearn.model package

Submodules

zensols.deeplearn.model.analyze module

Analysis tools to compare results.

class zensols.deeplearn.model.analyze.DataComparison(key, previous, current, compare_df)

Bases: Dictable

Contains the results from two runs used to compare. The data in this object is used to compare the validation loss from a previous run to a run that’s currently in progress. This is provided along with the performance metrics of the runs when written with write().

__init__(key, previous, current, compare_df)

compare_df: DataFrame

A dataframe with the validation loss from the previous and current results and that difference.

current: ModelResult

The current results, which is probably a model currently running.

key: str

The results key used with a ModelResultManager.

previous: ModelResult

The previous resuls of the model from a previous run.

write(depth=0, writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Write this instance as either a Writable or as a Dictable. If class attribute _DICTABLE_WRITABLE_DESCENDANTS is set as True, then use the write() method on children instead of writing the generated dictionary. Otherwise, write this instance by first creating a dict recursively using asdict(), then formatting the output.

If the attribute _DICTABLE_WRITE_EXCLUDES is set, those attributes are removed from what is written in the write() method.

Note that this attribute will need to be set in all descendants in the instance hierarchy since writing the object instance graph is done recursively.

Parameters:

• depth (int) – the starting indentation depth

• writer (TextIOBase) – the writer to dump the content of this writable

class zensols.deeplearn.model.analyze.ResultAnalyzer(executor, previous_results_key, cache_previous_results)

Bases: object

Load results from a previous run of the ModelExecutor and a more recent run. This run is usually a currently running model to compare the results during training. This might provide meaningful information such as whether to early stop training.

__init__(executor, previous_results_key, cache_previous_results)

cache_previous_results: bool

If True, globally cache the previous results to avoid having to reload each time.

clear()

Clear the previous results, if cached.

property comparison: DataComparison

Load the results data and create a comparison instance read to write or jsonify.

property current_results: Tuple[ModelResult, ModelResult]

Return the current results (see class docs).

executor: ModelExecutor

The executor (not the running executor necessary) that will load the results if not already loadded.

property previous_results: ModelResult

Return the previous results (see class docs).

previous_results_key: str

The key given to retreive the previous results with ModelResultManager.

zensols.deeplearn.model.batchiter module

Contains a class to assist in the batch loop during training, validation and testing.

class zensols.deeplearn.model.batchiter.BatchIterator(executor, logger)

Bases: object

This class assists in the batch loop during training, validation and testing. Any special handling of a model related to its loss function can be overridden in this class.

_decode_outcomes(outcomes)

Transform the model output (and optionally the labels) that will be added to the EpochResult, which composes a ModelResult.

This implementation returns :py:meth:~`Tensor.argmax`, which are the indexes of the max value across columns.

Return type:

Tensor

__init__(executor, logger)

executor: InitVar

The owning executor.

iterate(model, optimizer, criterion, batch, epoch_result, split_type)

Train, validate or test on a batch. This uses the back propogation algorithm on training and does a simple feed forward on validation and testing.

One call of this method represents a single batch iteration

Parameters:

• model (BaseNetworkModule) – the model to excercise

• optimizer (Optimizer) – the optimization algorithm (i.e. adam) to iterate

• criterion – the loss function (i.e. cross entropy loss) used for the backward propogation step

• batch (Batch) – contains the data to test, predict, and optionally the labels for training and validation

• epoch_result (EpochResult) – to be populated with the results of this epoch’s run

• split_type (DatasetSplitType) – indicates if we’re training, validating or testing

Return type:

Tensor

Returns:

the singleton tensor containing the loss

logger: Logger

The status logger from the executor.

zensols.deeplearn.model.executor module

This file contains the network model and data that holds the results.

class zensols.deeplearn.model.executor.ModelExecutor(config_factory, config, name, model_settings, net_settings, dataset_stash, dataset_split_names, result_path=None, update_path=None, intermediate_results_path=None, progress_bar=False, progress_bar_cols=None)

Bases: PersistableContainer, Deallocatable, Writable

This class creates and uses a network to train, validate and test the model. This class is either configured using a ConfigFactory or is unpickled with ModelManager. If the later, it’s from a previously trained (and possibly tested) state.

Typically, after creating a nascent instance, train() is called to train the model. This returns the results, but the results are also available via the ResultManager using the model_manager property. To load previous results, use executor.result_manager.load().

During training, the training set is used to train the weights of the model provided by the executor in the model_settings, then validated using the validation set. When the validation loss is minimized, the following is saved to disk:

• Settings: net_settings, model_settings,

• the model weights,

• the results of the training and validation thus far,

• the entire configuration (which is later used to restore the executor),

• random seed information, which includes Python, Torch and GPU random state.

After the model is trained, you can immediately test the model with test(). To be more certain of being able to reproduce the same results, it is recommended to load the model with model_manager.load_executor(), which loads the last instance of the model that produced a minimum validation loss.

See:

ModelExecutor

See:

NetworkSettings

See:

zensols.deeplearn.model.ModelSettings

ATTR_EXP_META = ('model_settings',)

__init__(config_factory, config, name, model_settings, net_settings, dataset_stash, dataset_split_names, result_path=None, update_path=None, intermediate_results_path=None, progress_bar=False, progress_bar_cols=None)

property batch_iterator: BatchIterator

The train manager that assists with the training process.

property batch_stash: DatasetSplitStash

The stash used to obtain the data for training and testing. This stash should have a training, validation and test splits. The names of these splits are given in the dataset_split_names.

config: Configurable

The configuration used in the configuration factory to create this instance.

config_factory: ConfigFactory

The configuration factory that created this instance.

property criterion_optimizer_scheduler: Tuple[L1Loss, Optimizer, Any]

Return the loss function and descent optimizer.

dataset_split_names: List[str]

train, validation, test (see _get_dataset_splits())

Type:

The list of split names in the dataset_stash in the order

dataset_stash: DatasetSplitStash

The split data set stash that contains the BatchStash, which contains the batches on which to train and test.

deallocate()

Deallocate all resources for this instance.

deallocate_batches()

property debug: bool | int

property feature_stash: Stash

The stash used to generate the feature, which is not to be confused with the batch source stash``batch_stash``.

get_model_parameter(name)

Return a parameter of the model, found in model_settings.

get_network_parameter(name)

Return a parameter of the network, found in network_settings.

intermediate_results_path: Path = None

If this is set, then save the model and results to this path after validation for each training epoch.

load()

Clear all results and trained state and reload the last trained model from the file system.

Return type:

Module

Returns:

the model that was loaded and registered in this instance of the executor

property model: BaseNetworkModule

Get the PyTorch module that is used for training and test.

property model_exists: bool

Return whether the executor has a model.

Returns:

True if the model has been trained or loaded

property model_manager: ModelManager

Return the manager used for controlling the train of the model.

property model_result_report: str | None

A human readable summary of the model’s performance.

Returns:

the report if the model has been trained

model_settings: ModelSettings

The configuration of the model.

name: str

The name given in the configuration.

net_settings: NetworkSettings

The settings used to configure the network.

predict(batches)

Create predictions on ad-hoc data.

Parameters:

batches (List[Batch]) – contains the data (X) on which to predict

Return type:

ModelResult

Returns:

the results of the predictions

progress_bar: bool = False

Create text/ASCII based progress bar if True.

progress_bar_cols: int = None

The number of console columns to use for the text/ASCII based progress bar.

reset()

Reset the executor to it’s nascent state.

property result_manager: ModelResultManager

Return the manager used for controlling the life cycle of the results generated by this executor.

result_path: Path = None

If not None, a path to a directory where the results are to be dumped; the directory will be created if it doesn’t exist when the results are generated.

set_model_parameter(name, value)

Safely set a parameter of the model, found in model_settings. This makes the corresponding update in the configuration, so that when it is restored (i.e for test) the parameters are consistent with the trained model. The value is converted to a string as the configuration representation stores all data values as strings.

Important: eval syntaxes are not supported, and probably not the kind of values you want to set a parameters with this interface anyway.

Parameters:

• name (str) – the name of the value to set, which is the key in the configuration file

• value (Any) – the value to set on the model and the configuration

set_network_parameter(name, value)

Safely set a parameter of the network, found in network_settings. This makes the corresponding update in the configuration, so that when it is restored (i.e for test) the parameters are consistent with the trained network. The value is converted to a string as the configuration representation stores all data values as strings.

Important: eval syntaxes are not supported, and probably not the kind of values you want to set a parameters with this interface anyway.

Parameters:

• name (str) – the name of the value to set, which is the key in the configuration file

• value (Any) – the value to set on the network and the configuration

test(description=None)

Test the model.

Return type:

ModelResult

property torch_config: TorchConfig

Return the PyTorch configuration used to convert models and data (usually GPU) during training and test.

train(description=None)

Train the model.

Return type:

ModelResult

property train_manager: TrainManager

Return the train manager that assists with the training process.

train_production(description=None)

Train and test the model on the training and test datasets. This is used for a “production” model that is used for some purpose other than evaluation.

Return type:

ModelResult

update_path: Path = None

The path to check for commands/updates to make while training. If this is set, and the file exists, then it is parsed as a JSON file. If the file cannot be parsed, or 0 size etc., then the training is (early) stopped.

If the file can be parsed, and there is a single epoch dict entry, then the current epoch is set to that value.

write(depth=0, writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, include_settings=False, include_model=False)

Write the contents of this instance to writer using indention depth.

Parameters:

• depth (int) – the starting indentation depth

• writer (TextIOBase) – the writer to dump the content of this writable

write_model(depth=0, writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

write_settings(depth=0, writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

zensols.deeplearn.model.facade module

Client entry point to the model.

class zensols.deeplearn.model.facade.ModelFacade(config, config_factory=<property object>, progress_bar=True, progress_bar_cols='term', executor_name='executor', writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, predictions_dataframe_factory_class=<class 'zensols.deeplearn.result.pred.PredictionsDataFrameFactory'>)

Bases: PersistableContainer, Writable

This class provides easy to use client entry points to the model executor, which trains, validates, tests, saves and loads the model.

More common attributes, such as the learning rate and number of epochs, are properties that dispatch to executor. For the others, go directly to the property.

See:

zensols.deeplearn.domain.ModelSettings

__init__(config, config_factory=<property object>, progress_bar=True, progress_bar_cols='term', executor_name='executor', writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, predictions_dataframe_factory_class=<class 'zensols.deeplearn.result.pred.PredictionsDataFrameFactory'>)

property batch_metadata: BatchMetadata

Return the batch metadata used on the executor.

See:

zensols.deepnlp.model.module.EmbeddingNetworkSettings

property batch_stash: BatchStash

The stash used to encode and decode batches by the executor.

property cache_batches: bool

The cache_batches for the entire network.

property class_explorer: FacadeClassExplorer

clear()

Clear out any cached executor.

config: Configurable

The configuraiton used to create the facade, and used to create a new configuration factory to load models.

property config_factory

The configuration factory used to create this facade, or None if no factory was used.

configure_cli_logging(log_level=None)

“Configure command line (or Python REPL) debugging. Each facade can turn on name spaces that make sense as useful information output for long running training/testing iterations.

This calls “meth:_configure_cli_logging to collect the names of loggers at various levels.

static configure_default_cli_logging(log_level=30)

Configure the logging system with the defaults.

configure_jupyter(log_level=30, progress_bar_cols=120)

Configures logging and other configuration related to a Jupyter notebook. This is just like configure_cli_logging(), but adjusts logging for what is conducive for reporting in Jupyter cells.

;param log_level: the default logging level for the logging system

Parameters:

progress_bar_cols (int) – the number of columns to use for the progress bar

property dataset_stash: DatasetSplitStash

The stash used to encode and decode batches split by dataset.

deallocate()

Deallocate all resources for this instance.

debug(debug_value=True)

Debug the model by setting the configuration to debug mode and invoking a single forward pass. Logging must be configured properly to get the output, which is typically just invoking logging.basicConfig().

Parameters:

debug_value (Union[bool, int]) – True turns on executor debugging; if an int, the higher the value, the more the logging

property dropout: float

The dropout for the entire network.

property epochs: int

The number of epochs for training and validation.

property executor: ModelExecutor

A cached instance of the executor tied to the instance of this class.

executor_name: str = 'executor'

The configuration entry name for the executor, which defaults to executor.

property feature_stash: Stash

The stash used to generate the feature, which is not to be confused with the batch source stash batch_stash.

get_described_results(res_id=None)

Create Zensols LaTeX ready results. This includes a summary from the ModelResultReporter and detailed results using res_id.

Parameters:

res_id (str) – the result ID or use the last if not given

Return type:

DataDescriber

static get_encode_sparse_matrices()

Return whether or not sparse matricies are encoded.

See:

set_sparse()

Return type:

bool

get_predictions(*args, **kwargs)

Generate a Pandas dataframe containing all labels and predictions from the test data set. This method is meant to be overridden by application specific facades to customize prediction output.

See:

get_predictions_factory()

Parameters:

• args – arguments passed to get_predictions_factory()

• kwargs – arguments passed to get_predictions_factory()

Return type:

DataFrame

get_predictions_factory(column_names=None, transform=None, batch_limit=9223372036854775807, name=None)

Generate a predictions factory from the test data set.

Parameters:

• column_names (List[str]) – the list of string column names for each data item the list returned from data_point_transform to be added to the results for each label/prediction

• transform (Callable[[DataPoint], tuple]) – a function that returns a tuple, each with an element respective of column_names to be added to the results for each label/prediction; if None (the default), str used (see the Iris Jupyter Notebook example)

• batch_limit (int) – the max number of batche of results to output

• name (str) – the name/ID (name of the file sans extension in the results directory) of the previously archived saved results to fetch or None to get the last result

Return type:

PredictionsDataFrameFactory

get_result_analyzer(key=None, cache_previous_results=False)

Return a results analyzer for comparing in flight training progress.

Return type:

ResultAnalyzer

classmethod get_singleton(*args, **kwargs)

Return the singleton application using args and kwargs as initializer arguments.

Return type:

ModelFacade

property label_attribute_name

Get the label attribute name.

property last_result: ModelResult

The last recorded result during an ModelExecutor.train() or ModelExecutor.test() invocation is used.

property learning_rate: float

The learning rate to set on the optimizer.

classmethod load_from_path(path, *args, **kwargs)

Construct a new facade from the data saved in a persisted model file. This uses the ModelManager.load_from_path() to reconstruct the returned facade, which means some attributes are taken from default if not taken from *args or **kwargs.

Parameters:

• path (Path) – the path of the model file on the file system

• model_config_overwrites – a Configurable used to overrwrite configuration in the model package config

• args – passed through to the initializer of invoking class cls

• kwargs – passed through to the initializer of invoking class cls

Return type:

ModelFacade

Returns:

a new instance of a ModelFacade

See:

ModelManager.load_from_path()

property model_config: Configurable

The configurable packaged with the model if this facade was created with :mth:`load_from_path`.

property model_settings: ModelSettings

Return the executor’s model settings.

property net_settings: NetworkSettings

Return the executor’s network settings.

persist_result()

Save the last recorded result during an Executor.train() or Executor.test() invocation to disk. Optionally also save a plotted graphics file to disk as well when persist_plot_result is set to True.

Note that in Jupyter notebooks, this method has the side effect of plotting the results in the cell when persist_plot_result is True.

Parameters:

persist_plot_result – if True, plot and save the graph as a PNG file to the results directory

plot_result(result=None, save=False, show=False)

Plot results and optionally save and show them. If this is called in a Jupyter notebook, the plot will be rendered in a cell.

Parameters:

• result (ModelResult) – the result to plot, or if None, use last_result()

• save (bool) – if True, save the plot to the results directory with the same naming as the last data results

• show (bool) – if True, invoke matplotlib’s show function to visualize in a non-Jupyter environment

Return type:

ModelResult

Returns:

the result used to graph, which comes from the executor when none is given to the invocation

predict(datas)

Make ad-hoc predictions on batches without labels, and return the results.

Parameters:

datas (Iterable[Any]) – the data predict on, each as a separate element as a data point in a batch

Return type:

Any

predictions_dataframe_factory_class

The factory class used to create predictions, or the string of the configuration section. If the latter, the factory is created using config_factory.

See:

get_predictions_factory()

alias of PredictionsDataFrameFactory

progress_bar: bool = True

Create text/ASCII based progress bar if True.

progress_bar_cols: Union[str, int] = 'term'

The number of console columns to use for the text/ASCII based progress bar. If the value is term, then use the terminal width.

reload()

Clears all state and reloads the configuration.

remove_metadata_mapping_field(attr)

Remove a field by attribute if it exists across all metadata mappings.

This is useful when a very expensive vectorizer slows down tasks, such as prediction, on a single run of a program. For this use case, override predict() to call this method before calling the super predict method.

Parameters:

attr (str) – the name of the field’s attribute to remove

Return type:

bool

Returns:

True if the field was removed, False otherwise

property result_manager: ModelResultManager

Return the executor’s result manager.

static set_encode_sparse_matrices(use_sparse=False)

If called before batches are created, encode all tensors the would be encoded as dense rather than sparse when use_sparse is False. Oherwise, tensors will be encoded as sparse where it makes sense on a per vectorizer basis.

stop_training()

Early stop training if the model is currently training. This invokes the TrainManager.stop(), communicates to the training process to stop on the next check.

Returns:

True if the application is configured to early stop and the signal has not already been given

test(description=None)

Load the model from disk and test it.

Return type:

ModelResult

train(description=None)

Train and test or just debug the model depending on the configuration.

Parameters:

description (str) – a description used in the results, which is useful when making incremental hyperparameter changes to the model

Return type:

ModelResult

train_production(description=None)

Like train() but uses the test dataset in addition to the training dataset to train the model.

Parameters:

description (str) – a description used in the results, which is useful when making incremental hyperparameter changes to the model

Return type:

ModelResult

update_model_config_factory(path=None, config_factory=None)

Update the model’s configuration factory and configuration that was previously used to train the model. The modification is made in the model’s state file.

Important: the caller is responsible for creating the executor model (i.e. either directly with ModelExecutor._get_or_create_model() or indirectly with ModelExector.write_model()).

Parameters:

• path (Path) – the path of the model directory, which defaults to the model data directory (rather than the model results directory)

• config_factory (ConfigFactory) – the configuration factory that will be replaced, which defaults to config_factory

property vectorizer_manager_set: FeatureVectorizerManagerSet

Return the vectorizer manager set used for the facade. This is taken from the executor’s batch stash.

write(depth=0, writer=None, include_executor=True, include_metadata=True, include_settings=True, include_model=True, include_config=False, include_object_graph=False)

Write the contents of this instance to writer using indention depth.

Parameters:

• depth (int) – the starting indentation depth

• writer (TextIOBase) – the writer to dump the content of this writable

write_predictions(lines=10)

Print the predictions made during the test phase of the model execution.

Parameters:

• lines (int) – the number of lines of the predictions data frame to be printed

• writer – the data sink

write_result(depth=0, writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, include_settings=False, include_converged=False, include_config=False)

Load the last set of results from the file system and print them out. The result to print is taken from last_result

Parameters:

• depth (int) – the number of indentation levels

• writer (TextIOBase) – the data sink

• include_settings (bool) – whether or not to include model and network settings in the output

• include_config (bool) – whether or not to include the configuration in the output

writer: TextIOBase = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>

The writer to this in methods like train(), and test() for writing performance metrics results and predictions or None to not output them.

zensols.deeplearn.model.format module

Contains a class to write performance metrics.

class zensols.deeplearn.model.format.LatexPerformanceMetricsDumper(facade, summary_columns=('mF1t', 'mPt', 'mRt', 'MF1t', 'MPt', 'MRt'), by_label_columns=('mF1', 'mP', 'mR', 'MF1', 'MP', 'MR', 'acc', 'count'), name_replace=None, sort_column='mF1', majority_label_res_id=True, precision=3, results_dir=PosixPath('results/perf'), config_dir=PosixPath('../config'))

Bases: PerformanceMetricsDumper

Writes model performance metrics in data formats then used to import to the LaTeX typesetting system used by the Zensols build framework. The class writes a YAML configuration used by mklatextbl.py script in the Zensols Build repo, which generates a LaTeX table. The output is a .sty` style file with the table, which is included with ``usepackage and then added with a command.

See:

Zensols Build

See:

mklatextbl.py

__init__(facade, summary_columns=('mF1t', 'mPt', 'mRt', 'MF1t', 'MPt', 'MRt'), by_label_columns=('mF1', 'mP', 'mR', 'MF1', 'MP', 'MR', 'acc', 'count'), name_replace=None, sort_column='mF1', majority_label_res_id=True, precision=3, results_dir=PosixPath('results/perf'), config_dir=PosixPath('../config'))

config_dir: Path = PosixPath('../config')

The path to the YAML configuration files used by the mklatextbl.py Zensols LaTeX table generator.

dump_by_label()

Dump per label of metrics of the highest performing model to a LaTeX mktable YAML and CSV files.

Return type:

Tuple[Path, Path]

dump_summary()

Dump summary of metrics to a LaTeX mktable YAML and CSV files.

Return type:

Tuple[Path, Path]

Returns:

a tuple of the output CSV and YAML files

results_dir: Path = PosixPath('results/perf')

The path to the output CSV files with performance metrics.

class zensols.deeplearn.model.format.PerformanceMetricsDumper(facade, summary_columns=('mF1t', 'mPt', 'mRt', 'MF1t', 'MPt', 'MRt'), by_label_columns=('mF1', 'mP', 'mR', 'MF1', 'MP', 'MR', 'acc', 'count'), name_replace=None, sort_column='mF1', majority_label_res_id=True, precision=3)

Bases: Writable

Formats performance metrics, which can be used in papers.

See:

LatexPerformanceMetricsDumper

__init__(facade, summary_columns=('mF1t', 'mPt', 'mRt', 'MF1t', 'MPt', 'MRt'), by_label_columns=('mF1', 'mP', 'mR', 'MF1', 'MP', 'MR', 'acc', 'count'), name_replace=None, sort_column='mF1', majority_label_res_id=True, precision=3)

by_label_columns: Tuple[str] = ('mF1', 'mP', 'mR', 'MF1', 'MP', 'MR', 'acc', 'count')

The columns used in the by-label report.

property by_label_dataframe: DataFrame

static capitalize(name)

Return type:

str

facade: ModelFacade

The facade used to fetch previously written results.

static format_thousand(x, apply_k=True, add_comma=True)

Return type:

str

majority_label_res_id: Union[str, bool] = True

Indicates how to create (if any) the majority label performance metrics. If a string, use as the result id (res_id) of previous result set used to compute the majority label statitics to include in the summary. If True use the results from the last tested model. If None the majority label is not added.

name_replace: Tuple[str, str] = None

If provided, a tuple of (regular expression, replacement) string given to re.sub() in the name column of generated tables.

precision: int = 3

The number of signification digits to format results.

sort_column: str = 'mF1'

The column to sort, with the exception of the majority label, which is always first.

summary_columns: Tuple[str] = ('mF1t', 'mPt', 'mRt', 'MF1t', 'MPt', 'MRt')

The columns used in the summary report.

property summary_dataframe: DataFrame

write(depth=0, writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, indent=0)

Write the contents of this instance to writer using indention depth.

Parameters:

• depth (int) – the starting indentation depth

• writer (TextIOBase) – the writer to dump the content of this writable

zensols.deeplearn.model.manager module

A utility class to help with a ModelExecutor life cycle.

class zensols.deeplearn.model.manager.ModelManager(path, config_factory, model_executor_name=None, persist_random_seed_context=True, keep_last_state_dict=False)

Bases: object

This class manages the lifecycle of an instance of a ModelExecutor. This class is mostly used by the executor to control it’s lifecycle. However, a client can also use an instance of this to revive a model that’s been saved to disk with the ModelResultManager

See ModelExecutor:

__init__(path, config_factory, model_executor_name=None, persist_random_seed_context=True, keep_last_state_dict=False)

config_factory: ConfigFactory

The configuration factory to be used to create the ModelExecutor.

keep_last_state_dict: bool = False

Whether to store the PyTorch module state in attribute last_saved_state_dict.

load_executor(config_overwrites=None)

Load the model the last saved model from the disk. This is used load an instance of a ModelExecutor with all previous state completely in tact. It does this by using an instance of zensols.config.factory.Configurable and a zensols.config.factory.ImportConfigFactory to reconstruct the executor and it’s state by recreating all instances.

After the executor has been recreated with the factory, the previous model results and model weights are restored.

Parameters:

load_factory – whether to load the configuration factory from the check point; which you probably don’t want when loading from load_from_path()

Return type:

ModelExecutor

Returns:

an instance of ModelExecutor

See:

zensols.deeplearn.model.ModelExecutor

classmethod load_from_path(path)

Load and return an instance of this class from a previously saved model. This method exists to recreate a ModelManager from a saved file from scratch. The returned model manager can be used to create the executor or ModelFacade using :obj:config_factory.

Parameters:

path (Path) – points to the model file persisted with _save_executor()

Return type:

ModelManager

Returns:

an instance of ModelManager that was used to save the executor pointed by path

model_executor_name: str = None

The configuration entry and name of the ModelExecutor instance.

path: Path

The path of where the model results saved to disk by zensols.deeplearn.results.ModelResultManager.

persist_random_seed_context: bool = True

If True persist the current random seed state, which helps in creating consistent results across train/test/validate.

zensols.deeplearn.model.meta module

Explore the facade in memory object graph

class zensols.deeplearn.model.meta.FacadeClassExplorer(*args, **kwargs)

Bases: ClassExplorer

A class explorer that includes interesting and noteable framework classes to print.

__init__(*args, **kwargs)

zensols.deeplearn.model.module module

Base class PyTorch module and utilities.

class zensols.deeplearn.model.module.BaseNetworkModule(net_settings, sub_logger=None)

Bases: DebugModule, PersistableContainer

A utility base network module that contains ubiquitous, but optional layers, such as dropout and batch layeres, activation, etc.

abstract _forward(x, *args, **kwargs)

The model’s forward implementation. Normal backward semantics are no different.

Parameters:

• x (Union[Batch, Tensor]) – the batch or tensor to train, validate or test on; the type depends on the needs of the model

• args – additional model specific arguments needed by classes that need more context

• kwargs – additional model specific arguments needed by classes that need more context

Return type:

Tensor

__init__(net_settings, sub_logger=None)

Initialize.

Parameters:

• net_settings (NetworkSettings) – contains common layers such as droput and batch normalization

• sub_logger (Logger) – used to log activity in this module so they logged module comes from some parent model

property device: device

Return the device on which the model is configured.

static device_from_module(module)

Return the device on which the model is configured.

Parameters:

module (Module) – the module containing the parameters used to get the device

Return type:

device

forward(x, *args, **kwargs)

Main forward takes a batch for top level modules, or a tensor for framework based layers. Return the transformed tensor.

Return type:

Tensor

class zensols.deeplearn.model.module.DebugModule(sub_logger=None)

Bases: Module

A utility base class that makes logging more understandable.

DEBUG_CLASS: ClassVar[bool] = True

If True, add the logging class to log messages.

DEBUG_DEVICE: ClassVar[bool] = False

If True, add tensor devices to log messages.

DEBUG_TYPE: ClassVar[bool] = False

If True, add tensor shapes to log messages.

MODULE_NAME: ClassVar[str] = None

The module name used in the logging message. This is set in each inherited class.

__init__(sub_logger=None)

Initialize.

Parameters:

sub_logger (Logger) – used to log activity in this module so they logged module comes from some parent model

zensols.deeplearn.model.optimizer module

Base classes for overriding optimizers.

class zensols.deeplearn.model.optimizer.ModelResourceFactory

Bases: ABC

An abstract factory that creates either an optimizer or criteria by the ModelExecutor. This is more of a marker class for other sub classes to be configured and created by an ImportConfigFactory.

zensols.deeplearn.model.pack module

Model packaging and distribution.

class zensols.deeplearn.model.pack.ModelPacker(version, executor, installer=None)

Bases: _PackerBase

Creates distribution model packages by creating a zip file of everything needed to by a client to use the model.

__init__(version, executor, installer=None)

executor: ModelExecutor

The result manager used to obtain the results and model to package.

installer: Optional[Installer] = None

If set, used to create a path to the model file.

pack(res_id, output_dir)

Create a distribution model package on the file system.

Parameters:

res_id (str) – the result ID or use the last if not given (if optional)

Return type:

Path

Returns:

the path to the generated distribution model package

class zensols.deeplearn.model.pack.ModelUnpacker(version, config_factory, installer, model_packer_name='deeplearn_model_packer', facade_source='facade', model_config_overwrites=None)

Bases: _PackerBase, PersistableContainer, Writable

Unpacks a model created by ModelPacker. Intances of this class should be deallocated with Deallocatable() after using it, which in turn deallocates the facade.

__init__(version, config_factory, installer, model_packer_name='deeplearn_model_packer', facade_source='facade', model_config_overwrites=None)

config_factory: ConfigFactory

Used to get the model facade class when facade_source is a string.

property facade: ModelFacade

The cached facade from installed model. This installs the model if isn’t already.

Returns:

a model facade that allows the caller to deallocate

facade_source: Union[str, Type[ModelFacade]] = 'facade'

`.ModelFacade.

Type:

The client facade section name or the

Type:

class

install_model(**kwargs) → Path

Install the model if it isn’t already and return a path to it.

Return type:

Path

property installed_model_path: Path

Return the path to the model to be PyTorch loaded.

property installed_version: str

The version of the model installed on the file system per the model.version file.

installer: Installer

Used to create a path to the model file.

model_config_overwrites: Configurable = None

Configuration that overwrites the packaged model configuration.

model_packer_name: str = 'deeplearn_model_packer'

The section of the ModelPacker used to create the model. This defaults to the section in resource library resources/cli-pack.conf.

write(depth=0, writer=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Write the contents of this instance to writer using indention depth.

Parameters:

• depth (int) – the starting indentation depth

• writer (TextIOBase) – the writer to dump the content of this writable

class zensols.deeplearn.model.pack.SubsetConfig(config_factory, sections, options, option_delim=':')

Bases: DictionaryConfig

A Configurable that takes a subset of the application configuration. This is useful to pass to ModelFacade.load_from_path() to merge application into the packed model’s configuration.

__init__(config_factory, sections, options, option_delim=':')

Initialize the instance.

Parameters:

• config_factory (ConfigFactory) – the application config and factory

• sections (Tuple[str, ...]) – a list of sections to subset

• options (Tuple[str, ...]) – a list of <section>:<option>, each of which is added to the subset

• option_delim (str) – the string used to delimit sections and options in options

zensols.deeplearn.model.pred module

Contains classs for creating predictions from a trained model.

class zensols.deeplearn.model.pred.PredictionMapper(datas, batch_stash)

Bases: PersistableContainer

Used by a top level client to create features used to create instances of DataPoint and map label classes from nominal IDs to their string representations.

_create_data_point(cls, feature)

Create a data point. This base implementation creates it with the passed parameters.

Parameters:

• cls (Type[DataPoint]) – the data point class of which to make an instance

• stash – to be set as the batch stash on the data point and the caller

• feature (Any) – to be set as the third argument and generate from _create_features()

Return type:

DataPoint

abstract _create_features(data)

Create an instance of a feature from data.

Parameters:

data (Any) – data used to create data points

Return type:

Tuple[Any, ...]

Returns:

the data used in the initializer of the respective (in list) DataPoint

__init__(datas, batch_stash)

batch_stash: BatchStash

“The batch stash used to own batches and data points created by this instance.

property batches: List[Batch]

Create a prediction batch that is detached from any stash resources, except this instance that created it. This creates a tuple of features, each of which is used to create a DataPoint.

datas: Tuple[Any, ...]

The input data to create ad-hoc predictions.

abstract map_results(result)

Map ad-hoc prediction results from the ModelExecutor to an instance that makes sense to the client.

Parameters:

result (ResultsContainer) – contains the predictions produced by the model as predictions_raw

Return type:

Any

Returns:

a first class instance suitable for easy client consumption

zensols.deeplearn.model.sequence module

Sequence modules for sequence models.

class zensols.deeplearn.model.sequence.SequenceBatchIterator(executor, logger)

Bases: BatchIterator

Expects outputs as a list of lists of labels of indexes. Examples of use cases include CRFs (e.g. BiLSTM/CRFs).

__init__(executor, logger)

class zensols.deeplearn.model.sequence.SequenceNetworkContext(split_type, criterion)

Bases: object

The forward context for the SequenceNetworkModule. This is used in SequenceNetworkModule._forward() to provide the module additional information needed to score the model and produce the loss.

__init__(split_type, criterion)

criterion: Module

The criterion used to create the loss. This is provided for modules that produce the loss in the forward phase with the :meth:`torch.nn.module.forward` method.

split_type: DatasetSplitType

The split type, which informs the module when decoding to produce outputs or using the forward pass to prod.

See:

SequenceNetworkModule._forward()

class zensols.deeplearn.model.sequence.SequenceNetworkModule(net_settings, sub_logger=None)

Bases: BaseNetworkModule

A module that has a forward training pass and a separate scoring phase. Examples include layers with an ending linear CRF layer, such as a BiLSTM CRF. This module has a decode method that returns a 2D list of integer label indexes of a nominal class.

The context provides additional information needed to train, test and use the module.

See:

zensols.deeplearn.layer.RecurrentCRFNetwork

abstract _forward(batch, context)

The forward pass, which either trains the model and creates the loss and/or decodes the output for testing and evaluation.

Parameters:

• batch (Batch) – the batch to train, validate or test on

• context (SequenceNetworkContext) – contains the additional information needed for scoring and decoding the sequence

Return type:

SequenceNetworkOutput

class zensols.deeplearn.model.sequence.SequenceNetworkOutput(predictions, loss=None, score=None, labels=None, outputs=None)

Bases: Deallocatable

The output from :clas:`.SequenceNetworkModule` modules.

__init__(predictions, loss=None, score=None, labels=None, outputs=None)

Initialize the output of a sequence NN.

Parameters:

• predictions (Union[List[List[int]], Tensor]) – list of list predictions to convert in to a 1-D tensor if given and not already a tensor; if a tensor, the shape must also be 1-D

• loss (Tensor) – the loss tensor

• score (Tensor) – the score given by the CRF’s Verterbi algorithm

• labels (Union[List[List[int]], Tensor]) – list of list gold labels to convert in to a 1-D tensor if given and not already a tensor

• outputs (Tensor) – the logits from the model

deallocate()

Deallocate all resources for this instance.

righsize_labels(preds)

Convert the labels tensor as a 1-D tensor. This removes the padded values by iterating over preds using each sub list’s for copying the gold label tensor to the new tensor.

zensols.deeplearn.model.trainmng module

Contains a class to assist in the training of the ModelExecutor.

class zensols.deeplearn.model.trainmng.TrainManager(status_logger, progress_logger, update_path, max_consecutive_increased_count, progress_bar_number_width=6)

Bases: object

The class is used to assist in the training of the ModelExecutor. It updates validation loss and helps with early stopping decisions. It also watches for a file on the file system to provide instructions on what to do in the next epoch.

__init__(status_logger, progress_logger, update_path, max_consecutive_increased_count, progress_bar_number_width=6)

get_status()

Return the epoch to set in the training loop of the ModelExecutor.

Return type:

TrainStatus

max_consecutive_increased_count: int

See Domain.max_consecutive_increased_count.

progress_bar_number_width: int = 6

The string width of the train/validation loss metrics in the progress bar, which needs to be greater than 4.

progress_logger: Logger

The logger to record progress updates during training. This is used only when the progress bar is turned off (see ModelFacade._configure_cli_logging()).

start(optimizer, scheduler, n_epochs, pbar)

status_logger: Logger

The logger to record status updates during training.

stop()

Stops the execution of training the model. Currently this is done by creating a file the executor monitors.

Return type:

bool

Returns:

True if the application is configured to early stop and the signal has not already been given

update_loss(valid_epoch_result, train_epoch_result, ave_valid_loss)

Update the training and validation loss.

Return type:

Tuple[float, bool]

Returns:

a tuple of the latest minimum validation loss and whether or not the last validation loss has decreased

update_path: Path

See ModelExecutor.update_path.

class zensols.deeplearn.model.trainmng.TrainStatus(action, epoch=None, reason=None)

Bases: object

Indicates what to do in the next epoch of the training cycle.

__init__(action, epoch=None, reason=None)

action: UpdateAction

epoch: int = None

reason: str = None

class zensols.deeplearn.model.trainmng.UpdateAction(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

An action type to invoke on the ModelExecutor during training.

ITERATE_EPOCH = 0

SET_EPOCH = 1

STOP = 2

zensols.deeplearn.model.wgtexecutor module

A class that weighs labels non-uniformly.

class zensols.deeplearn.model.wgtexecutor.WeightedModelExecutor(config_factory, config, name, model_settings, net_settings, dataset_stash, dataset_split_names, result_path=None, update_path=None, intermediate_results_path=None, progress_bar=False, progress_bar_cols=None, weighted_split_name='train', weighted_split_path=None, use_weighted_criterion=True)

Bases: ModelExecutor

A class that weighs labels non-uniformly. This class uses invert class sampling counts to help the minority label.

__init__(config_factory, config, name, model_settings, net_settings, dataset_stash, dataset_split_names, result_path=None, update_path=None, intermediate_results_path=None, progress_bar=False, progress_bar_cols=None, weighted_split_name='train', weighted_split_path=None, use_weighted_criterion=True)

clear()

get_class_weights(**kwargs) → Tensor

Compute invert class sampling counts to return the weighted class.

Return type:

Tensor

get_label_counts(**kwargs) → Dict[int, int]

Return type:

Dict[int, int]

get_label_statistics()

Return a dictionary whose keys are the labels and values are dictionaries containing statistics on that label.

Return type:

Dict[str, Dict[str, Any]]

use_weighted_criterion: bool = True

If True, use the class weights in the initializer of the criterion. Setting this to False effectively disables this class.

weighted_split_name: str = 'train'

The split name used to re-weight labels.

weighted_split_path: InitVar = None

The path to the cached weithed labels.

Module contents

Classes that train, validate and test a model. The values of the predictions are made available using zensols.deeplearn.model.Executor.get_predictions.