inet.models.solvers package

Submodules

inet.models.solvers.common module

evaluate_solver_predictions(predicted_labels, predicted_bbs, validation_values, validation_labels, render_samples, model_name: Optional[str] = None)

Common evaluation method for solvers

Computes: - GIoU-Loss - RMSE - Accuracy - F1-Score

Parameters

• predicted_labels – predicted class labels by the solver

• predicted_bbs – predicted bboxes by the solver

• validation_values – input samples used to perform predictions

• validation_labels – ground truth labels for given samples

• render_samples – if True renders confusion matrix of classification and up to 25 samples of bbox regression

• model_name – used to save resulting plots

Returns

inet.models.solvers.independent module

class IndependentModel(regressor, classifier, input_shape: Tuple[int, ...] = (224, 224, 3), is_tflite=False)

Bases: inet.models.solvers.tf_lite.MultiTaskModel

Object detection model using independent methods to solve the localization and classification tasks. A regressor predicts the location and a classifier the class label, based on the original input.

[Similar to TwoStageModel]

Example:

>>> from tensorflow.keras.applications.mobilenet import MobileNet
>>> from inet.models.architectures.classifier import Classifier
>>> from inet.models.architectures.bounding_boxes import BoundingBoxRegressor
>>> clf_backbone = MobileNet(weights='imagenet', include_top=False, input_shape=(224, 224))
>>> reg_backbone = MobileNet(weights='imagenet', include_top=False, input_shape=(224, 224))
>>> regressor = BoundingBoxRegressor(reg_backbone)
>>> classifier = Classifier(clf_backbone)
>>> solver = IndependentModel(regressor, classifier, (224, 224, 3), False)

model_name: Optional[str] = 'independent-model'

predict(X)

Performs independent predictions on raw input X

Parameters

X – given input features

Returns

vector of prediction tuples [label, bounding box]

inet.models.solvers.tf_lite module

class MultiTaskModel(regressor, classifier, input_shape: Tuple[int, ...] = (224, 224, 3), is_tflite=False)

Bases: object

MultiTask solver implementation

Example:

>>> from tensorflow.keras.applications.mobilenet import MobileNet
>>> from inet.models.architectures.classifier import Classifier
>>> from inet.models.architectures.bounding_boxes import BoundingBoxRegressor
>>> clf_backbone = MobileNet(weights='imagenet', include_top=False, input_shape=(224, 224))
>>> reg_backbone = MobileNet(weights='imagenet', include_top=False, input_shape=(224, 224))
>>> regressor = BoundingBoxRegressor(reg_backbone)
>>> classifier = Classifier(clf_backbone)
>>> solver = MultiTaskModel(regressor, classifier, (224, 224, 3), False)
>>> solver.predict([some_input])

static create_classifier(cfg: Dict) → inet.models.architectures.classifier.Classifier

Helper to bootstrap classifier based on provided config dict.

Parameters

cfg – configuration dictionary

Returns

a classifier model

static create_regressor(cfg: Dict) → inet.models.architectures.bounding_boxes.BoundingBoxRegressor

Helper to bootstrap bbreg model based on provided config dict.

Parameters

cfg – configuration dictionary

Returns

a bounding box regression model

crop_image(elem: Tuple)

Method to crop an image.

Parameters

elem – Tuple of [image, bb]

Returns

cropped image

evaluate_model(validation_set, preprocessing_method: Callable, render_samples: bool = False) → None

Method to evaluate predictive power of solver.

Computes - Regression: * GIoU-Loss * RMSE - Classification: * Accuracy * F1-Score

Parameters

• validation_set – validation data set to use

• preprocessing_method – preprocessing method to apply before predicting

• render_samples – if True renders confusion matrix of classification and samples for bbox regression

Returns

classmethod from_config(cfg: Dict, is_tflite: bool = False) → inet.models.solvers.tf_lite.MultiTaskModel

Method to load MultiTaskModel from dictionary and essentially JSON files.

Parameters

• cfg – dict holding solver configuration

• is_tflite – if True treats config as for a tflite solver

Returns

new created solver

model_name: Optional[str] = None

predict(X)

Interface definition for prediction method.

Parameters

X – Vector of input features to perform predictions on.

Returns

predicted outputs

inet.models.solvers.two_in_one module

class TwoInOneHyperModel(name=None, tunable=True)

Bases: keras_tuner.engine.hypermodel.HyperModel

HPO wrapper for TwoInOne model.

Used Hyper parameters (HPs): - Regularization Factor alpha: [1e-4, 5e-2] - Dropout Factor dropout: [0.1, 0.75] - Learning rate learning_rate: [1e-4, 1e-2]

Example:

>>> import keras_tuner as kt
>>> hpo_model = TwoInOneModel()
>>> tuner = kt.BayesianOptimization(
...    hpo_model,
...    objective=kt.Objective('val_accuracy', 'max'),
...    max_trials=36,
...    directory=f'./model-selection/my-model/',
...    project_name='proj_name',
...    seed=42,
...    overwrite=False,
...    num_initial_points=12
...)
>>> tuner.search(
...     train_set=train_set.unbatch(),
...     validation_set=validation_set.unbatch(),
...     monitoring_val='val_accuracy',
...     epochs=50,
... )

build(hp)

Builds model of next HPO iteration

Parameters

hp – current HP state

Returns

next HPO model

model_data: Optional[inet.models.data_structures.ModelArchitecture] = None

class TwoInOneModel(*args, **kwargs)

Bases: inet.models.architectures.base_model.TaskModel

Two-In-One model implementation, meaning the model solves both tasks simultaneously based on a single pass through the backbone (CNN)

Example:

>>> from tensorflow.keras.applications.mobilenet import MobileNet
>>> backbone = MobileNet(weights='imagenet', include_top=False, input_shape=(224, 224))
>>> solver = TwoInOneModel(backbone, 5)
>>> solver.load_weights('my-weights.h5', by_name=True)
>>> solver.predict([some_input])

compile(learning_rate: float = 0.001, loss_weights: Optional[List[float]] = None, losses: Optional[List[float]] = None, metrics: Optional[List] = None, *args, **kwargs)

Extended keras.Model.compile. Adds default Adam optimizer and Accuracy metric

Parameters

• learning_rate – the learning rate to train with

• loss_weights – Task individual weight, when calculating overall loss

• losses – the loss functions to optimize

• metrics – additional metrics to calculate during training

• args – will be passed as args to parent implementation

• kwargs – will be passed as kwargs to parent implementation

Returns

evaluate_model(validation_set, preprocessing_method: Optional[Callable] = None, render_samples: bool = False) → None

Method to evaluate predictive power of solver.

Computes - Regression: * GIoU-Loss * RMSE - Classification: * Accuracy * F1-Score

Parameters

• validation_set – validation data set to use

• preprocessing_method – preprocessing method to apply before predicting

• render_samples – if True renders confusion matrix of classification and samples for bbox regression

Returns

static evaluate_predictions(predictions, labels, features, render_samples=False) → None

shadowed method, nothing implemented here.

classmethod from_config(cfg) → inet.models.solvers.two_in_one.TwoInOneModel

Helper to bootstrap solver out of configuration dictionary

Parameters

cfg – configuration dictionary

Returns

new TwoInOneModel instance

model_type: inet.data.constants.ModelType = 2

class TwoInOneTFLite(weights)

Bases: object

TFLite implementation of TwoInOneModel

Example:

>>> solver = TwoInOneModel('my-weights.tflite')
>>> solver.predict([some_input])

evaluate_model(validation_set, preprocessing_method, render_samples: bool = False)

Method to evaluate predictive power of solver.

Computes - Regression: * GIoU-Loss * RMSE - Classification: * Accuracy * F1-Score

Parameters

• validation_set – validation data set to use

• preprocessing_method – preprocessing method to apply before predicting

• render_samples – if True renders confusion matrix of classification and samples for bbox regression

Returns

classmethod from_config(cfg) → inet.models.solvers.two_in_one.TwoInOneTFLite

Constructor to load model from config-dict

Parameters

cfg – configuration dictionary

Returns

predict(X)

Performs prediction

Parameters

X – vector of input features

Returns

prediction done by the model

inet.models.solvers.two_stage module

class TwoStageModel(regressor, classifier, input_shape: Tuple[int, ...] = (224, 224, 3), is_tflite=False)

Bases: inet.models.solvers.tf_lite.MultiTaskModel

Object detection model using dependent/sequential methods to solve the localization and classification tasks. A regressor predicts the location, the original input image gets cropped to a patch containing the extracted Bounding Box. Afterwards a classifier predicts the class label, based on the cropped input.

[Similar to IndependentModel]

Example:

>>> from tensorflow.keras.applications.mobilenet import MobileNet
>>> from inet.models.architectures.classifier import Classifier
>>> from inet.models.architectures.bounding_boxes import BoundingBoxRegressor
>>> clf_backbone = MobileNet(weights='imagenet', include_top=False, input_shape=(224, 224))
>>> reg_backbone = MobileNet(weights='imagenet', include_top=False, input_shape=(224, 224))
>>> regressor = BoundingBoxRegressor(reg_backbone)
>>> classifier = Classifier(clf_backbone)
>>> solver = TwoStageModel(regressor, classifier, (224, 224, 3), False)

model_name: Optional[str] = 'two-stage-model'

predict(X)

Performs dependent predictions on input X.

Regressor receives raw X -> returns c X is cropped using c -> X_hat Classifier receives X_hat -> returns y

Parameters

X – vector of input images

Returns

Prediction Tuple [y, c]