alr

Main alr module

Classses

FitResult

ALRModel

class alr.ALRModel[source]

Bases: torch.nn.modules.module.Module, abc.ABC

A ALRModel provides generic methods required for common operations in active learning experiments.

forward(x: torch.Tensor) → torch.Tensor[source]

Regular forward pass. Usually reserved for training.

Parameters:x (torch.Tensor) – input tensor
Returns:output tensor
Return type:torch.Tensor
predict(x: torch.Tensor) → torch.Tensor[source]

Sets the model mode to eval and calls forward.

Parameters:x (torch.Tensor) – input tensor
Returns:output tensor
Return type:torch.Tensor
reset_weights() → None[source]

Resets the model’s weights to the last saved snapshot.

Returns:None
Return type:NoneType
snap() → None[source]

Take and store a snapshot of the current state.

Returns:None
Return type:NoneType

MCDropout

class alr.MCDropout(model: torch.nn.modules.module.Module, forward: Optional[int] = 100, reduce: Optional[str] = 'logsumexp', inplace: Optional[bool] = True, output_transform: Optional[Callable[[torch.Tensor], torch.Tensor]] = None, fast: Optional[bool] = False, consistent: Optional[bool] = False)[source]

Bases: alr.ALRModel

A wrapper that turns a regular PyTorch module into one that implements Monte Carlo Dropout (Gal & Ghahramani, 2016).

Parameters:
  • model (nn.Module) – torch.nn.Module object. This model’s forward pass should return (log) probabilities. I.e. the final layer should be softmax or log_softmax. Otherwise, output_transform can be used to convert model’s output into probabilities.
  • forward (int, optional) – number of stochastic forward passes
  • reduce (str, optional) – either “logsumexp” or “mean”. This is used to reduce the n forward stochastic passes during evaluation. If model or output_transform returns probabilities (i.e. F.softmax), this should be “mean”; otherwise it should be “logsumexp” if they return log-probabilities (i.e. F.log_softmax). [default = “logsumexp”]
  • inplace (bool, optional) – if True, the model is modified in-place when the dropout layers are replaced. If False, model is not modified and a new model is cloned.
  • output_transform (callable, optional) – model’s output is given as input and the output of this callable is expected to return (log) probabilities.
  • fast (bool, optional) – if true, stochastic_forward() will stack the batch dimension for faster MC dropout passes. If false, then forward passes are called in a for-loop. Note, the former will consume forward times more memory.
  • consistent (bool, optional) – if true, the dropout layers will be replaced with consistent variants.
base_model

provided base model (a clone if inplace=True)

Type:nn.Module
n_forward

number of forward passes (forward)

Type:int
forward(x: torch.Tensor) → torch.Tensor[source]

Forward pass. Note, this function has a different behaviour in eval mode. It returns the (log) mean score of stochastic_forward() passes. In other words, if self.training is False, the following is returned instead:

# if reduce = "logsumexp"
torch.logsumexp(self.stochastic_forward(x), dim=0) - log(self.n_forward)

# if reduce = "mean"
torch.mean(self.stochastic_forward(x), dim=0)
Parameters:x (torch.Tensor) – input tensor, any size
Returns:output tensor of size \(N \times C\) where \(N\) is the batch size and \(C\) is the number of target classes.
Return type:torch.Tensor

Note

if a single forward pass is required during eval mode, one could use the following instead: base_model(x)

stochastic_forward(x: torch.Tensor) → torch.Tensor[source]

Returns a \(m \times N \times C\) torch.Tensor where:

  1. \(m\) is equal to self.n_forward
  2. \(N\) is the batch size, equal to x.size(0)
  3. \(C\) is the number of units in the final layer (e.g. number of classes in a classification model)
Parameters:x (torch.Tensor) – input tensor
Returns:output tensor of shape \(m \times N \times C\)
Return type:torch.Tensor
Raises:RuntimeError – Occurs when the machine runs out of memory and fast was set to true.

Functions