Models module

The chemml.models.keras module includes (please click on links adjacent to function names for more information):

class chemml.models.MLP(engine, nfeatures, nneurons=None, activations=None, learning_rate=0.01, nepochs=100, batch_size=100, alpha=0.001, loss='mean_squared_error', is_regression=True, nclasses=None, layer_config_file=None, opt_config='sgd', random_seed=112, **params)

Class associated with Multi-Layer Perceptron (Neural Network)

engine: str

Determines the underlying ML library used to build the deep neural network can be either ‘tensorflow’ or ‘pytorch’

nfeatures: int

number of input features

nneurons: list, default = None

The number of nodes in each hidden layer, required if layer_config_file is not provided

activations: list, default = None

The activation type for each hidden layer (len of activations should be the same as len of nneurons) required if layer_config_file is not provided Refer https://keras.io/activations/ for list of valid activations for tensorflow Refer https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity or https://pytorch.org/docs/stable/nn.html#non-linear-activations-other for list of valid activations for pytorch e.g. [‘ReLU’,’ReLU’]

nepochs: int, optional, default: 100

Number of training epochs.

batch_size: int, optional, default: 100

Number of training samples in mini-batch

alpha: float, default: 0.001 (defaults for pytorch: 0, keras: 0.01, sklearn: 0.0001)

L2 regularization parameter. If engine is pytorch, this will override the weight_decay parameter in the SGD optimizer

loss: str, optional, default: ‘mean_squared_error’

Type of loss used to train the neural network. Refer https://keras.io/losses/ for list of valid losses for tensorflow Refer https://pytorch.org/docs/stable/nn.html#loss-functions for valid losses for pytorch

regression: bool, optional, default: True

Decides whether we are training for regression or classification task

nclasses: int, optional, default: None

Number of classes labels needs to be specified if regression is False

layer_config_file: str, optional, default: None

Path to the file that specifies layer configuration Refer MLP test to see a sample file Note: this variable SHOULD be consolidated with the layers variable to reduce redundancy

opt_config: list or str, optional, default: ‘sgd’

optimizer configuration. If str, should either be ‘sgd’ or ‘adam’ If list, should provide exact configurations and parameters corresponding to the respective engines, for e.g., [“Adam”,{“learning_rate”:0.01}] or [“SGD”,{“lr”:0.01, “momentum”:0.9, “lr_decay”:0.0, nesterov=False)]

get_model(include_output=True, n_layers=None)

Returns the entire tensorflow or pytorch model in its current state (fitted or compiled)

include_output: bool

if True it will return the entire model, if False it will return the model without the output layer

n_layers: int, optional (default=None)

remove the last ‘n’ hidden layers from the model in addition to the output layer. Note that this number should not include the output layer.

self.model: tensorflow.python.keras type object or torch.nn.Module type object

load(path_to_model)

Loads the chemml.models.MLP object along with the underlying tensorflow.python.keras object

path_to_model: str

path to the chemml.models.MLP csv file

save(path, filename)

Saves the chemml.models.MLP object along with the underlying tensorflow.python.keras object

path: str

the path to the directory where the models should be saved

filename: str

the name of the model file without the file type

class chemml.models.NeuralGraphHidden(*args, **kwargs)
Hidden Convolutional layer in a Neural Graph (as in Duvenaud et. al.,

2015). This layer takes a graph as an input. The graph is represented as by three tensors. - The atoms tensor represents the features of the nodes. - The bonds tensor represents the features of the edges. - The edges tensor represents the connectivity (which atoms are connected to

which)

It returns the convolved features tensor, which is very similar to the atoms tensor. Instead of each node being represented by a num_atom_features-sized vector, each node now is represented by a convolved feature vector of size conv_width. # Example

Define the input: ```python

atoms0 = Input(name=’atom_inputs’, shape=(max_atoms, num_atom_features)) bonds = Input(name=’bond_inputs’, shape=(max_atoms, max_degree, num_bond_features)) edges = Input(name=’edge_inputs’, shape=(max_atoms, max_degree), dtype=’int32’)

``` The NeuralGraphHidden can be initialised in three ways: 1. Using an integer conv_width and possible kwags (Dense layer is used)

`python atoms1 = NeuralGraphHidden(conv_width, activation='relu', bias=False)([atoms0, bonds, edges]) `

  1. Using an initialised Dense layer

    `python atoms1 = NeuralGraphHidden(Dense(conv_width, activation='relu', bias=False))([atoms0, bonds, edges]) `

  2. Using a function that returns an initialised Dense layer

    `python atoms1 = NeuralGraphHidden(lambda: Dense(conv_width, activation='relu', bias=False))([atoms0, bonds, edges]) `

Use NeuralGraphOutput to convert atom layer to fingerprint

# Arguments
inner_layer_arg: Either:
  1. an int defining the conv_width, with optional kwargs for the

    inner Dense layer

  2. An initialised but not build (Dense) keras layer (like a wrapper)

  3. A function that returns an initialised keras layer.

kwargs: For initialisation 1. you can pass Dense layer kwargs

# Input shape

List of Atom and edge tensors of shape: `[(samples, max_atoms, atom_features), (samples, max_atoms, max_degrees,

bond_features), (samples, max_atoms, max_degrees)]`

where degrees referes to number of neighbours

# Output shape

New atom featuers of shape (samples, max_atoms, conv_width)

# References
build(inputs_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Args:
input_shape: Instance of TensorShape, or list of instances of

TensorShape if the layer expects a list of inputs (one instance per input).

call(inputs, mask=None)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Args:
inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

  • NumPy array or Python scalar values in inputs get cast as tensors.

  • Keras mask metadata is only collected from inputs.

  • Layers are built (build(input_shape) method) using shape info from inputs only.

  • input_spec compatibility is only checked against inputs.

  • Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

  • The SavedModel input specification is generated using inputs only.

  • Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

  • mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

compute_output_shape(inputs_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Args:
input_shape: Shape tuple (tuple of integers)

or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Note that get_config() does not guarantee to return a fresh copy of dict every time it is called. The callers should make a copy of the returned dict if they want to modify it.

Returns:

Python dictionary.

class chemml.models.NeuralGraphOutput(*args, **kwargs)

Output Convolutional layer in a Neural Graph (as in Duvenaud et. al., 2015). This layer takes a graph as an input. The graph is represented as by three tensors.

  • The atoms tensor represents the features of the nodes.

  • The bonds tensor represents the features of the edges.

  • The edges tensor represents the connectivity (which atoms are connected to

    which)

It returns the fingerprint vector for each sample for the given layer.

According to the original paper, the fingerprint outputs of each hidden layer need to be summed in the end to come up with the final fingerprint.

# Example

Define the input: ```python

atoms0 = Input(name=’atom_inputs’, shape=(max_atoms, num_atom_features)) bonds = Input(name=’bond_inputs’, shape=(max_atoms, max_degree, num_bond_features)) edges = Input(name=’edge_inputs’, shape=(max_atoms, max_degree), dtype=’int32’)

```

The NeuralGraphOutput can be initialised in three ways: 1. Using an integer fp_length and possible kwags (Dense layer is used)

`python fp_out = NeuralGraphOutput(fp_length, activation='relu', bias=False)([atoms0, bonds, edges]) `

  1. Using an initialised Dense layer

    `python fp_out = NeuralGraphOutput(Dense(fp_length, activation='relu', bias=False))([atoms0, bonds, edges]) `

  2. Using a function that returns an initialised Dense layer

    `python fp_out = NeuralGraphOutput(lambda: Dense(fp_length, activation='relu', bias=False))([atoms0, bonds, edges]) `

Predict for regression: `python main_prediction = Dense(1, activation='linear', name='main_prediction')(fp_out) `

# Arguments
inner_layer_arg: Either:
  1. an int defining the fp_length, with optional kwargs for the

    inner Dense layer

  2. An initialised but not build (Dense) keras layer (like a wrapper)

  3. A function that returns an initialised keras layer.

kwargs: For initialisation 1. you can pass Dense layer kwargs

# Input shape

List of Atom and edge tensors of shape: `[(samples, max_atoms, atom_features), (samples, max_atoms, max_degrees,

bond_features), (samples, max_atoms, max_degrees)]`

where degrees referes to number of neighbours

# Output shape

Fingerprints matrix (samples, fp_length)

# References
build(inputs_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Args:
input_shape: Instance of TensorShape, or list of instances of

TensorShape if the layer expects a list of inputs (one instance per input).

call(inputs, mask=None)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Args:
inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

  • NumPy array or Python scalar values in inputs get cast as tensors.

  • Keras mask metadata is only collected from inputs.

  • Layers are built (build(input_shape) method) using shape info from inputs only.

  • input_spec compatibility is only checked against inputs.

  • Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

  • The SavedModel input specification is generated using inputs only.

  • Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

  • mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

compute_output_shape(inputs_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Args:
input_shape: Shape tuple (tuple of integers)

or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Note that get_config() does not guarantee to return a fresh copy of dict every time it is called. The callers should make a copy of the returned dict if they want to modify it.

Returns:

Python dictionary.

class chemml.models.TransferLearning(base_model, nfeatures=None, n_layers=None)

Class used to facilitate transfer learning from a parent (or head) model to a child model. Freezes the layers (weights and biases) of the pre-trained base model and removes the output layers and appends the child model to the base model.

base_model: chemml.models.MLP object, tensorflow.python.keras object, or a torch.nn.Module object

pre-trained base model

n_features: int or None (default=None)

no. of input features provided for training the base model. If base_model is a tensorflow.python.keras object or torch.nn.Modules object, then n_features must be int, else it can be None.

n_layers: int, optional (default=None)

remove the last ‘n’ hidden layers from the base model. Note that this number should not include the output layer.

transfer(X, y, child_model)

Adds the base model’s frozen layers (without its input and output layers) to the child model and fits the new model to the training data.

X, y: numpy.ndarray

X is an array of features and y is an array of the target values X should have the same input features (columns) as the base model.

child_model: chemml.models.mlp.MLP object

chemml model created with all the parameters and options required for the final transfer learned model

child_model: chemml.models.mlp.MLP object

The trained transfer-learned child model.