Models module

The chemml.models.keras module includes (please click on links adjacent to function names for more information):

class chemml.models.MLP(engine, nfeatures, nneurons=None, activations=None, learning_rate=0.01, nepochs=100, batch_size=100, alpha=0.001, loss='mean_squared_error', is_regression=True, nclasses=None, layer_config_file=None, opt_config='sgd', random_seed=112, **params)

Class associated with Multi-Layer Perceptron (Neural Network)

Parameters

engine: str

Determines the underlying ML library used to build the deep neural network can be either ‘tensorflow’ or ‘pytorch’

nfeatures: int

number of input features

nneurons: list, default = None

The number of nodes in each hidden layer, required if layer_config_file is not provided

activations: list, default = None

The activation type for each hidden layer (len of activations should be the same as len of nneurons) required if layer_config_file is not provided Refer https://keras.io/activations/ for list of valid activations for tensorflow Refer https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity or https://pytorch.org/docs/stable/nn.html#non-linear-activations-other for list of valid activations for pytorch e.g. [‘ReLU’,’ReLU’]

nepochs: int, optional, default: 100

Number of training epochs.

batch_size: int, optional, default: 100

Number of training samples in mini-batch

alpha: float, default: 0.001 (defaults for pytorch: 0, keras: 0.01, sklearn: 0.0001)

L2 regularization parameter. If engine is pytorch, this will override the weight_decay parameter in the SGD optimizer

loss: str, optional, default: ‘mean_squared_error’

Type of loss used to train the neural network. Refer https://keras.io/losses/ for list of valid losses for tensorflow Refer https://pytorch.org/docs/stable/nn.html#loss-functions for valid losses for pytorch

regression: bool, optional, default: True

Decides whether we are training for regression or classification task

nclasses: int, optional, default: None

Number of classes labels needs to be specified if regression is False

layer_config_file: str, optional, default: None

Path to the file that specifies layer configuration Refer MLP test to see a sample file Note: this variable SHOULD be consolidated with the layers variable to reduce redundancy

opt_config: list or str, optional, default: ‘sgd’

optimizer configuration. If str, should either be ‘sgd’ or ‘adam’ If list, should provide exact configurations and parameters corresponding to the respective engines, for e.g., [“Adam”,{“learning_rate”:0.01}] or [“SGD”,{“lr”:0.01, “momentum”:0.9, “lr_decay”:0.0, nesterov=False)]

get_model(include_output=True, n_layers=None)

Returns the entire tensorflow or pytorch model in its current state (fitted or compiled)

Parameters

include_output: bool

if True it will return the entire model, if False it will return the model without the output layer

n_layers: int, optional (default=None)

remove the last ‘n’ hidden layers from the model in addition to the output layer. Note that this number should not include the output layer.

Returns

self.model: tensorflow.python.keras type object or torch.nn.Module type object

load(path_to_model)

Loads the chemml.models.MLP object along with the underlying tensorflow.python.keras object

Parameters

path_to_model: str

path to the chemml.models.MLP csv file

save(path, filename)

Saves the chemml.models.MLP object along with the underlying tensorflow.python.keras object

Parameters

path: str

the path to the directory where the models should be saved

filename: str

the name of the model file without the file type

class chemml.models.NeuralGraphHidden(*args, **kwargs)
Hidden Convolutional layer in a Neural Graph (as in Duvenaud et. al.,

2015). This layer takes a graph as an input. The graph is represented as by three tensors. - The atoms tensor represents the features of the nodes. - The bonds tensor represents the features of the edges. - The edges tensor represents the connectivity (which atoms are connected to

which)

It returns the convolved features tensor, which is very similar to the atoms tensor. Instead of each node being represented by a num_atom_features-sized vector, each node now is represented by a convolved feature vector of size conv_width. # Example

Define the input: ```python

atoms0 = Input(name=’atom_inputs’, shape=(max_atoms, num_atom_features)) bonds = Input(name=’bond_inputs’, shape=(max_atoms, max_degree, num_bond_features)) edges = Input(name=’edge_inputs’, shape=(max_atoms, max_degree), dtype=’int32’)

``` The NeuralGraphHidden can be initialised in three ways: 1. Using an integer conv_width and possible kwags (Dense layer is used)

`python atoms1 = NeuralGraphHidden(conv_width, activation='relu', bias=False)([atoms0, bonds, edges]) `

  1. Using an initialised Dense layer

    `python atoms1 = NeuralGraphHidden(Dense(conv_width, activation='relu', bias=False))([atoms0, bonds, edges]) `

  2. Using a function that returns an initialised Dense layer

    `python atoms1 = NeuralGraphHidden(lambda: Dense(conv_width, activation='relu', bias=False))([atoms0, bonds, edges]) `

Use NeuralGraphOutput to convert atom layer to fingerprint

# Arguments
inner_layer_arg: Either:
  1. an int defining the conv_width, with optional kwargs for the

    inner Dense layer

  2. An initialised but not build (Dense) keras layer (like a wrapper)

  3. A function that returns an initialised keras layer.

kwargs: For initialisation 1. you can pass Dense layer kwargs

# Input shape

List of Atom and edge tensors of shape: `[(samples, max_atoms, atom_features), (samples, max_atoms, max_degrees,

bond_features), (samples, max_atoms, max_degrees)]`

where degrees referes to number of neighbours

# Output shape

New atom featuers of shape (samples, max_atoms, conv_width)

# References
classmethod from_config(config)

Creates an operation from its config.

This method is the reverse of get_config, capable of instantiating the same operation from the config dictionary.

Note: If you override this method, you might receive a serialized dtype config, which is a dict. You can deserialize it as follows:

```python if “dtype” in config and isinstance(config[“dtype”], dict):

policy = dtype_policies.deserialize(config[“dtype”])

```

Args:

config: A Python dictionary, typically the output of get_config.

Returns:

An operation instance.

get_config()

Returns the config of the object.

An object config is a Python dictionary (serializable) containing the information needed to re-instantiate it.

class chemml.models.NeuralGraphOutput(*args, **kwargs)

Output Convolutional layer in a Neural Graph (as in Duvenaud et. al., 2015). This layer takes a graph as an input. The graph is represented as by three tensors.

  • The atoms tensor represents the features of the nodes.

  • The bonds tensor represents the features of the edges.

  • The edges tensor represents the connectivity (which atoms are connected to

    which)

It returns the fingerprint vector for each sample for the given layer.

According to the original paper, the fingerprint outputs of each hidden layer need to be summed in the end to come up with the final fingerprint.

# Example

Define the input: ```python

atoms0 = Input(name=’atom_inputs’, shape=(max_atoms, num_atom_features)) bonds = Input(name=’bond_inputs’, shape=(max_atoms, max_degree, num_bond_features)) edges = Input(name=’edge_inputs’, shape=(max_atoms, max_degree), dtype=’int32’)

```

The NeuralGraphOutput can be initialised in three ways: 1. Using an integer fp_length and possible kwags (Dense layer is used)

`python fp_out = NeuralGraphOutput(fp_length, activation='relu', bias=False)([atoms0, bonds, edges]) `

  1. Using an initialised Dense layer

    `python fp_out = NeuralGraphOutput(Dense(fp_length, activation='relu', bias=False))([atoms0, bonds, edges]) `

  2. Using a function that returns an initialised Dense layer

    `python fp_out = NeuralGraphOutput(lambda: Dense(fp_length, activation='relu', bias=False))([atoms0, bonds, edges]) `

Predict for regression: `python main_prediction = Dense(1, activation='linear', name='main_prediction')(fp_out) `

# Arguments
inner_layer_arg: Either:
  1. an int defining the fp_length, with optional kwargs for the

    inner Dense layer

  2. An initialised but not build (Dense) keras layer (like a wrapper)

  3. A function that returns an initialised keras layer.

kwargs: For initialisation 1. you can pass Dense layer kwargs

# Input shape

List of Atom and edge tensors of shape: `[(samples, max_atoms, atom_features), (samples, max_atoms, max_degrees,

bond_features), (samples, max_atoms, max_degrees)]`

where degrees referes to number of neighbours

# Output shape

Fingerprints matrix (samples, fp_length)

# References
classmethod from_config(config)

Creates an operation from its config.

This method is the reverse of get_config, capable of instantiating the same operation from the config dictionary.

Note: If you override this method, you might receive a serialized dtype config, which is a dict. You can deserialize it as follows:

```python if “dtype” in config and isinstance(config[“dtype”], dict):

policy = dtype_policies.deserialize(config[“dtype”])

```

Args:

config: A Python dictionary, typically the output of get_config.

Returns:

An operation instance.

get_config()

Returns the config of the object.

An object config is a Python dictionary (serializable) containing the information needed to re-instantiate it.

class chemml.models.TransferLearning(base_model, nfeatures=None, n_layers=None)

Class used to facilitate transfer learning from a parent (or head) model to a child model. Freezes the layers (weights and biases) of the pre-trained base model and removes the output layers and appends the child model to the base model.

Parameters

base_model: chemml.models.MLP object, tensorflow.python.keras object, or a torch.nn.Module object

pre-trained base model

n_features: int or None (default=None)

no. of input features provided for training the base model. If base_model is a tensorflow.python.keras object or torch.nn.Modules object, then n_features must be int, else it can be None.

n_layers: int, optional (default=None)

remove the last ‘n’ hidden layers from the base model. Note that this number should not include the output layer.

transfer(X, y, child_model)

Adds the base model’s frozen layers (without its input and output layers) to the child model and fits the new model to the training data.

Parameters

X, y: numpy.ndarray

X is an array of features and y is an array of the target values X should have the same input features (columns) as the base model.

child_model: chemml.models.mlp.MLP object

chemml model created with all the parameters and options required for the final transfer learned model

Returns

child_model: chemml.models.mlp.MLP object

The trained transfer-learned child model.