Models module

The chemml.models.keras module includes (please click on links adjacent to function names for more information):

class chemml.models.MLP(engine, nfeatures, nneurons=None, activations=None, learning_rate=0.01, nepochs=100, batch_size=100, alpha=0.001, loss='mean_squared_error', is_regression=True, nclasses=None, noutputs=1, layer_config_file=None, opt_config='sgd', random_seed=112, verbose=None, device=None, use_batching=False, **params)

Class associated with Multi-Layer Perceptron (Neural Network)

Parameters

engine: str

Determines the underlying ML library used to build the deep neural network can be either ‘tensorflow’ or ‘pytorch’

nfeatures: int

number of input features

nneurons: list, default = None

The number of nodes in each hidden layer, required if layer_config_file is not provided

activations: list, default = None

The activation type for each hidden layer (len of activations should be the same as len of nneurons) required if layer_config_file is not provided Refer https://keras.io/activations/ for list of valid activations for tensorflow Refer https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity or https://pytorch.org/docs/stable/nn.html#non-linear-activations-other for list of valid activations for pytorch e.g. [‘ReLU’,’ReLU’]

nepochs: int, optional, default: 100

Number of training epochs.

batch_size: int, optional, default: 100

Number of training samples in mini-batch

alpha: float, default: 0.001 (defaults for pytorch: 0, keras: 0.01, sklearn: 0.0001)

L2 regularization parameter. If engine is pytorch, this will override the weight_decay parameter in the SGD optimizer

loss: str, optional, default: ‘mean_squared_error’

Type of loss used to train the neural network. Refer https://keras.io/losses/ for list of valid losses for tensorflow Refer https://pytorch.org/docs/stable/nn.html#loss-functions for valid losses for pytorch

regression: bool, optional, default: True

Decides whether we are training for regression or classification task

nclasses: int, optional, default: None

Number of classes labels needs to be specified if regression is False

noutputs: int, optional, default: 1

Number of output nodes. Only used when is_regression is True. For multi-output regression, set this to the desired number of outputs. For classification, output count is determined by nclasses.

layer_config_file: str, optional, default: None

Path to the file that specifies layer configuration Refer MLP test to see a sample file Note: this variable SHOULD be consolidated with the layers variable to reduce redundancy

opt_config: list or str, optional, default: ‘sgd’

optimizer configuration. If str, should either be ‘sgd’ or ‘adam’ If list, should provide exact configurations and parameters corresponding to the respective engines, for e.g., [“Adam”,{“learning_rate”:0.01}] or [“SGD”,{“lr”:0.01, “momentum”:0.9, “lr_decay”:0.0, nesterov=False)]

random_seed: int, optional, default: 112

verbose: bool, optional, default: None

Determines whether TensorFlow models print train loss every epoch Mainly used to make AutoML outputs easier to read.

device: str, optional, default: None

PyTorch engine only. None or ‘cpu’ runs on the CPU, ‘auto’ uses a GPU if available, ‘cuda’ or ‘cuda:N’ requests a specific GPU (works for both CUDA and ROCm builds of PyTorch). TensorFlow uses any visible GPU automatically.

use_batching: bool, optional, default: False

PyTorch engine only. If True, transfers each mini-batch to the target device instead of moving the full training dataset there before fitting.

get_model(include_output=True, n_layers=None)

Returns the entire tensorflow or pytorch model in its current state (fitted or compiled)

Parameters

include_output: bool

if True it will return the entire model, if False it will return the model without the output layer

n_layers: int, optional (default=None)

remove the last ‘n’ hidden layers from the model in addition to the output layer. Note that this number should not include the output layer.

Returns

self.model: tensorflow.python.keras type object or torch.nn.Module type object

get_params(deep=True)

Get parameters for this model.

Follows scikit-learn’s convention for model parameter retrieval. Compatible with scikit-learn’s model selection tools.

Parameters

deepbool, default=True

If True, will return parameters for this estimator and contained subobjects (for other estimators). Not used in this implementation but kept for API consistency.

Returns

paramsdict

Parameter names mapped to their values.

load(path_to_model)

Loads the chemml.models.MLP object along with the underlying tensorflow.python.keras object

Parameters

path_to_model: str

path to the chemml.models.MLP csv file

save(path, filename)

Saves the chemml.models.MLP object along with the underlying tensorflow.python.keras object

Parameters

path: str

the path to the directory where the models should be saved

filename: str

the name of the model file without the file type

class chemml.models.NeuralGraphFingerprint(engine, conv_width=8, fp_length=200, n_conv_layers=2, hidden_sizes=None, hidden_activations=None, conv_activation='relu', fp_activation='softmax', epochs=100, batch_size=32, learning_rate=0.001, optimizer='adam', alpha=0.001, use_bias=True, verbose=True, random_seed=None, regression=True, n_classes=None, device=None)

Neural graph fingerprint model with engine selection.

Generates fixed-size molecular fingerprints using graph neural networks. Supports lazy model building and automatic multi-output detection.

Parameters

enginestr

‘tensorflow’ or ‘pytorch’

conv_widthint, optional

Width of convolution layers. Default 8.

fp_lengthint, optional

Fingerprint output dimension. Default 200.

n_conv_layersint, optional

Number of convolutional layers. Default 2.

hidden_sizeslist, optional

Sizes of optional MLP head layers. Default None (no MLP head).

hidden_activationslist, optional

Activations for MLP head. Default None.

conv_activationstr, optional

Activation for conv layers. Default ‘relu’.

fp_activationstr, optional

Activation for fingerprint layers. Default ‘softmax’.

epochsint, optional

Training epochs. Default 100.

batch_sizeint, optional

Batch size. Default 32.

learning_ratefloat, optional

Learning rate. Default 0.001.

optimizerstr, optional

‘adam’ or ‘sgd’. Default ‘adam’.

alphafloat, optional

L2 regularization weight (PyTorch only). Default 0.001.

use_biasbool, optional

Whether to use bias in layers. Default True.

verbosebool, optional

Print training progress. Default True.

random_seedint, optional

Random seed. Default None.

regressionbool, optional

If True, model performs regression. If False, multiclass classification. Default True.

n_classesint, optional

Number of classes for classification (required if regression=False). Default None.

devicestr, optional

PyTorch engine only. None/’cpu’, ‘auto’ (GPU if available), ‘cuda’ or ‘cuda:N’ (works for CUDA and ROCm builds). TensorFlow uses any visible GPU automatically. Default None.

fit(atoms, bonds=None, edges=None, y=None, tensorize_kwargs=None, **kwargs)

Train the model.

Supports both raw molecular inputs and pre-tensorized inputs.

Parameters

atomsarray_like or str/list/Molecule

Either: - Pre-tensorized atom features, shape (n_samples, max_atoms, num_atom_features) - Raw molecular input: SMILES string, list of SMILES, Molecule, or list of Molecules

bondsarray_like, optional

Bond features if atoms is pre-tensorized. Not needed if atoms is raw molecular input.

edgesarray_like, optional

Edge connectivity if atoms is pre-tensorized. Not needed if atoms is raw molecular input.

yarray_like

Target values, shape (n_samples,) for single output or (n_samples, n_outputs) for multi-output

tensorize_kwargsdict, optional

Arguments for _tensorize_input when using raw molecular input. Supported keys: max_degree, max_atoms, n_jobs, batch_size, verbose. Default: {max_degree=5, max_atoms=None, n_jobs=1, batch_size=10, verbose=False}

Returns

self

Examples

# Pre-tensorized input (existing workflow) ngf.fit(atoms, bonds, edges, y)

# Raw SMILES input (new workflow) ngf.fit([‘CC’, ‘CCO’, ‘CCCO’], y=y)

# Raw Molecule objects (new workflow) ngf.fit(mol_objs_list, y=y)

# With tensorization options ngf.fit(smiles_list, y=y, tensorize_kwargs={‘max_degree’: 5, ‘n_jobs’: -1})

get_model()

Return the underlying model object.

Returns

modeltensorflow.keras.Model or _NeuralGraphNetworkPT

The underlying model

predict(atoms, bonds=None, edges=None, tensorize_kwargs=None)

Generate predictions.

Supports both raw molecular inputs and pre-tensorized inputs.

Parameters

atomsarray_like or str/list/Molecule

Either: - Pre-tensorized atom features, shape (n_samples, max_atoms, num_atom_features) - Raw molecular input: SMILES string, list of SMILES, Molecule, or list of Molecules

bondsarray_like, optional

Bond features if atoms is pre-tensorized. Not needed if atoms is raw molecular input.

edgesarray_like, optional

Edge connectivity if atoms is pre-tensorized. Not needed if atoms is raw molecular input.

tensorize_kwargsdict, optional

Arguments for _tensorize_input when using raw molecular input. Supported keys: max_degree, max_atoms, n_jobs, batch_size, verbose.

Returns

predictionsndarray

Predictions, shape (n_samples,) for single output or (n_samples, n_outputs) for multi-output

Examples

# Pre-tensorized input predictions = ngf.predict(atoms_test, bonds_test, edges_test)

# Raw SMILES input predictions = ngf.predict([‘CC’, ‘CCO’, ‘CCCO’])

# Raw Molecule objects predictions = ngf.predict(mol_objs_test)

predict_proba(atoms, bonds=None, edges=None, tensorize_kwargs=None)

Get class probabilities.

Supports both raw molecular inputs and pre-tensorized inputs.

Parameters

atomsarray_like or str/list/Molecule

Either: - Pre-tensorized atom features, shape (n_samples, max_atoms, num_atom_features) - Raw molecular input: SMILES string, list of SMILES, Molecule, or list of Molecules

bondsarray_like, optional

Bond features if atoms is pre-tensorized.

edgesarray_like, optional

Edge connectivity if atoms is pre-tensorized.

tensorize_kwargsdict, optional

Arguments for _tensorize_input when using raw molecular input.

Returns

probabilitiesndarray

Class probabilities, shape (n_samples, n_classes)

Raises

ValueError

If model is regression or not built

class chemml.models.NeuralGraphHidden(*args, **kwargs)

Hidden Convolutional layer in a Neural Graph (as in Duvenaud et. al.,

2015). This layer takes a graph as an input. The graph is represented as by three tensors. - The atoms tensor represents the features of the nodes. - The bonds tensor represents the features of the edges. - The edges tensor represents the connectivity (which atoms are connected to

which)

It returns the convolved features tensor, which is very similar to the atoms tensor. Instead of each node being represented by a num_atom_features-sized vector, each node now is represented by a convolved feature vector of size conv_width. # Example

Define the input: ```python

atoms0 = Input(name=’atom_inputs’, shape=(max_atoms, num_atom_features)) bonds = Input(name=’bond_inputs’, shape=(max_atoms, max_degree, num_bond_features)) edges = Input(name=’edge_inputs’, shape=(max_atoms, max_degree), dtype=’int32’)

``` The NeuralGraphHidden can be initialised in three ways: 1. Using an integer conv_width and possible kwags (Dense layer is used)

`python atoms1 = NeuralGraphHidden(conv_width, activation='relu', bias=False)([atoms0, bonds, edges]) `

2. Using an initialised Dense layer

   `python atoms1 = NeuralGraphHidden(Dense(conv_width, activation='relu', bias=False))([atoms0, bonds, edges]) `

3. Using a function that returns an initialised Dense layer

   `python atoms1 = NeuralGraphHidden(lambda: Dense(conv_width, activation='relu', bias=False))([atoms0, bonds, edges]) `

Use NeuralGraphOutput to convert atom layer to fingerprint

# Arguments

inner_layer_arg: Either:

1. an int defining the conv_width, with optional kwargs for the

   inner Dense layer

2. An initialised but not build (Dense) keras layer (like a wrapper)

3. A function that returns an initialised keras layer.

kwargs: For initialisation 1. you can pass Dense layer kwargs

# Input shape

List of Atom and edge tensors of shape: `[(samples, max_atoms, atom_features), (samples, max_atoms, max_degrees,

bond_features), (samples, max_atoms, max_degrees)]`

where degrees referes to number of neighbours

# Output shape

New atom featuers of shape (samples, max_atoms, conv_width)

# References

• [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://arxiv.org/abs/1509.09292)

classmethod from_config(config)

Creates an operation from its config.

This method is the reverse of get_config, capable of instantiating the same operation from the config dictionary.

Note: If you override this method, you might receive a serialized dtype config, which is a dict. You can deserialize it as follows:

```python if “dtype” in config and isinstance(config[“dtype”], dict):

policy = dtype_policies.deserialize(config[“dtype”])

```

Args:

config: A Python dictionary, typically the output of get_config.

Returns:

An operation instance.

get_config()

Returns the config of the object.

An object config is a Python dictionary (serializable) containing the information needed to re-instantiate it.

class chemml.models.NeuralGraphOutput(*args, **kwargs)

Output Convolutional layer in a Neural Graph (as in Duvenaud et. al., 2015). This layer takes a graph as an input. The graph is represented as by three tensors.

• The atoms tensor represents the features of the nodes.

• The bonds tensor represents the features of the edges.

• The edges tensor represents the connectivity (which atoms are connected to

  which)

It returns the fingerprint vector for each sample for the given layer.

According to the original paper, the fingerprint outputs of each hidden layer need to be summed in the end to come up with the final fingerprint.

# Example

Define the input: ```python

atoms0 = Input(name=’atom_inputs’, shape=(max_atoms, num_atom_features)) bonds = Input(name=’bond_inputs’, shape=(max_atoms, max_degree, num_bond_features)) edges = Input(name=’edge_inputs’, shape=(max_atoms, max_degree), dtype=’int32’)

```

The NeuralGraphOutput can be initialised in three ways: 1. Using an integer fp_length and possible kwags (Dense layer is used)

`python fp_out = NeuralGraphOutput(fp_length, activation='relu', bias=False)([atoms0, bonds, edges]) `

2. Using an initialised Dense layer

   `python fp_out = NeuralGraphOutput(Dense(fp_length, activation='relu', bias=False))([atoms0, bonds, edges]) `

3. Using a function that returns an initialised Dense layer

   `python fp_out = NeuralGraphOutput(lambda: Dense(fp_length, activation='relu', bias=False))([atoms0, bonds, edges]) `

Predict for regression: `python main_prediction = Dense(1, activation='linear', name='main_prediction')(fp_out) `

# Arguments

inner_layer_arg: Either:

1. an int defining the fp_length, with optional kwargs for the

   inner Dense layer

2. An initialised but not build (Dense) keras layer (like a wrapper)

3. A function that returns an initialised keras layer.

kwargs: For initialisation 1. you can pass Dense layer kwargs

# Input shape

List of Atom and edge tensors of shape: `[(samples, max_atoms, atom_features), (samples, max_atoms, max_degrees,

bond_features), (samples, max_atoms, max_degrees)]`

where degrees referes to number of neighbours

# Output shape

Fingerprints matrix (samples, fp_length)

# References

• [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://arxiv.org/abs/1509.09292)

classmethod from_config(config)

Creates an operation from its config.

This method is the reverse of get_config, capable of instantiating the same operation from the config dictionary.

Note: If you override this method, you might receive a serialized dtype config, which is a dict. You can deserialize it as follows:

```python if “dtype” in config and isinstance(config[“dtype”], dict):

policy = dtype_policies.deserialize(config[“dtype”])

```

Args:

config: A Python dictionary, typically the output of get_config.

Returns:

An operation instance.

get_config()

Returns the config of the object.

An object config is a Python dictionary (serializable) containing the information needed to re-instantiate it.

class chemml.models.TransferLearning(base_model, nfeatures=None, n_layers=None)

Class used to facilitate transfer learning from a parent (or head) model to a child model. Freezes the layers (weights and biases) of the pre-trained base model and removes the output layers and appends the child model to the base model.

Parameters

base_model: chemml.models.MLP object, tensorflow.python.keras object, or a torch.nn.Module object

pre-trained base model

n_features: int or None (default=None)

no. of input features provided for training the base model. If base_model is a tensorflow.python.keras object or torch.nn.Modules object, then n_features must be int, else it can be None.

n_layers: int, optional (default=None)

remove the last ‘n’ hidden layers from the base model. Note that this number should not include the output layer.

transfer(X, y, child_model)

Adds the base model’s frozen layers (without its input and output layers) to the child model and fits the new model to the training data.

Parameters

X, y: numpy.ndarray

X is an array of features and y is an array of the target values X should have the same input features (columns) as the base model.

child_model: chemml.models.mlp.MLP object

chemml model created with all the parameters and options required for the final transfer learned model

Returns

child_model: chemml.models.mlp.MLP object

The trained transfer-learned child model.