Table of Contents

rectorch.models

Class list

rectorch.models.RecSysModel

Abstract base class that any Recommendation model must inherit from.

rectorch.models.TorchNNTrainer(net[, …])

Abstract class representing a neural network-based model.

rectorch.models.AETrainer(ae_net[, …])

Base class for Autoencoder-based models.

rectorch.models.VAE(ae_net[, learning_rate])

Class representing a standard Variational Autoencoder.

rectorch.models.MultiDAE(mdae_net[, lam, …])

Denoising Autoencoder with multinomial likelihood for collaborative filtering.

rectorch.models.MultiVAE(mvae_net[, beta, …])

Variational Autoencoder for collaborative Filtering.

rectorch.models.CMultiVAE(cmvae_net[, beta, …])

Conditioned Variatonal Autoencoder for collaborative filtering.

rectorch.models.EASE([lam])

Embarrassingly Shallow AutoEncoders for Sparse Data (EASE) model.

rectorch.models.CFGAN(generator, discriminator)

A Generic Collaborative Filtering Framework based on Generative Adversarial Networks.

rectorch.models.ADMM_Slim([lambda1, …])

ADMM SLIM: Sparse Recommendations for Many Users.

rectorch.models.SVAE(svae_net[, beta, …])

Sequential Variational Autoencoders for Collaborative Filtering.

This module includes the training algorithm for a bunch of state-of-the-art recommender systems.

Each new model must be a sub-class of the abstract class RecSysModel. Moreover, if the model is a standard neural network (NN) then it is advisable to inherit from TorchNNTrainer that offers a good base structure to develop a new NN training algorithm. In these first releases of rectorch all models will be located in this module, but in the future we plan to improve the structure of the library creating sub-modules.

Currently the implemented training algorithms are:

It is also implemented a generic Variational autoencoder trainer (VAE) based on the classic loss function cross-entropy based reconstruction loss, plus the KL loss.

See also

Modules: nets samplers

References

Rd0d07cdfe4c4-VAE(1,2)

Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 689–698. DOI: https://doi.org/10.1145/3178876.3186150

Rd0d07cdfe4c4-CVAE

Tommaso Carraro, Mirko Polato and Fabio Aiolli. Conditioned Variational Autoencoder for top-N item recommendation, 2020. arXiv pre-print: https://arxiv.org/abs/2004.11141

Rd0d07cdfe4c4-CFGAN

Dong-Kyu Chae, Jin-Soo Kang, Sang-Wook Kim, and Jung-Tae Lee. 2018. CFGAN: A Generic Collaborative Filtering Framework based on Generative Adversarial Networks. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM ’18). Association for Computing Machinery, New York, NY, USA, 137–146. DOI: https://doi.org/10.1145/3269206.3271743

Rd0d07cdfe4c4-EASE

Harald Steck. 2019. Embarrassingly Shallow Autoencoders for Sparse Data. In The World Wide Web Conference (WWW ’19). Association for Computing Machinery, New York, NY, USA, 3251–3257. DOI: https://doi.org/10.1145/3308558.3313710

Rd0d07cdfe4c4-ADMMS

Harald Steck, Maria Dimakopoulou, Nickolai Riabov, and Tony Jebara. 2020. ADMM SLIM: Sparse Recommendations for Many Users. In Proceedings of the 13th International Conference on Web Search and Data Mining (WSDM ’20). Association for Computing Machinery, New York, NY, USA, 555–563. DOI: https://doi.org/10.1145/3336191.3371774

Rd0d07cdfe4c4-SVAE

Noveen Sachdeva, Giuseppe Manco, Ettore Ritacco, and Vikram Pudi. 2019. Sequential Variational Autoencoders for Collaborative Filtering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (WSDM ’19). Association for Computing Machinery, New York, NY, USA, 600–608. DOI: https://doi.org/10.1145/3289600.3291007

class rectorch.models.RecSysModel[source]

Bases: object

Abstract base class that any Recommendation model must inherit from.

load_model(self, filepath, \*args, \*\*kwargs)[source]

Load the model from file.

Parameters
  • filepathstr

      • String representing the path to the file where the model is saved.

  • *argslist [optional]

      • These are the potential additional parameters useful to the model for performing the prediction.

  • **kwargsdict [optional]

      • These are the potential keyword parameters useful to the model for performing the prediction.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    predict(self, x, \*args, \*\*kwargs)[source]

    Perform the prediction using a trained model.

    The prediction is preformed over a generic input x and the method should be callable after the training procedure (RecSysModel.train()).

    Parameters
  • xrectorch.samplers.Sampler or scipy.sparse.csr_matrix or

      • torch.Tensor The input for which the prediction has to be computed.

  • *argslist [optional]

      • These are the potential additional parameters useful to the model for performing the prediction.

  • **kwargsdict [optional]

      • These are the potential keyword parameters useful to the model for performing the prediction.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    save_model(self, filepath, \*args, \*\*kwargs)[source]

    Save the model to file.

    Parameters
  • filepathstr

      • String representing the path to the file to save the model.

  • *argslist [optional]

      • These are the potential additional parameters useful to the model for performing the prediction.

  • **kwargsdict [optional]

      • These are the potential keyword parameters useful to the model for performing the prediction.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    train(self, train_data, \*\*kwargs)[source]

    Training procedure.

    This method is meant to execute all the training phase. Once the method ends, the model should be ready to be queried for predictions.

    Parameters
  • train_datarectorch.samplers.Sampler or scipy.sparse.csr_matrix or
  • **kargsdict [optional]

      • These are the potential keyword parameters useful to the model for performing the training.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    class rectorch.models.TorchNNTrainer(net, learning_rate=0.001)[source]

    Bases: rectorch.models.RecSysModel

    Abstract class representing a neural network-based model.

    This base class assumes that the model can be trained using a standard backpropagation procedure. It is not meant to manage complex training patterns, such as alternate training between more than one network as done with Generative Adversarial Networks. Thus, it assumes that there is a neural network (i.e., torch.nn.Module)for which the parameters must be learned.

    Parameters
  • nettorch.nn.Module

      • The neural network architecture.

  • learning_ratefloat [optional]

      • The learning rate for the optimizer, by default 1e-3.

    Attributes
  • networktorch.nn.Module

      • The neural network architecture.

  • learning_ratefloat

      • The learning rate for the optimizer.

  • optimizertorch.optim.Optimizer

      • Optimizer used for performing the training.

  • devicetorch.device

      • Device where the pytorch tensors are saved.

    loss_function(self, prediction, ground_truth, \*args, \*\*kwargs)[source]

    The loss function that the model wants to minimize.

    Parameters
  • predictiontorch.Tensor

      • The prediction tensor.

  • ground_truthtorch.Tensor

      • The ground truth tensor that the model should have reconstructed correctly.

  • *argslist [optional]

      • These are the potential additional parameters useful to the model for computing the loss.

  • **kwargsdict [optional]

      • These are the potential keyword parameters useful to the model for computing the loss.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    predict(self, x, \*args, \*\*kwargs)[source]

    Perform the prediction using a trained model.

    The prediction is preformed over a generic input x and the method should be callable after the training procedure (RecSysModel.train()).

    Parameters
  • xrectorch.samplers.Sampler or scipy.sparse.csr_matrix or

      • torch.Tensor The input for which the prediction has to be computed.

  • *argslist [optional]

      • These are the potential additional parameters useful to the model for performing the prediction.

  • **kwargsdict [optional]

      • These are the potential keyword parameters useful to the model for performing the prediction.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    train(self, train_data, valid_data=None, valid_metric=None, valid_func=ValidFunc(fun='evaluate', params=None), num_epochs=100, verbose=1, \*\*kwargs)[source]

    Training of a neural network-based model.

    Parameters
  • train_datarectorch.samplers.Sampler

      • The sampler object that load the training set in mini-batches.

  • valid_datarectorch.samplers.Sampler [optional]

      • The sampler object that load the validation set in mini-batches, by default None. If the model does not have any validation procedure set this parameter to None.

  • valid_metricstr [optional]

      • The metric used during the validation to select the best model, by default None. If valid_data is not None then valid_metric must be not None. To see the valid strings for the metric please see the module metrics.

  • valid_funcevaluation.ValidFunc [optional]

      • The validation function, by default a standard validation procedure, i.e., evaluation.evaluate().

  • num_epochsint [optional]

      • Number of training epochs, by default 100.

  • verboseint [optional]

      • The level of verbosity of the logging, by default 1. The level can have any integer value greater than 0. However, after reaching a maximum (that depends on the size of the training set) verbosity higher values will not have any effect.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    train_batch(self, epoch, tr_batch, te_batch, \*args, \*\*kwargs)[source]

    Training of a single batch.

    Parameters
  • epochint

      • Epoch’s number.

  • tr_batchtorch.Tensor

      • Traning part of the current batch.

  • te_batchtorch.Tensor or None

      • Test part of the current batch, if any, otherwise None.

  • *argslist [optional]

      • These are the potential additional parameters useful to the model for performing the training on the batch.

  • **kwargsdict [optional]

      • These are the potential keyword parameters useful to the model for performing the training on the batch.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    train_epoch(self, epoch, train_data, \*args, \*\*kwargs)[source]

    Training of a single epoch.

    Parameters
  • epochint

      • Epoch’s number.

  • train_datarectorch.samplers.Sampler

      • The sampler object that load the training set in mini-batches.

  • *argslist [optional]

      • These are the potential additional parameters useful to the model for performing the training.

  • **kwargsdict [optional]

      • These are the potential keyword parameters useful to the model for performing the training.

    Raises
  • NotImplementedError

      • Raised when not implemeneted in the sub-class.

    class rectorch.models.AETrainer(ae_net, learning_rate=0.001)[source]

    Bases: rectorch.models.TorchNNTrainer

    Base class for Autoencoder-based models.

    Parameters
  • ae_nettorch.nn.Module

      • The autoencoder neural network.

  • learning_ratefloat [optional]

      • The learning rate for the optimizer, by default 1e-3.

    Attributes
  • optimizertorch.optim.Adam

      • The optimizer is an Adam optimizer with default parameters and learning rate equals to learning_rate.

  • other attributessee the base class TorchNNTrainer.
      • load_model(self, filepath)[source]

      Load the model from file.

      Parameters
    • filepathstr

        • String representing the path to the file where the model is saved.

      Returns
    • dict

        • A dictionary that summarizes the state of the model when it has been saved. Note: not all the information about the model are stored in the saved ‘checkpoint’.

      loss_function(self, prediction, ground_truth)[source]

      Vanilla Autoencoder loss function.

      This is a standard Mean Squared Error (squared L2 norm) loss, that is

      \(\mathcal{L} = \frac{1}{L} \sum_{i=1}^L (x_i - y_i)^2\)

      where L is the batch size x and y are the ground truth and the prediction, respectively.

      Parameters
    • predictiontorch.Tensor

        • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over a batch.

    • ground_truthtorch.Tensor

        • The input, and hence the target tensor. It is meant to be a batch size input.

    • mutorch.Tensor

        • The mean part of latent space for the given x. Together with logvar represents the representation of the input x before the reparameteriation trick.

    • logvartorch.Tensor

        • The (logarithm of the) variance part of latent space for the given x. Together with mu represents the representation of the input x before the reparameteriation trick.

      Returns
    • torch.Tensor

        • Tensor (\(1 \times 1\)) representing the average loss incurred over the input batch.

      predict(self, x, remove_train=True)[source]

      Perform the prediction using a trained Autoencoder.

      Parameters
    • xtorch.Tensor

        • The input for which the prediction has to be computed.

    • remove_trainbool [optional]

        • Whether to remove the training set from the prediction, by default True. Removing the training items means set their scores to \(-\infty\).

      Returns
    • recon_x,tuple with a single element
      • recon_xtorch.Tensor

          • The reconstructed input, i.e., the output of the autoencoder. It is meant to be the reconstruction over the input batch x.

      save_model(self, filepath, cur_epoch)[source]

      Save the model to file.

      Parameters
    • filepathstr

        • String representing the path to the file to save the model.

    • cur_epochint

        • The last training epoch.

      train(self, train_data, valid_data=None, valid_metric=None, valid_func=ValidFunc(fun='evaluate', params=None), num_epochs=100, verbose=1)[source]

      Training of a neural network-based model.

      Parameters
    • train_datarectorch.samplers.Sampler

        • The sampler object that load the training set in mini-batches.

    • valid_datarectorch.samplers.Sampler [optional]

        • The sampler object that load the validation set in mini-batches, by default None. If the model does not have any validation procedure set this parameter to None.

    • valid_metricstr [optional]

        • The metric used during the validation to select the best model, by default None. If valid_data is not None then valid_metric must be not None. To see the valid strings for the metric please see the module metrics.

    • valid_funcevaluation.ValidFunc [optional]

        • The validation function, by default a standard validation procedure, i.e., evaluation.evaluate().

    • num_epochsint [optional]

        • Number of training epochs, by default 100.

    • verboseint [optional]

        • The level of verbosity of the logging, by default 1. The level can have any integer value greater than 0. However, after reaching a maximum (that depends on the size of the training set) verbosity higher values will not have any effect.

      Raises
    • NotImplementedError

        • Raised when not implemeneted in the sub-class.

      train_batch(self, tr_batch, te_batch=None)[source]

      Training of a single batch.

      Parameters
    • epochint

        • Epoch’s number.

    • tr_batchtorch.Tensor

        • Traning part of the current batch.

    • te_batchtorch.Tensor or None [optional]

        • Test part of the current batch, if any, otherwise None, by default None.

      Returns
    • float

        • The loss incurred in the batch.

      train_epoch(self, epoch, train_loader, verbose=1)[source]

      Training of a single epoch.

      Parameters
    • epochint

        • Epoch’s number.

    • train_datarectorch.samplers.Sampler

        • The sampler object that load the training set in mini-batches.

    • *argslist [optional]

        • These are the potential additional parameters useful to the model for performing the training.

    • **kwargsdict [optional]

        • These are the potential keyword parameters useful to the model for performing the training.

      Raises
    • NotImplementedError

        • Raised when not implemeneted in the sub-class.

      class rectorch.models.VAE(ae_net, learning_rate=0.001)[source]

      Bases: rectorch.models.AETrainer

      Class representing a standard Variational Autoencoder.

      The learning follows standard backpropagation minimizing the loss described in [R7e07b32cacd8-KINGMA]. See VAE.loss_function() for more details.

      Note

      Parameters and Attributes are the same as in the base class AETrainer.

      References

      R7e07b32cacd8-KINGMA

      Kingma, Diederik P and Welling, Max Auto-Encoding Variational Bayes, 2013. arXiv pre-print: https://arxiv.org/abs/1312.6114.

      loss_function(self, recon_x, x, mu, logvar)[source]

      Standard VAE loss function.

      This method implements the loss function described in [KINGMA] assuming a Gaussian latent, that is:

      \(\mathcal{L} = \mathcal{L}_{rec} + \mathcal{L}_{KL}\)

      where

      \(\mathcal{L}_{rec} = -\frac{1}{L}\ \sum_{l} E_{\sim q_{\theta}(z | x_{i})}[\log p(x_{i} | z^{(i, l)})]\)

      and

      \(\mathcal{L}_{KL} = -\frac{1}{2} \sum_{j=1}^{J}\ [1+\log (\sigma_{i}^{2})-\sigma_{i}^{2}-\mu_{i}^{2}]\)

      with J is the dimension of the latent vector z, and L is the number of samples stochastically drawn according to reparameterization trick.

      Parameters
    • recon_xtorch.Tensor

        • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over a batch.

    • xtorch.Tensor

        • The input, and hence the target tensor. It is meant to be a batch size input.

    • mutorch.Tensor

        • The mean part of latent space for the given x. Together with logvar represents the representation of the input x before the reparameteriation trick.

    • logvartorch.Tensor

        • The (logarithm of the) variance part of latent space for the given x. Together with mu represents the representation of the input x before the reparameteriation trick.

      Returns
    • torch.Tensor

        • Tensor (\(1 \times 1\)) representing the average loss incurred over the input batch.

      References

      KINGMA

      Kingma, Diederik P and Welling, Max Auto-Encoding Variational Bayes, 2013. arXiv pre-print: https://arxiv.org/abs/1312.6114.

      predict(self, x, remove_train=True)[source]

      Perform the prediction using a trained Variational Autoencoder.

      Parameters
    • xtorch.Tensor

        • The input batch tensor for which the prediction must be computed.

    • remove_trainbool [optional]

        • Whether to remove the training set from the prediction, by default True. Removing the training items means set their scores to \(-\infty\).

      Returns
    • recon_x, mu, logvartuple
      • recon_xtorch.Tensor

          • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over the input batch x.

      • mutorch.Tensor

          • The mean part of latent space for the given x. Together with logvar represents the representation of the input x before the reparameteriation trick.

      • logvartorch.Tensor

          • The (logarithm of the) variance part of latent space for the given x. Together with mu represents the representation of the input x before the reparameteriation trick.

      train_batch(self, tr_batch, te_batch=None)[source]

      Training of a single batch.

      Parameters
    • epochint

        • Epoch’s number.

    • tr_batchtorch.Tensor

        • Traning part of the current batch.

    • te_batchtorch.Tensor or None [optional]

        • Test part of the current batch, if any, otherwise None, by default None.

      Returns
    • float

        • The loss incurred in the batch.

      class rectorch.models.MultiDAE(mdae_net, lam=0.2, learning_rate=0.001)[source]

      Bases: rectorch.models.AETrainer

      Denoising Autoencoder with multinomial likelihood for collaborative filtering.

      This model has been proposed in [R124de05ddb52-VAE] as a baseline method to compare with Mult-VAE. The model represent a standard denoising autoencoder in which the data is assumed to be multinomial distributed.

      Parameters
    • mdae_nettorch.nn.Module

        • The autoencoder neural network.

    • lamfloat [optional]

        • The regularization hyper-parameter \(\lambda\) as defined in [R124de05ddb52-VAE], by default 0.2.

    • learning_ratefloat [optional]

        • The learning rate for the optimizer, by default 1e-3.

      References

      R124de05ddb52-VAE(1,2)

      Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 689–698. DOI: https://doi.org/10.1145/3178876.3186150

      loss_function(self, recon_x, x)[source]

      Multinomial likelihood denoising autoencoder loss.

      Since the model assume a multinomial distribution over the input, then te reconstruction loss must be modified with respect to a vanilla VAE. In particular, the MultiDAE loss function is a combination of a reconstruction loss and a regularization loss, i.e.,

      \(\mathcal{L}(\mathbf{x}_{u} ; \theta, \phi) =\ \mathcal{L}_{rec}(\mathbf{x}_{u} ; \theta, \phi) + \lambda\ \mathcal{L}_{reg}(\mathbf{x}_{u} ; \theta, \phi)\)

      where

      \(\mathcal{L}_{rec}(\mathbf{x}_{u} ; \theta, \phi) =\ \mathbb{E}_{q_{\phi}(\mathbf{z}_{u} | \mathbf{x}_{u})}[\log p_{\theta}\ (\mathbf{x}_{u} | \mathbf{z}_{u})]\)

      and

      \(\mathcal{L}_{reg}(\mathbf{x}_{u} ; \theta, \phi) = \| \theta \|_2 + \| \phi \|_2\),

      with \(\mathbf{x}_u\) the input vector and \(\mathbf{z}_u\) the latent vector representing the user u.

      Parameters
    • recon_xtorch.Tensor

        • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over a batch.

    • xtorch.Tensor

        • The input, and hence the target tensor. It is meant to be a batch size input.

      Returns
    • torch.Tensor

        • Tensor (\(1 \times 1\)) representing the average loss incurred over the input batch.

      class rectorch.models.MultiVAE(mvae_net, beta=1.0, anneal_steps=0, learning_rate=0.001)[source]

      Bases: rectorch.models.VAE

      Variational Autoencoder for collaborative Filtering.

      MultiVAE (dubbed Mult-VAE in [Rd1a0b0462d7a-VAE]) is a vanilla VAE in which the data distribution is assumed to be multinomial and the objective function is an under-regularized version of the standard VAE loss function. Specifically, the Kullbach-Liebler divergence term is weighted by an hyper-parameter (\(\beta\)) that shows to improve de recommendations’ quality when < 1. So, the regularization term is weighted less giving to the model more freedom in representing the input in the latent space. More details about this loss are given in MultiVAE.loss_function().

      Parameters
    • mvae_nettorch.nn.Module

        • The variational autoencoder neural network.

    • betafloat [optional]

        • The \(\beta\) hyper-parameter of Multi-VAE. When anneal_steps > 0 then this value is the value to anneal starting from 0, otherwise the beta will be fixed to the given value for the duration of the training. By default 1.

    • anneal_stepsint [optional]

        • Number of annealing step for reaching the target value beta, by default 0. 0 means that no annealing will be performed and the regularization parameter will be fixed to beta.

    • learning_ratefloat [optional]

        • The learning rate for the optimizer, by default 1e-3.

      References

      Rd1a0b0462d7a-VAE

      Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 689–698. DOI: https://doi.org/10.1145/3178876.3186150

      Attributes
    • anneal_stepsint

        • See anneal_steps parameter.

    • self.annealingbool

        • Whether the annealing is active or not. It is implicitely set to True if anneal_steps > 0, otherwise is set to False.

    • gradient_updatesint

        • Number of gradient updates since the beginning of the training. Once gradient_updates >= anneal_steps, then the annealing is complete and the used \(\beta\) in the loss function is beta.

    • betafloat

        • See beta parameter.

    • optimizertorch.optim.Adam

        • The optimizer is an Adam optimizer with default parameters and learning rate equals to learning_rate.

    • other attributessee the base class VAE.
        • load_model(self, filepath)[source]

        Load the model from file.

        Parameters
      • filepathstr

          • String representing the path to the file where the model is saved.

        Returns
      • dict

          • A dictionary that summarizes the state of the model when it has been saved. Note: not all the information about the model are stored in the saved ‘checkpoint’.

        loss_function(self, recon_x, x, mu, logvar, beta=1.0)[source]

        VAE for collaborative filtering loss function.

        MultiVAE assumes a multinomial distribution over the input and this is reflected in the loss function. The loss is a \(\beta\) ELBO (Evidence Lower BOund) in which the regularization part is weighted by a hyper-parameter \(\beta\). Moreover, as in MultiDAE, the reconstruction loss is based on the multinomial likelihood. Specifically, the loss function of MultiVAE is defined as:

        \(\mathcal{L}_{\beta}(\mathbf{x}_{u} ; \theta, \phi)=\ \mathbb{E}_{q_{\phi}(\mathbf{z}_{u} | \mathbf{x}_{u})}[\log p_{\theta}\ (\mathbf{x}_{u} | \mathbf{z}_{u})]-\beta \cdot \operatorname{KL}(q_{\phi}\ (\mathbf{z}_{u} | \mathbf{x}_{u}) \| p(\mathbf{z}_{u}))\)

        Parameters
      • recon_xtorch.Tensor

          • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over a batch.

      • xtorch.Tensor

          • The input, and hence the target tensor. It is meant to be a batch size input.

      • mutorch.Tensor

          • The mean part of latent space for the given x. Together with logvar represents the representation of the input x before the reparameteriation trick.

      • logvartorch.Tensor

          • The (logarithm of the) variance part of latent space for the given x. Together with mu represents the representation of the input x before the reparameteriation trick.

      • betafloat [optional]

          • The current \(\beta\) regularization hyper-parameter, by default 1.0.

        Returns
      • torch.Tensor

          • Tensor (\(1 \times 1\)) representing the average loss incurred over the input batch.

        save_model(self, filepath, cur_epoch)[source]

        Save the model to file.

        Parameters
      • filepathstr

          • String representing the path to the file to save the model.

      • cur_epochint

          • The last training epoch.

        train(self, train_data, valid_data=None, valid_metric=None, valid_func=ValidFunc(fun='evaluate', params=None), num_epochs=200, best_path='chkpt_best.pth', verbose=1)[source]

        Training procedure for Multi-VAE.

        The training of MultiVAE follows pretty much the same as a standard VAE with the only difference in the (possible) annealing of the hyper-parameter \(\beta\). This model also offer the possibility of keeping track of the best performing model in validation by setting the validation data (valid_data) and metric (valid_metric). The model will be saved in the file indicated in the parameter best_path.

        Parameters
      • train_datarectorch.samplers.Sampler

          • The sampler object that load the training set in mini-batches.

      • valid_datarectorch.samplers.Sampler [optional]

          • The sampler object that load the validation set in mini-batches, by default None. If the model does not have any validation procedure set this parameter to None.

      • valid_metricstr [optional]

          • The metric used during the validation to select the best model, by default None. If valid_data is not None then valid_metric must be not None. To see the valid strings for the metric please see the module metrics.

      • valid_funcevaluation.ValidFunc [optional]

          • The validation function, by default a standard validation procedure, i.e., evaluation.evaluate().

      • num_epochsint [optional]

          • Number of training epochs, by default 100.

      • best_pathstr [optional]

          • String representing the path where to save the best performing model on the validation set. By default "chkpt_best.pth".

      • verboseint [optional]

          • The level of verbosity of the logging, by default 1. The level can have any integer value greater than 0. However, after reaching a maximum (that depends on the size of the training set) verbosity higher values will not have any effect.

        train_batch(self, tr_batch, te_batch=None)[source]

        Training of a single batch.

        Parameters
      • epochint

          • Epoch’s number.

      • tr_batchtorch.Tensor

          • Traning part of the current batch.

      • te_batchtorch.Tensor or None [optional]

          • Test part of the current batch, if any, otherwise None, by default None.

        Returns
      • float

          • The loss incurred in the batch.

        class rectorch.models.CMultiVAE(cmvae_net, beta=1.0, anneal_steps=0, learning_rate=0.001)[source]

        Bases: rectorch.models.MultiVAE

        Conditioned Variatonal Autoencoder for collaborative filtering.

        Conditioned Variational Autoencoder (C-VAE) for constrained top-N item recommendation can recommend items that have to satisfy a given condition. The architecture is similar to a standard VAE in which the condition vector is fed into the encoder. The loss function can be seen in two ways:

        • same as in MultiVAE but with a different target reconstruction. Infact, the

            • network has to reconstruct only those items satisfying a specific condition;

        • a modified loss which performs the filtering by itself.

        More details about the loss function are given in the paper [R616c5a496243-CVAE].

        The training process is almost identical to the one of MultiVAE but the sampler must be a samplers.ConditionedDataSampler.

        Note

        For parameters and attributes please refer to MultiVAE.

        References

        R616c5a496243-CVAE

        Tommaso Carraro, Mirko Polato and Fabio Aiolli. Conditioned Variational Autoencoder for top-N item recommendation, 2020. arXiv pre-print: https://arxiv.org/abs/2004.11141

        predict(self, x, remove_train=True)[source]

        Perform the prediction using a trained Variational Autoencoder.

        Parameters
      • xtorch.Tensor

          • The input batch tensor for which the prediction must be computed.

      • remove_trainbool [optional]

          • Whether to remove the training set from the prediction, by default True. Removing the training items means set their scores to \(-\infty\).

        Returns
      • recon_x, mu, logvartuple
        • recon_xtorch.Tensor

            • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over the input batch x.

        • mutorch.Tensor

            • The mean part of latent space for the given x. Together with logvar represents the representation of the input x before the reparameteriation trick.

        • logvartorch.Tensor

            • The (logarithm of the) variance part of latent space for the given x. Together with mu represents the representation of the input x before the reparameteriation trick.

        class rectorch.models.EASE(lam=100.0)[source]

        Bases: rectorch.models.RecSysModel

        Embarrassingly Shallow AutoEncoders for Sparse Data (EASE) model.

        This model has been proposed in [R616dc10ef7fc-EASE] and it can be summarized as follows. Given the rating matrix \(\mathbf{X} \in \mathbb{R}^{n \times m}\) with n users and m items, EASE aims at solving the following optimization problem:

        \(\min_{\mathbf{B}} \|\mathbf{X}-\mathbf{X} \mathbf{B}\|_{F}^{2}+\ \lambda \cdot\|\mathbf{B}\|_{F}^{2}\)

        subject to \(\operatorname{diag}(\mathbf{B})=0\).

        where \(\mathbf{B} \in \mathbb{R}^{m \times m}\) is like a kernel matrix between items. Then, a prediction for a user-item pair (u,j) will be computed by \(S_{u j}=\mathbf{X}_{u,:} \cdot \mathbf{B}_{:, j}\)

        It can be shown that estimating \(\mathbf{B}\) can be done in closed form by computing

        \(\hat{\mathbf{B}}=(\mathbf{X}^{\top} \mathbf{X}+\lambda \mathbf{I})^{-1} \cdot\ (\mathbf{X}^{\top} \mathbf{X}-\mathbf{I}^\top \gamma)\)

        where \(\gamma \in \mathbb{R}^m\) is the vector of Lagragian multipliers, and \(\mathbf{I}\) is the identity matrix.

        Parameters
      • lamfloat [optional]

          • The regularization hyper-parameter, by default 100.

        References

        R616dc10ef7fc-EASE

        Harald Steck. 2019. Embarrassingly Shallow Autoencoders for Sparse Data. In The World Wide Web Conference (WWW ’19). Association for Computing Machinery, New York, NY, USA, 3251–3257. DOI: https://doi.org/10.1145/3308558.3313710

        Attributes
      • lamfloat

          • See lam parameter.

      • modelnumpy.ndarray

          • Represent the model, i.e.m the matrix score S. If the model has not been trained yet model is set to None.

        load_model(self, filepath)[source]

        Load the model from file.

        Parameters
      • filepathstr

          • String representing the path to the file where the model is saved.

      • *argslist [optional]

          • These are the potential additional parameters useful to the model for performing the prediction.

      • **kwargsdict [optional]

          • These are the potential keyword parameters useful to the model for performing the prediction.

        Raises
      • NotImplementedError

          • Raised when not implemeneted in the sub-class.

        predict(self, ids_te_users, test_tr, remove_train=True)[source]

        Prediction using the EASE model.

        For the EASE model the prediction list for a user u is done by computing

        \(S_{u}=\mathbf{X}_{u,:} \cdot \mathbf{B}\).

        However, in the rectorch implementation the prediction is simply a look up is the score matrix S.

        Parameters
      • ids_te_usersarray_like

          • List of the test user indexes.

      • test_trscipy.sparse.csr_matrix

          • Training portion of the test users.

      • remove_trainbool [optional]

          • Whether to remove the training set from the prediction, by default True. Removing the training items means set their scores to \(-\infty\).

        Returns
      • pred,tuple with a single element
        • prednumpy.ndarray

            • The items’ score (on the columns) for each user (on the rows).

        save_model(self, filepath)[source]

        Save the model to file.

        Parameters
      • filepathstr

          • String representing the path to the file to save the model.

      • *argslist [optional]

          • These are the potential additional parameters useful to the model for performing the prediction.

      • **kwargsdict [optional]

          • These are the potential keyword parameters useful to the model for performing the prediction.

        Raises
      • NotImplementedError

          • Raised when not implemeneted in the sub-class.

        train(self, train_data)[source]

        Training of the EASE model.

        Parameters
      • train_datascipy.sparse.csr_matrix

          • The training data.

        class rectorch.models.CFGAN(generator, discriminator, alpha=0.1, s_pm=0.7, s_zr=0.5, learning_rate=0.001)[source]

        Bases: rectorch.models.RecSysModel

        A Generic Collaborative Filtering Framework based on Generative Adversarial Networks.

        This recommender systems is based on GAN where the (conditioned) generator aims at generating the ratings of users while the discriminiator tries to discriminate between genuine users profile and generated ones. The two networks try to optimize the followng loss functions:

        • Discriminator (D):

            • \(J^{D}=-\sum_{u}\left(\log D(\mathbf{r}_{u} | \ \mathbf{c}_{u})+\log (1-D(\hat{\mathbf{r}}_{u}\ \odot(\mathbf{e}_{u}+\mathbf{k}_{u}) | \mathbf{c}_{u}))\right)\)

        • Generator (G):

            • \(J^{G}=\sum_{u}\left(\log (1-D(\hat{\mathbf{r}}_{u}\ \odot(\mathbf{e}_{u}+\mathbf{k}_{u}) | \mathbf{c}_{u}))+\ \alpha \cdot \sum_{j}(r_{uj}-\hat{r}_{uj})^{2}\right)\)

        where \(\mathbf{c}_u\) is the condition vector, \(\mathbf{k}_u\) is an n-dimensional indicator vector such that \(k_{uj} = 1\) iff j is a negative sampled item, \(\hat{\mathbf{r}}_u\) is a fake user profile, and \(\mathbf{e}_u\) is the masking vector to remove the non-rated items. For more details please refer to the original paper [Rd8d00c5b2c0a-CFGAN].

        Parameters
      • generatortorch.nn.Module

          • The generator neural network. The expected architecture is \([m, l_1, \dots, l_h, m]\) where m is the number of items \(l_i, i \in [1,h]\) is the number of nodes of the hidden layer i.

      • discriminatortorch.nn.Module

          • The discriminator neural network. The expected architecture is \([2m, l_1, \dots, l_h, 1]\) where m is the number of items \(l_i, i \in [1,h]\) is the number of nodes of the hidden layer i.

      • alphafloat [optional]

          • The ZR-coefficient, by default .1.

      • s_pmfloat [optional]

          • The sampling parameter for the partial-masking approach, by default .7.

      • s_zrfloat [optional]

          • The sampling parameter for the zero-reconstruction regularization, by default .5.

      • learning_ratefloat [optional]

          • The optimization learning rate, by default 0.001.

        References

        Rd8d00c5b2c0a-CFGAN

        Dong-Kyu Chae, Jin-Soo Kang, Sang-Wook Kim, and Jung-Tae Lee. 2018. CFGAN: A Generic Collaborative Filtering Framework based on Generative Adversarial Networks. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM ’18). Association for Computing Machinery, New York, NY, USA, 137–146. DOI: https://doi.org/10.1145/3269206.3271743

        Attributes
      • generatortorch.nn.Module

          • See generator parameter.

      • discriminatortorch.nn.Module

          • See discriminator parameter.

      • alphafloat

          • See alpha paramater.

      • s_pmfloat

          • See s_pm paramater.

      • s_zrfloat

          • See s_zr paramater.

      • learning_ratefloat

          • See learning_rate paramater.

      • opt_gtorch.optim.Adam

          • Optimizer used for performing the training of the generator.

      • opt_dtorch.optim.Adam

          • Optimizer used for performing the training of the discriminator.

        load_model(self, filepath)[source]

        Load the model from file.

        Parameters
      • filepathstr

          • String representing the path to the file where the model is saved.

      • *argslist [optional]

          • These are the potential additional parameters useful to the model for performing the prediction.

      • **kwargsdict [optional]

          • These are the potential keyword parameters useful to the model for performing the prediction.

        Raises
      • NotImplementedError

          • Raised when not implemeneted in the sub-class.

        predict(self, x, remove_train=True)[source]

        Perform the prediction using a trained model.

        The prediction is preformed over a generic input x and the method should be callable after the training procedure (RecSysModel.train()).

        Parameters
      • xrectorch.samplers.Sampler or scipy.sparse.csr_matrix or

          • torch.Tensor The input for which the prediction has to be computed.

      • *argslist [optional]

          • These are the potential additional parameters useful to the model for performing the prediction.

      • **kwargsdict [optional]

          • These are the potential keyword parameters useful to the model for performing the prediction.

        Raises
      • NotImplementedError

          • Raised when not implemeneted in the sub-class.

        save_model(self, filepath, cur_epoch)[source]

        Save the model to file.

        Parameters
      • filepathstr

          • String representing the path to the file to save the model.

      • *argslist [optional]

          • These are the potential additional parameters useful to the model for performing the prediction.

      • **kwargsdict [optional]

          • These are the potential keyword parameters useful to the model for performing the prediction.

        Raises
      • NotImplementedError

          • Raised when not implemeneted in the sub-class.

        train(self, train_data, valid_data=None, valid_metric=None, valid_func=ValidFunc(fun='evaluate', params=None), num_epochs=1000, g_steps=5, d_steps=5, verbose=1)[source]

        Training procedure of CFGAN.

        The training works in an alternate way between generator and discriminator. The number of training batches in each epochs are g_steps and d_steps, respectively.

        Parameters
      • train_datasamplers.CFGAN_TrainingSampler

          • The sampler object that load the training set in mini-batches.

      • valid_datasamplers.DataSampler [optional]

          • The sampler object that load the validation set in mini-batches, by default None. If the model does not have any validation procedure set this parameter to None.

      • valid_metricstr [optional]

          • The metric used during the validation to select the best model, by default None. If valid_data is not None then valid_metric must be not None. To see the valid strings for the metric please see the module metrics.

      • valid_funcevaluation.ValidFunc [optional]

          • The validation function, by default a standard validation procedure, i.e., evaluation.evaluate().

      • num_epochsint [optional]

          • Number of training epochs, by default 1000.

      • g_stepsint [optional]

          • Number of steps for a generator epoch, by default 5.

      • d_stepsint [optional]

          • Number of steps for a discriminator epoch, by default 5.

      • verboseint [optional]

          • The level of verbosity of the logging, by default 1. The level can have any integer value greater than 0. However, after reaching a maximum (that depends on the size of the training set) verbosity higher values will not have any effect.

        train_disc_batch(self, batch)[source]

        Training on a single batch for the discriminator.

        Parameters
      • batchtorch.Tensor

          • The current batch.

        Returns
      • float

          • The loss incurred in the batch by the discriminator.

        train_gen_batch(self, batch)[source]

        Training on a single batch for the generator.

        Parameters
      • batchtorch.Tensor

          • The current batch.

        Returns
      • float

          • The loss incurred in the current batch by the generator.

        class rectorch.models.ADMM_Slim(lambda1=5.0, lambda2=1000.0, rho=100000.0, nn_constr=True, l1_penalty=True, item_bias=False)[source]

        Bases: rectorch.models.RecSysModel

        ADMM SLIM: Sparse Recommendations for Many Users.

        ADMM SLIM [R767f3a747c1d-ADMMS] is a model similar to SLIM [R767f3a747c1d-SLIM] in which the objective function is solved using Alternating Directions Method of Multipliers (ADMM). In particular, given the rating matrix \(\mathbf{X} \in \mathbb{R}^{n \times m}\) with n users and m items, ADMM SLIM aims at solving the following optimization problem:

        \(\min_{B,C,\Gamma} \frac{1}{2}\|X-X B\|_{F}^{2}+\frac{\lambda_{2}}{2} \cdot\|B\|_{F}^{2}+\ \lambda_{1} \cdot\|C\|_{1} +\ \langle\Gamma, B-C\rangle_{F}+\frac{\rho}{2} \cdot\|B-C\|_{F}^{2}\)

        with \(\textrm{diag}(B)=0\), \(\Gamma \in \mathbb{R}^{m \times m}\), and the entry of C are all greater or equal than 0.

        The prediction for a user-item pair (u,j) is then computed by \(S_{u j}=\mathbf{X}_{u,:} \cdot \mathbf{B}_{:, j}\).

        Parameters
      • lambda1float [optional]

          • Elastic net regularization hyper-parameters \(\lambda_1\), by default 5.

      • lambda2float [optional]

          • Elastic net regularization hyper-parameters \(\lambda_2\), by default 1e3.

      • rhofloat [optional]

          • The penalty hyper-parameter \(\rho>0\) that applies to \(\|B-C\|^2_F\), by default 1e5.

      • nn_constrbool [optional]

          • Whether to keep the non-negativity constraint, by default True.

      • l1_penaltybool [optional]

          • Whether to keep the L1 penalty, by default True. When l1_penalty = False then is like to set \(\lambda_1 = 0\).

      • item_biasbool [optional]

          • Whether to model the item biases, by default False. When item_bias = True then the scoring function for the user-item pair (u,i) becomes: \(S_{ui}=(\mathbf{X}_{u,:} - \mathbf{b})\cdot \mathbf{B}_{:, i} + \mathbf{b}_i\).

        References

        R767f3a747c1d-ADMMS

        Harald Steck, Maria Dimakopoulou, Nickolai Riabov, and Tony Jebara. 2020. ADMM SLIM: Sparse Recommendations for Many Users. In Proceedings of the 13th International Conference on Web Search and Data Mining (WSDM ’20). Association for Computing Machinery, New York, NY, USA, 555–563. DOI: https://doi.org/10.1145/3336191.3371774

        R767f3a747c1d-SLIM

        X. Ning and G. Karypis. 2011. SLIM: Sparse Linear Methods for Top-N Recommender Systems. In Proceedings of the IEEE 11th International Conference on Data Mining, Vancouver,BC, 2011, pp. 497-506. DOI: https://doi.org/10.1109/ICDM.2011.134.

        Attributes
      • See the parameters’ section.
          • load_model(self, filepath)[source]

          Load the model from file.

          Parameters
        • filepathstr

            • String representing the path to the file where the model is saved.

        • *argslist [optional]

            • These are the potential additional parameters useful to the model for performing the prediction.

        • **kwargsdict [optional]

            • These are the potential keyword parameters useful to the model for performing the prediction.

          Raises
        • NotImplementedError

            • Raised when not implemeneted in the sub-class.

          predict(self, ids_te_users, test_tr, remove_train=True)[source]

          Perform the prediction using a trained model.

          The prediction is preformed over a generic input x and the method should be callable after the training procedure (RecSysModel.train()).

          Parameters
        • xrectorch.samplers.Sampler or scipy.sparse.csr_matrix or

            • torch.Tensor The input for which the prediction has to be computed.

        • *argslist [optional]

            • These are the potential additional parameters useful to the model for performing the prediction.

        • **kwargsdict [optional]

            • These are the potential keyword parameters useful to the model for performing the prediction.

          Raises
        • NotImplementedError

            • Raised when not implemeneted in the sub-class.

          save_model(self, filepath)[source]

          Save the model to file.

          Parameters
        • filepathstr

            • String representing the path to the file to save the model.

        • *argslist [optional]

            • These are the potential additional parameters useful to the model for performing the prediction.

        • **kwargsdict [optional]

            • These are the potential keyword parameters useful to the model for performing the prediction.

          Raises
        • NotImplementedError

            • Raised when not implemeneted in the sub-class.

          train(self, train_data, num_iter=50, verbose=1)[source]

          Training of ADMM SLIM.

          The training procedure of ADMM SLIM highly depends on the setting of the hyper-parameters. By setting them in specific ways it is possible to define different variants of the algorithm. That are:

          1. (Vanilla) ADMM SLIM - \(\lambda_1, \lambda_2, \rho>0\), item_bias = False, and both nn_constr and l1_penalty set to True;

          2. ADMM SLIM w/o non-negativity constraint over C - nn_constr = False and l1_penalty set to True;

          3. ADMM SLIM w/o the L1 penalty - l1_penalty = False and nn_constr set to True;

          4. ADMM SLIM w/o L1 penalty and non-negativity constraint: nn_constr = l1_penalty = False.

          All these variants can also be combined with the inclusion of the item biases by setting item_bias to True.

          Parameters
        • train_datascipy.sparse.csr_matrix

            • The training data.

        • num_iterint [optional]

            • Maximum number of training iterations, by default 50. This argument has no effect if both nn_constr and l1_penalty are set to False.

        • verboseint [optional]

            • The level of verbosity of the logging, by default 1. The level can have any integer value greater than 0. However, after reaching a maximum (that depends on the size of the training set) verbosity higher values will not have any effect.

          class rectorch.models.SVAE(svae_net, beta=1.0, anneal_steps=0, learning_rate=0.001)[source]

          Bases: rectorch.models.MultiVAE

          Sequential Variational Autoencoders for Collaborative Filtering.

          UNDOCUMENTED [R7eeeb4f18906-SVAE]

          Parameters
        • mvae_nettorch.nn.Module

            • The variational autoencoder neural network.

        • betafloat [optional]

            • The \(\beta\) hyper-parameter of Multi-VAE. When anneal_steps > 0 then this value is the value to anneal starting from 0, otherwise the beta will be fixed to the given value for the duration of the training. By default 1.

        • anneal_stepsint [optional]

            • Number of annealing step for reaching the target value beta, by default 0. 0 means that no annealing will be performed and the regularization parameter will be fixed to beta.

        • learning_ratefloat [optional]

            • The learning rate for the optimizer, by default 1e-3.

          References

          R7eeeb4f18906-SVAE

          Noveen Sachdeva, Giuseppe Manco, Ettore Ritacco, and Vikram Pudi. 2019. Sequential Variational Autoencoders for Collaborative Filtering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (WSDM ‘19). Association for Computing Machinery, New York, NY, USA, 600–608. DOI: https://doi.org/10.1145/3289600.3291007

          loss_function(self, recon_x, x, mu, logvar, beta=1.0)[source]

          VAE for collaborative filtering loss function.

          MultiVAE assumes a multinomial distribution over the input and this is reflected in the loss function. The loss is a \(\beta\) ELBO (Evidence Lower BOund) in which the regularization part is weighted by a hyper-parameter \(\beta\). Moreover, as in MultiDAE, the reconstruction loss is based on the multinomial likelihood. Specifically, the loss function of MultiVAE is defined as:

          \(\mathcal{L}_{\beta}(\mathbf{x}_{u} ; \theta, \phi)=\ \mathbb{E}_{q_{\phi}(\mathbf{z}_{u} | \mathbf{x}_{u})}[\log p_{\theta}\ (\mathbf{x}_{u} | \mathbf{z}_{u})]-\beta \cdot \operatorname{KL}(q_{\phi}\ (\mathbf{z}_{u} | \mathbf{x}_{u}) \| p(\mathbf{z}_{u}))\)

          Parameters
        • recon_xtorch.Tensor

            • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over a batch.

        • xtorch.Tensor

            • The input, and hence the target tensor. It is meant to be a batch size input.

        • mutorch.Tensor

            • The mean part of latent space for the given x. Together with logvar represents the representation of the input x before the reparameteriation trick.

        • logvartorch.Tensor

            • The (logarithm of the) variance part of latent space for the given x. Together with mu represents the representation of the input x before the reparameteriation trick.

        • betafloat [optional]

            • The current \(\beta\) regularization hyper-parameter, by default 1.0.

          Returns
        • torch.Tensor

            • Tensor (\(1 \times 1\)) representing the average loss incurred over the input batch.

          predict(self, x, remove_train=True)[source]

          Perform the prediction using a trained Variational Autoencoder.

          Parameters
        • xtorch.Tensor

            • The input batch tensor for which the prediction must be computed.

        • remove_trainbool [optional]

            • Whether to remove the training set from the prediction, by default True. Removing the training items means set their scores to \(-\infty\).

          Returns
        • recon_x, mu, logvartuple
          • recon_xtorch.Tensor

              • The reconstructed input, i.e., the output of the variational autoencoder. It is meant to be the reconstruction over the input batch x.

          • mutorch.Tensor

              • The mean part of latent space for the given x. Together with logvar represents the representation of the input x before the reparameteriation trick.

          • logvartorch.Tensor

              • The (logarithm of the) variance part of latent space for the given x. Together with mu represents the representation of the input x before the reparameteriation trick.