accelbrainbase.observabledata._mxnet.restrictedboltzmannmachines package

Submodules

accelbrainbase.observabledata._mxnet.restrictedboltzmannmachines.deep_boltzmann_machines module

class accelbrainbase.observabledata._mxnet.restrictedboltzmannmachines.deep_boltzmann_machines.DeepBoltzmannMachines(computable_loss, rbm_list, initializer=None, optimizer_name='SGD', learning_rate=0.005, learning_attenuate_rate=1.0, attenuate_epoch=50, ctx=gpu(0), **kwargs)

Bases: accelbrainbase.observabledata._mxnet.restricted_boltzmann_machines.RestrictedBoltzmannMachines

Deep Boltzmann Machines(DBM).

As is well known, DBM is composed of layers of RBMs stacked on top of each other(Salakhutdinov, R., & Hinton, G. E. 2009). This model is a structural expansion of Deep Belief Networks(DBN), which is known as one of the earliest models of Deep Learning (Le Roux, N., & Bengio, Y. 2008). Like RBM, DBN places nodes in layers. However, only the uppermost layer is composed of undirected edges, and the other consists of directed edges.

According to graph theory, the structure of RBM corresponds to a complete bipartite graph which is a special kind of bipartite graph where every node in the visible layer is connected to every node in the hidden layer. Based on statistical mechanics and thermodynamics(Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. 1985), the state of this structure can be reflected by the energy function.

In relation to RBM, the Contrastive Divergence(CD) is a method for approximation of the gradients of the log-likelihood(Hinton, G. E. 2002). This algorithm draws a distinction between a positive phase and a negative phase. Conceptually, the positive phase is to the negative phase what waking is to sleeping.

The procedure of this method is similar to Markov Chain Monte Carlo method(MCMC). However, unlike MCMC, the visbile variables to be set first in visible layer is not randomly initialized but the observed data points in training dataset are set to the first visbile variables. And, like Gibbs sampler, drawing samples from hidden variables and visible variables is repeated k times. Empirically (and surprisingly), k is considered to be 1.

Note that this class does not support a Hybrid of imperative and symbolic programming. Only mxnet.ndarray is supported.

References

  • Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive science, 9(1), 147-169.
  • Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural computation, 14(8), 1771-1800.
  • Le Roux, N., & Bengio, Y. (2008). Representational power of restricted Boltzmann machines and deep belief networks. Neural computation, 20(6), 1631-1649.
  • Salakhutdinov, R., & Hinton, G. E. (2009). Deep boltzmann machines. InInternational conference on artificial intelligence and statistics (pp. 448-455).
collect_params(select=None)

Overrided collect_params in mxnet.gluon.HybridBlok.

feature_points_arr

getter for mxnet.ndarray of feature points.

get_feature_points_arr()

getter for mxnet.ndarray of feature points.

get_init_deferred_flag()

getter for bool that means initialization in this class will be deferred or not.

get_rbm_list()

getter

inference(observed_arr)

Inference samples drawn by IteratableData.generate_inferenced_samples().

Parameters:observed_arr – rank-2 Array like or sparse matrix as the observed data points. The shape is: (batch size, feature points)
Returns:mxnet.ndarray of inferenced feature points.
init_deferred_flag

getter for bool that means initialization in this class will be deferred or not.

rbm_list

getter

regularize()

Regularization.

set_init_deferred_flag(value)

setter for bool that means initialization in this class will be deferred or not.

set_rbm_list(value)

setter

set_readonly(value)

setter

Module contents