pydbm.rnn.lstmmodel package

Submodules

pydbm.rnn.lstmmodel.attention_lstm_model module

class pydbm.rnn.lstmmodel.attention_lstm_model.AttentionLSTMModel

Bases: pydbm.rnn.lstm_model.LSTMModel

Attention model of Long short term memory(LSTM) networks.

The function of this class is to behave as decoder of Encoder/Decoder. This decoder model has a mechanism of attention so as to decides parts of the source sequences to pay attention to. This mechanism enalbes the encoder to reduce the burden of having to encode all information in the source sequence into a fixed-length context vector. With this new approach the information can be spread throughout the sequence of annotations, which can be selectively retrieved by the decoder accordingly.

References

  • Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998-6008).
back_propagation()

Back propagation.

Parameters:
  • pred_arr – np.ndarray of predicted data points.
  • delta_output_arr – Delta.
Returns:

Tuple data. - np.ndarray of Delta, - list of gradations
context_backward()
context_forward()
forward_propagation()

Forward propagation.

Parameters:batch_observed_arr – Array like or sparse matrix as the observed data points.
Returns:Array like or sparse matrix as the predicted data points.
optimize()

Optimization.

Parameters:
  • grads_list – list of graduations.
  • learning_rate – Learning rate.
  • epoch – Now epoch.
output_back_propagate()

Back propagation in output layer.

Parameters:
  • pred_arr – np.ndarray of predicted data points.
  • delta_output_arr – Delta.
Returns:

Tuple data. - np.ndarray of Delta, - list of gradations.
output_forward_propagate()

Forward propagation in output layer.

Parameters:pred_arr – np.ndarray of predicted data points.
Returns:np.ndarray of propagated data points.
weight_backward()
weight_forward()

pydbm.rnn.lstmmodel.conv_lstm_model module

class pydbm.rnn.lstmmodel.conv_lstm_model.ConvLSTMModel

Bases: pydbm.rnn.interface.reconstructable_model.ReconstructableModel

Convolutional LSTM(ConvLSTM).

Convolutional LSTM(ConvLSTM)(Xingjian, S. H. I. et al., 2015), which is a model that structurally couples convolution operators to LSTM networks, can be utilized as components in constructing the Encoder/Decoder. The ConvLSTM is suitable for spatio-temporal data due to its inherent convolutional structure.

References

back_propagation

Back propagation.

Parameters:
  • pred_arr – np.ndarray of predicted data points.
  • delta_output_arr – Delta.
Returns:

Tuple data. - np.ndarray of Delta, - list of gradations.
forgot_conv

getter

forward_propagation

Forward propagation.

Parameters:batch_observed_arr – Array like or sparse matrix as the observed data points.
Returns:Array like or sparse matrix as the predicted data points.
get_feature_points

Extract the activities in hidden layer and reset it, considering this method will be called per one cycle in instances of time-series.

Returns:The list of array like or sparse matrix of feature points or virtual visible observed data points.
get_forgot_conv

getter

get_given_conv

getter

get_graph

getter

get_input_conv

getter

get_opt_params

getter

get_output_conv

getter

get_verificatable_result

getter

given_conv

getter

graph

getter

hidden_back_propagate

Back propagation in hidden layer.

Parameters:delta_output_arr – Delta.
Returns:Tuple data. - np.ndarray of Delta in observed data points, - np.ndarray of Delta in hidden units, - list of gradations
hidden_forward_propagate

Forward propagation in LSTM gate.

Parameters:observed_arr – np.ndarray of observed data points.
Returns:Predicted data points.
inference

Inference the feature points to reconstruct the time-series.

Override.

Parameters:
  • observed_arr – Array like or sparse matrix as the observed data points.
  • hidden_activity_arr – Array like or sparse matrix as the state in hidden layer.
  • cec_activity_arr – Array like or sparse matrix as the state in RNN.
Returns:

Tuple data. - Array like or sparse matrix of reconstructed instances of time-series, - Array like or sparse matrix of the state in hidden layer, - Array like or sparse matrix of the state in RNN.
input_conv

getter

learn

Learn the observed data points for vector representation of the input time-series.

Override.

Parameters:
  • observed_arr – Array like or sparse matrix as the observed data points.
  • target_arr – Array like or sparse matrix as the target data points. To learn as Auto-encoder, this value must be None or equivalent to observed_arr.
learn_generated

Learn features generated by FeatureGenerator.

Parameters:feature_generator – is-a FeatureGenerator.
load_pre_learned_params

Load pre-learned parameters.

Parameters:
  • dir_name – Path of dir. If None, the file is saved in the current directory.
  • file_name – File name.
lstm_backward

Back propagation in LSTM gate.

Parameters:
  • delta_hidden_arr – Delta from output layer to hidden layer.
  • delta_cec_arr – Delta in LSTM gate.
  • cycle – Now cycle or time.
Returns:

Tuple data. - Delta from hidden layer to input layer, - Delta in hidden layer at previous time, - Delta in LSTM gate at previous time, - list of gradations.
opt_params

getter

optimize

Optimization.

Parameters:
  • grads_list – list of graduations.
  • learning_rate – Learning rate.
  • epoch – Now epoch.
output_back_propagate

Back propagation in output layer.

Parameters:
  • pred_arr – np.ndarray of predicted data points.
  • delta_output_arr – Delta.
Returns:

Tuple data. - np.ndarray of Delta, - list of gradations.
output_conv

getter

output_forward_propagate

Forward propagation in output layer.

Parameters:pred_arr – np.ndarray of predicted data points.
Returns:np.ndarray of propagated data points.
save_pre_learned_params

Save pre-learned parameters.

Parameters:
  • dir_name – Path of dir. If None, the file is saved in the current directory.
  • file_name – File name.
set_forgot_conv

setter

set_given_conv

setter

set_graph

setter

set_input_conv

setter

set_opt_params

setter

set_output_conv

setter

set_verificatable_result

setter

verificatable_result

getter

Module contents