pydbm.activation.signfunction package¶
Submodules¶
pydbm.activation.signfunction.deterministic_binary_neurons module¶
-
class
pydbm.activation.signfunction.deterministic_binary_neurons.DeterministicBinaryNeurons¶ Bases:
pydbm.activation.sign_function.SignFunctionDeterministic Binary Neurons as a Sign function.
The binary neurons are neurons that output binary valued predictions as a sign-like function. The function of deterministic binary neurons is to act like neurons with hard thresholding functions as their activation functions.
This class entries a sigmoid-adjusted Straight-Through Estimator for backpropagation. In the backward pass, the Straight-Through Estimator simply treats the binary neurons as identify functions and ignores their gradients. A variant of the Straight-Through Estimator is the sigmoid-adjusted Straight-Through Estimator, which multiplies the gradients in the backward pass by the derivative of the sigmoid function.
References
- Chung, J., Ahn, S., & Bengio, Y. (2016). Hierarchical multiscale recurrent neural networks. arXiv preprint arXiv:1609.01704.
- Dong, H. W., & Yang, Y. H. (2018). Convolutional generative adversarial networks with binary neurons for polyphonic music generation. arXiv preprint arXiv:1804.09399.
- Oza, M., Vaghela, H., & Srivastava, K. (2019). Progressive Generative Adversarial Binary Networks for Music Generation. arXiv preprint arXiv:1903.04722.
-
activate¶ Activate and extract feature points in forward propagation.
Parameters: np.ndarray of observed data points. (x) – Returns: np.ndarray of the activated feature points.
-
backward¶ Back propagation but not operate the activation.
Parameters: y – np.ndarray of delta. Returns: The result.
-
derivative¶ Derivative and extract delta in back propagation.
Parameters: y – np.ndarray of delta. Returns: np.ndarray of delta.
-
forward¶ Forward propagation but not retain the activation.
Parameters: np.ndarray of observed data points. (x) – Returns: The result.
-
get_zero_value¶ getter for the value of the Heaviside step function when input is 0.
-
set_zero_value¶ setter for the value of the Heaviside step function when input is 0.
-
zero_value¶ getter for the value of the Heaviside step function when input is 0.
pydbm.activation.signfunction.stochastic_binary_neurons module¶
-
class
pydbm.activation.signfunction.stochastic_binary_neurons.StochasticBinaryNeurons¶ Bases:
pydbm.activation.sign_function.SignFunctionStochastic Binary Neurons as a Sign function.
The binary neurons are neurons that output binary valued predictions as a sign-like function. The stochastic binary neurons, in contrast, binarize an input according to a probability.
This class entries a sigmoid-adjusted Straight-Through Estimator for backpropagation. In the backward pass, the Straight-Through Estimator simply treats the binary neurons as identify functions and ignores their gradients. A variant of the Straight-Through Estimator is the sigmoid-adjusted Straight-Through Estimator, which multiplies the gradients in the backward pass by the derivative of the sigmoid function.
References
- Chung, J., Ahn, S., & Bengio, Y. (2016). Hierarchical multiscale recurrent neural networks. arXiv preprint arXiv:1609.01704.
- Dong, H. W., & Yang, Y. H. (2018). Convolutional generative adversarial networks with binary neurons for polyphonic music generation. arXiv preprint arXiv:1804.09399.
- Oza, M., Vaghela, H., & Srivastava, K. (2019). Progressive Generative Adversarial Binary Networks for Music Generation. arXiv preprint arXiv:1903.04722.
-
activate¶ Activate and extract feature points in forward propagation.
Parameters: np.ndarray of observed data points. (x) – Returns: np.ndarray of the activated feature points.
-
backward¶ Back propagation but not operate the activation.
Parameters: y – np.ndarray of delta. Returns: The result.
-
derivative¶ Derivative and extract delta in back propagation.
Parameters: y – np.ndarray of delta. Returns: np.ndarray of delta.
-
forward¶ Forward propagation but not retain the activation.
Parameters: np.ndarray of observed data points. (x) – Returns: The result.
-
get_zero_value¶ getter for the value of the Heaviside step function when input is 0.
-
set_zero_value¶ setter for the value of the Heaviside step function when input is 0.
-
zero_value¶ getter for the value of the Heaviside step function when input is 0.