# pydbm.activation.signfunction package¶

## pydbm.activation.signfunction.deterministic_binary_neurons module¶

class pydbm.activation.signfunction.deterministic_binary_neurons.DeterministicBinaryNeurons

Deterministic Binary Neurons as a Sign function.

The binary neurons are neurons that output binary valued predictions as a sign-like function. The function of deterministic binary neurons is to act like neurons with hard thresholding functions as their activation functions.

This class entries a sigmoid-adjusted Straight-Through Estimator for backpropagation. In the backward pass, the Straight-Through Estimator simply treats the binary neurons as identify functions and ignores their gradients. A variant of the Straight-Through Estimator is the sigmoid-adjusted Straight-Through Estimator, which multiplies the gradients in the backward pass by the derivative of the sigmoid function.

References

• Chung, J., Ahn, S., & Bengio, Y. (2016). Hierarchical multiscale recurrent neural networks. arXiv preprint arXiv:1609.01704.
• Dong, H. W., & Yang, Y. H. (2018). Convolutional generative adversarial networks with binary neurons for polyphonic music generation. arXiv preprint arXiv:1804.09399.
• Oza, M., Vaghela, H., & Srivastava, K. (2019). Progressive Generative Adversarial Binary Networks for Music Generation. arXiv preprint arXiv:1903.04722.
activate

Activate and extract feature points in forward propagation.

Parameters: np.ndarray of observed data points. (x) – np.ndarray of the activated feature points.
backward

Back propagation but not operate the activation.

Parameters: y – np.ndarray of delta. The result.
derivative

Derivative and extract delta in back propagation.

Parameters: y – np.ndarray of delta. np.ndarray of delta.
forward

Forward propagation but not retain the activation.

Parameters: np.ndarray of observed data points. (x) – The result.
get_zero_value

getter for the value of the Heaviside step function when input is 0.

set_zero_value

setter for the value of the Heaviside step function when input is 0.

zero_value

getter for the value of the Heaviside step function when input is 0.

## pydbm.activation.signfunction.stochastic_binary_neurons module¶

class pydbm.activation.signfunction.stochastic_binary_neurons.StochasticBinaryNeurons

Stochastic Binary Neurons as a Sign function.

The binary neurons are neurons that output binary valued predictions as a sign-like function. The stochastic binary neurons, in contrast, binarize an input according to a probability.

This class entries a sigmoid-adjusted Straight-Through Estimator for backpropagation. In the backward pass, the Straight-Through Estimator simply treats the binary neurons as identify functions and ignores their gradients. A variant of the Straight-Through Estimator is the sigmoid-adjusted Straight-Through Estimator, which multiplies the gradients in the backward pass by the derivative of the sigmoid function.

References

• Chung, J., Ahn, S., & Bengio, Y. (2016). Hierarchical multiscale recurrent neural networks. arXiv preprint arXiv:1609.01704.
• Dong, H. W., & Yang, Y. H. (2018). Convolutional generative adversarial networks with binary neurons for polyphonic music generation. arXiv preprint arXiv:1804.09399.
• Oza, M., Vaghela, H., & Srivastava, K. (2019). Progressive Generative Adversarial Binary Networks for Music Generation. arXiv preprint arXiv:1903.04722.
activate

Activate and extract feature points in forward propagation.

Parameters: np.ndarray of observed data points. (x) – np.ndarray of the activated feature points.
backward

Back propagation but not operate the activation.

Parameters: y – np.ndarray of delta. The result.
derivative

Derivative and extract delta in back propagation.

Parameters: y – np.ndarray of delta. np.ndarray of delta.
forward

Forward propagation but not retain the activation.

Parameters: np.ndarray of observed data points. (x) – The result.
get_zero_value

getter for the value of the Heaviside step function when input is 0.

set_zero_value

setter for the value of the Heaviside step function when input is 0.

zero_value

getter for the value of the Heaviside step function when input is 0.