Perceptron

Next: Naive Bayes Up: Basic Learning Rules Previous: Winnow Contents

A Perceptron update proceeds as follows:

Let ${\cal A}_t = \{i_1, \ldots, i_m \}$ be the set of active features in a given example that are linked to target node , and let be the strength associated with feature in the example.
If the algorithm predicts negative (that is, $\sum_{i \in {\cal A}_{t}} w_{t,i} s_i < \theta_t$ ), and the specified label is positive, the weights of features active in the current example are promoted in an additive fashion:

$\displaystyle \forall i \in {\cal A}_t, w_{t,i} \leftarrow w_{t,i} + \alpha_t s_i$
If the algorithm predicts positive ( $\sum_{i \in { \cal A}_{t}} w_{t,i} s_i \ge \theta_t$ ) and the specified label is negative, the weights of features active in the current example are demoted:

$\displaystyle \forall i \in {\cal A}_t, w_{t,i} \leftarrow w_{t,i} - \alpha_t s_i$
All other weights are unchanged.

Perceptron's sigmoid activation is calculated with the following formula:

$\displaystyle \sigma(\theta, \Omega) = \frac{1}{1 + e^{\theta - \Omega}}$

This is exactly the formula used by [Mitchell, 1997] as a sigmoid function in the context of neural networks.

Next: Naive Bayes Up: Basic Learning Rules Previous: Winnow Contents

Cognitive Computations 2004-08-20