What is a perceptron?

Perceptrons are known from the back propagation (BP) training algorithm. Usually when we think about a neural network architecture we think of the architecture itself. Well... there is nothing wrong with that, however, what we should know is that the architecture itself is not the main thing here. We can even say that it mainly a way to visualize the computations in the learning algorithm.

So, what is a perceptron, and what does it correspond to in the brain? More on this topic later.