A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects:
1. Knowledge is acquired by the network through a learning process.
2. Interneuron connection strengths known as synaptic weights are used to store the knowledge.

As cited in: Brian D. Ripley (2008) Pattern Recognition and Neural Networks. p.4 - An introduction to neural computing (1990)