First of all on this page, the manuscript download links are presented to prevent users from unnessecary searching. Consequently, further information is given which may be relevant for the reader: Version information, terms of use and a summary of the manuscript “A Brief Introduction to Neural Networks”.

If you want to place links, please use only this sub page as a target. Do not place links that point directly to the files. Filenames may change without notice, and moreover, it is useful to see this summary page before downloading.

This manuscript relies very much on your feedback to improve it. As you can see from the lots of helpers mentioned in my frontmatter, I really appreciate and make use of feedback I receive from readers. If you have any complaints, bug-fixes, suggestions, or acclamations send emails to me or place a comment in the newly-added discussion section below at the bottom of this page. Be sure you get a response.

There's no official publisher, so you need to be careful with your citation. For now, use this:

David Kriesel, 2007, A Brief Introduction to Neural Networks, available at http://www.dkriesel.com

This reference is, of course, for the english version. Please look at the German translation of this page to find the German reference.

Please always include the URL – it's the only unique identifier to the text (for now)! Note the lack of edition name, which changes with every new edition, and Google Scholar and Citeseer both have trouble with fast-changing editions. If you prefer BibTeX:

@Book{ Kriesel2007NeuralNetworks, 
       author = { David Kriesel }, 
       title =  { A Brief Introduction to Neural Networks },
       year =   { 2007 }, 
       note =   { available at http://www.dkriesel.com } 
     }

From the epsilon edition, the text is licensed under the Creative Commons Attribution-No Derivative Works 3.0 Unported License, except for some little portions of the work licensed under more liberal licenses as mentioned in the frontmatter or throughout the text. Note that this license does not extend to the source files used to produce the document. Those are still mine.

In the epsilon version, the technical base of the manuscript was completely renewed. In the text, there are lots of updates, too.

• New Chapter: Biological Neural Networks
• New layout and design, which makes use of common typography rules. Relayouted type area and spaces.
• New page headers and footers, containing section and chapter titles.
• New, modern fonts. Not that you'll see lots of difference, but they can be read easier.
• “Speaking headlines” througout the entire manuscript boil down the material of the following section to one sharp sentence.
• Clickable links and pdf-contained table of contents throughout the entire manuscript.
• Whole lots of little enhancements and corrections that I was advised to by readers. Thanks a lot, guys! In this context, the list of helpful readers in the preface was extended thoroughly
• Removed the two Table of contents and replaced them by one single TOC of intermediate lenght.
• Several rearrangements, in particular parts of the perceptron chapter (5) were moved towards the learning chapter (4). The Regional and Online Learnable Fields Chapter was moved into the Excursus about Clustering. The Part “Further Views on Learning” was removed, its chapters were rearranged.
• New CC-license, see above
• Cool new cover

A very, very big “thank you” goes to Beate Kuhl again for the translation!!!

• Lots of remarks put in that I got from readers, in particular concerning many tiny inconsistencies
• Enhanced layout, changed some images Layout verbessert, Kleinigkeiten an Abbildungen geändert.
• Some changes made in the overall arrangement
• Translation into the English finished (Thanks to Beate Kuhl!!)

Neural networks are a bio-inspired mechanism of data processing, that enables computers to learn technically similar to a brain and even generalize once solutions to enough problem instances are tought.

The manuscript “A Brief Introduction to Neural Networks” is divided into several parts, that are again split to chapters. The contents of each chapter are summed up in the following.

How to teach a computer? You can either write a rigid program – or you can enable the computer to learn on its own. Living beings don't have any programmer writing a program for developing their skills, which only has to be executed. They learn by themselves – without the initial experience of external knowledge – and thus can solve problems better than any computer today. KaWhat qualities are needed to achieve such a behavior for devices like computers? Can such cognition be adapted from biology? History, development, decline and resurgence of a wide approach to solve problems.

How do biological systems solve problems? How is a system of neurons working? How can we understand its functionality? What are different quantities of neurons able to do? Where in the nervous system are information processed? A short biological overview of the complexity of simple elements of neural information processing followed by some thoughts about their simplification in order to technically adapt them.

Formal definitions and colloquial explanations of the components that realize the technical adaptations of biological neural networks. Initial descriptions of how to combine these components to a neural network.

Approaches and thoughts of how to teach machines. Should neural networks be corrected? Should they only be encouraged? Or should they even learn without any help? Thoughts about what we want to change during the learning procedure and how we will change it, about the measurement of errors and when we have learned enough.

A classic among the neural networks. If we talk about a neural network, then in the majority of cases we speak about a percepton or a variation of it. Perceptrons are multi-layer networks without recurrence and with fixed input and output layers. Description of a perceptron, its limits and extensions that should avoid the limitations. Derivation of learning procedures and discussion about their problems.

RBF networks approximate functions by stretching and compressing Gaussians and then summing them spatially shifted. Description of their functions and their learning process. Comparison with multi-layer perceptrons.

Some thoughts about networks with internal states. Learning approaches using such networks, overview of their dynamics.

In a magnetic field, each particle applies a force to any other particle so that all particles adjust their movements in the energetically most favorable way. This natural mechanism is copied to adjust noisy inputs in order to match their real models.

Learning vector quantization is a learning procedure with the aim to reproduce the vector training sets divided in predefined classes as good as possible by using a few representative vectors. If this has been managed, vectors which were unkown until then could easily be assigned to one of these classes.

A paradigm of unsupervised learning neural networks, which maps an input space by its fixed topology and thus independently looks for simililarities. Function, learning procedure, variations and neural gas.

An ART network in its original form shall classify binary input vectors, i.e. to assign them to a 1-out-of-n output. Simultaneously, the so far unclassified patterns shall be recognized and assigned to a new class.

In Grimm's dictionary the extinct German word “Kluster” is described by “was dicht und dick zusammensitzet (a thick and dense group of sth.)”. In static cluster analysis, the formation of groups within point clouds is explored. Introduction of some procedures, comparison of their advantages and disadvantages. Discussion of an adaptive clustering method based on neural networks. A regional and online learnable field models from a point cloud, possibly with a lot of points, a comparatively small set of neurons being representative for the point cloud.

Discussion of an application of neural networks: A look ahead into the future of time series.

What if there were no training examples but it would nevertheless be possible to evaluate how good we have learned to solve a problem? et us regard a learning paradigm that is situated between supervised and unsupervised learning.