Hardware
Computers are made of electronic components which were also known as radio details because before the advent of microprocessors computers were made of the same details as radio, TV, and other consumer electronics. Elementary components of the brain are neurons – specialized nerve cells which can generate electric pulses and conduct them to long distances of tens of centimetres. There are different types of radio details: resistors, capacitors, transistors, and a few others. Likewise, there are a few (of the order of 10) different types of neurons which may be repeatedly encountered in the different parts of the nervous system. The most substantial difference between them is that some are excitatory, others – inhibitory. That is, firing of the first neuron may force the second to fire too, or suppress its background firing rate instead. Neurons are not the only cells of the brain. There is also glia (which serves as damper for neurons and insulator for long wires – nerves) and blood vessels which are indeed a special type of muscles.
Computer is a device that processes information. How is information represented in the brain? We can't determine immediately how our ideas are represented, but we can draw conclusions watching what happens when they come out to periphery and convert themselves into physical actions. It was definitely established that muscular tension depends upon the average firing rate in the nerve that ends up on this muscle. Hence, we can suppose that a single spike of a single neuron in the central nervous system doesn't matter and our ideas are encoded by the pulse activity averaged over a group of adjacent cells (called a cluster) and a certain period of time. The typical number of elements in such clusters is of the order of 1000. As to time parameters, the duration of one spike is approximately 1 millisecond so the firing rate of 1 neuron can't be more than 1 kilohertz. Multiplying it by the number of elements in the cluster, we get 1 megahertz, but keep in mind that the reaction time still can't be less than 1 millisecond because spikes are not rectangular.
At this time you might realize the shocking truth: our brain is so different from our computers that it is an analog (more exactly digital-analog) device at all. Meanwhile there is something that unites them. It is very symbolical that computer programs and musical records may be stored on the same type of media such as optical disks.
Anatomically, the brain consists of several parts which may be clearly distinguished and reproduce themselves in all humans. Their cell structure is different from the adjacent regions or they simply are visible from the surface. The inner space may be of two types – gray and white matter. The former is composed of cell bodies, the latter – of nerve fibers. Different parts of the brain are heavily interconnected. This supports the hypothesis that regions which look different are functionally different as well. All in all, anatomy distinguishes some couple of dozen different parts, but how to arrange them into a functionally meaningful construct?
Basic principles of a live neurocomputer are different from the Von Neumann architecture. Computer operative memory changes data by an instruction (the differential principle) and keeps data while power is on. Regeneration is a separate unconditional process. In live neural net, dynamic memory is a pattern of neural activity which should be explicitly supported by the system of nonspecific activation or by reverberation. In the second case, circulation of activity is also controlled by the nonspecific system. That is, in a computer, instructions are quick and their results remain forever. In a live neurocomputer actions are lengthy and continue while the activation signal remains. Retaining results also requires continuing activation.
Neurocomputing may be studied by purely mathematical methods. We can take 2D (or 3D) image as a main data unit, take associative instead of linear memory, and design a completely different computational model. The Von Neumann processor retrieves data from memory by the address of a memory cell. Associative memory uses keys instead. Also, a single neural net usually keeps multiple images which are superimposed in distributed storage. There are 2 types of such memory: autoassociative and heteroassociative. In the first case, the goal is just to memorize many images, then to recall one of them using some hints. In the second – associations between images of different types are remembered. This may be used to implement stimulus – reaction or event – handler pairs.
Two types of continuous computation are visible right away. In the second case, the system is placed into real-world conditions where external events come permanently so some reactions will be generated permanently too. In the first, we can use an image retrieved from autoassociative memory as a key for the next operation. This implements "free thinking" or "flight of ideas".
A computer comes with a ready set of instructions. The set of elementary, hardware-supported actions for a neurocomputer is smaller. This resembles the situation with fonts. The first typewriters had fixed sets. Then, they were replaced by graphics and fonts are usually generated by software now.
Science doesn't use the term software in application to living systems. Researchers study behavior instead. There is even a separate branch called behaviorism. It regards the nervous system as a black box and tries to formulate laws which link its inputs and outputs. The goal of brain research is seemingly to establish how different structures participate in generation of complex behavior. The task turned out to be tricky. The primitive approach is to determine correspondence between elementary actions and various parts of the nervous system. This encountered resistance from the opposite group which claimed that it is impossible and any function is equally distributed over the brain as a whole. For a computer engineer, the solution is obvious. Mapping is possible, but it is internal rather than external actions which should be mapped. Such as memory read/write operations.
The brain as a whole may be best approximated as a finite-state automaton, but this approach has one problem. Neural activity is highly dynamic and requires energy consumption. If you change the state and relax, it will always slip into the zero state. This issue is resolved using a specific architecture of neural nets. Karl Pribram in his "Languages of the brain" highlighted that many connections in the nervous system are reciprocal (bidirectional). If the biofeedback is negative, this serves for stabilization. If it is positive, this will create a generator which can maintain an activity once it was launched. As a result, you may look at an object, then close your eyes, but the image will remain and you even will be able to examine its details.
Abstract neurocomputer
Our computers are based on a Turing machine. Its principle is very simple. Complexity of computers comes from software, not from hardware. The same should be true for the human brain. The problem is that there are no tape and read/write head inside the human scull. Then, what is the prototype for the Turing model? Thorough consideration reveals that he formalized work of a human which uses some external storage such as paper. This approach resembles what is known in neuroscience as behaviorism. It doesn't try to penetrate into the head, regards the brain as a black box with inputs and outputs. What we need now – to describe how this box operates inside.
The first hint comes already from the input signal. Turing supposed that this is text while for the brain it is video. Speech and written textual input emerge only later in biological evolution and human civilization. Neurophysiological study shows that topology of an input image is retained well into the deep parts of visual analyzer so we can conclude that the format of 2D images is the main data format of brain hardware. The next principle is determined by the type of memory used. For any processor, memory input/output is one of the most important operations simply because a processor should process some data and data is stored in memory. The Von Neumann processor uses linear memory, that is a sequence of bytes which are accessed by their address. The human brain has associative memory instead. An elementary block uses a 2d (or 3D) array (millions) of neurons where information is represented by the pattern of activity. The image may be retrieved only as a whole, but this storage is still efficient because a single block can retain many different images. The method of retrieval uses a key which is also some image.
These two principles – a 2D image instead of a byte and associative instead of linear memory may be used as an axiomatic foundation of mathematical neurocomputing. The science may theoretically explore all the possible methods of data processing and all the possible constructs of neuro-machines. Meanwhile the brain is a ready working solution optimized by the nature so we can add further details from data gathered by biological experiments. The next striking difference between common computers and a neurocomputer is the absence of the clock generator. A live neurocomputer is asynchronous. Brain rhythms are well described, but alpha-rhythm of the visual cortex emerges when eyes are closed. Instead, activation is manifested by desynchronization. This creates a major problem. At each step of its operation, a computer should decide what to do next. A neurocomputer should decide when in addition. On the other hand, this feature creates additional flexibility and it is used. Human computations are based on insight. It is well known as a source for great discoveries, but indeed is used on routine basis in tens, maybe hundreds per day. The principle of a computer: take the next instruction from memory at the next pulse of a clock generator. In a neurocomputer, associative memory provides an appropriate idea at an appropriate moment. This sounds like the Holy Grail, but unfortunately this formula contains a lot of uncertainty. More specifically, insight is: 1. generated by hardware that is has no psychological explanation, 2. heavily dependent on previous experience. There are two types of insight: sensory and motor, that is related to input (perception) and output (action). These images are generated in different parts of the neocortex, but subjectively we feel them similarly because all the neocortex has almost the same structure like computer memory. Insights of the first type are especially interesting because they have an explanation from the theory of information. Visual input conveys digital-analog data which according to Shannon's formula contains a virtually infinite amount of information. The nervous system is simply unable to process it all so only some portions are captured when appropriate.
How are insights generated? This is related to another problem. The brain has memory for sure, but does it have a processor?
Theory of neurocomputing
Hardware and software are just parts of another item – a whole computational system and it is obvious that a computer based on the Von Neumann architecture and the human brain are very different. Paradox is that computers were thoroughly developed for decades and we know them in details while evolution of the brain spans millions of years, we use it permanently, but have no clues about its operation. Substantial efforts were made during 19 and 20 centuries so as to fill this gap. Let's try to formulate explicitly the most basic principles of neurocomputing. First, we need to determine our goal. What is a neurocomputer? Any construct created of neural nets? Their range would be as wide as diversity of live nervous systems. Let's confine our interest to the human brain, but keep in mind that it isn't the top of perfectness. Maybe the brain of some animals is better in some aspect. Maybe there are better solutions unknown in the nature. The prototype is only a hint for the theory.
Let's formulate the answer at the very beginning, then explain it in details. Thorough consideration shows that the following 3 concepts are workable for both types of computing, but they have different implementation. Moreover, distribution of overall computing between memory, a processor, and software is different. A neurocomputer uses associative memory implemented as blocks of homogeneous neural nets. Even in the simplest form with 2 layers, they are already capable of some processing such as branching or simple arithmetic (addition and subtraction). That's it. We have processing without a processor. Add the difference in data representation. Computer memory stores data immediately. In neural nets, data is an image represented as a pattern of neural activity, but memory is kept in modifiable synapses. Nets don't store images immediately. They store the ability to generate particular images in response to particular keys. This entails another difference. In a computer, data is loaded from memory into a processor, undergo modification, then is stored back. In a neurocomputer, images are processed immediately in the memory. Elementary processor instructions are replaced by hard-wired ability of a particular block of memory to perform particular image transformations. This ability is defined by local interconnections within the same layer or between different layers of the same net. In a computer it corresponds to some procedure or algorithm which implements a particular method of data processing. On the other hand, each homogeneous net working as heteroassociative memory keeps associations between input and output images. This corresponds to rules of traditional programming, only blocks of those rules are kept in different, genetically predetermined hardware locations. You see that in a neurocomputer, the main paradigm is rule-based and types of rules are predetermined at hardware level.
So does a neurocomputer have a processor? Seemingly, yes, but it is striped of many functions known for its counterpart in a computer. Basically, if we regard the neocortex as the memory, the role of a processor will be to activate or suppress different areas and links between them. A good example is attention. That is, a processor of a neurocomputer accepts various signals from outside and inside the brain, but its output is just internal "turn on" or "turn off". Meanwhile there is something else. Software may be further subdivided into applications, system-level, and firmware (hardware emulation). While the first part is associated with the neocortex and constitutes various externally visible human abilities, the last is stored in a processor. Here we encounter an interesting turn. We have already seen that the memory of a neurocomputer is capable of processing. On the other hand, its processor is made of similar blocks of associative memory. Only now processing is regarded as a main function while memory provides storage for system programming.
Subcortical nuclei of the brain are well described both anatomically and functionally. Many of them provide low-level control which is unrelated to the psyche and behavior. Others are a part of motor and sensory systems. For example, the thalamus is a major relay station which conducts signals from sensory organs to the neocortex. It is the remainder that may be regarded as a candidate to the role of a processor. Two main parts may be named: the basal ganglia and the limbic system. Functional study of the basal ganglia hints that this is also memory indeed. Seeking analogy with computers, one can say that the basal ganglia are BIOS – the memory which contains the most basic subroutines. So the only part remains – the limbic system and its blocks are well connected into a functionally complete unit. Only not all of this complex may be regarded as a CPU. The limbic system is usually associated with motivation and emotions. Human motivation is subdivided into 2 absolutely different types which come from different parts of the brain. High-level goals and long-term plans are generated in the frontal neocortex, while biological desires such as hunger – in the hypothalamus. If we put aside such blocks, 2 major structures remain – the hippocampus and the amygdala.
Let's try and figure out how this processor operates in general.
A Turing machine is claimed to be universal. The human brain is even more universal. It can work in the completely analog mode when a person monitors some object and follows its movements. The first thing to do is to determine the class of tasks which the brain processor is used for. The brain as a whole is a regulator so if everything is normal, it may stay in the state of idle run. Despite not sleeping, it may do nothing special, just perform random actions without particular use. An alternative is goal-oriented behavior. That's what is interesting for us. That's when activity becomes highly structured and the processor operates at full power. The typical example is the task to reach some place walking in the city. Other tasks may be described by analogy. Consider manufacturing. The goal is to assemble some product from parts. Performed operations are separate steps or crossroads on the route, only motion happens in virtual space. The method to generate goal-oriented behavior is problem solving. That's the difference. Creativity. A Turing machine is designed to perform ready algorithms, the processor of the brain – to generate algorithms.
The main approach to creativity remains ancient trials-and-errors method. For computers, more orderly variants such as full scan of possible solutions are used. In any case, the processor should prompt some actions, then assess their results. That's exactly what the limbic system does. Looks like the amygdala is a block of system-level memory similar to the basal ganglia. Only they keep elementary programs for sensorimotor coordination while the amygdala keeps genetically predetermined states of the brain itself such as fear or aggression. Human emotions correspond to processor instructions of computers, only the brain doesn't use the Von Neumann architecture. It uses a finite-state machine which has elementary states rather than actions. This approach may be very powerful. It is well known that the brain parts tend to have connections according to the principle of all-to-all. Not all of them are used at once. Instead, only a fraction is employed when necessary. This means that you can dynamically create different working machines for different circumstances.
On the other hand, the hippocampus has all the necessary means to assess the results. It receives input from major sensory channels and can generate sharp pulses of activation at the output. Again, this works differently. When you write a control program, you would create a variable to measure the assessment. Say, in the range [-1,1]. The program will input data, process it, set that variable, then use it for decision making. Looks like the brain has not such a separate variable. Instead, associative memory immediately links an input situation to the appropriate emotional reaction such as attraction or aversion. That's why negative emotions are harmful. You get tensed and if this tension has no exit, you must contain it. That leads to double tension and quick tiredness.
Other parts of functional architecture
If the neocortex is the associative memory of the brain and the limbic system – its processor, the next step would be to describe Input/Output system. In contrast to PC, human memory is anatomically partitioned and different parts are assigned for different purposes. Comparison of human and computer I/O systems may be as lengthy as the preceding comparison of central processing, but now we will just mention that they are very different as well and concentrate on the first. In particular, I/O system of PC was very asymmetric. These computers had quickly got a graphical display with a complicated controller and a video card, but the main input remained a keyboard and a mouse. When decent video input became available, PC itself already lost its market share. Output subsystem was obviously better developed here and the term "programming" reflects this. It refers more to generating output actions and pays less attention to analysis of input data.
In humans, the main input comes from vision and the main output goes to muscles via the motor system. Both are approximately equally developed. They have cortical and subcortical representation of comparable complexity. The human sensory system has several different receptors (vision, hearing, etc.) with separate sensory channels. At the top, all of them converge in the TPO (Temporalis-Parietalis-Occipitalis) region of the neocortex which resides between cortical representations of different sensory modalities. The TPO zone obviously syntheses an abstract picture of the world and is also related to human consciousness.
While the sensory neocortex occupies the back half of the brain surface, the front half is designated for motor output and actions in more general sense. Humans have only one channel of external output – muscles, but the overall output system is slightly more complicated than its input counterpart. It includes the cerebellum sometimes called a small brain inside the big one. The cerebellum may be considered as a close analogy of a video card in PC. It has its own cortex and subcortical nuclei that is seemingly memory and a processor, but is used for motor tasks.
At last, yet another part of architecture. The reticular formation has a few distinct features. It is located among the most ancient parts of the brain, is large, but has virtually no internal structure, and has diffuse output to other parts. This is an activating block of the brain. We could outline the analogy with the power unit of PC, but here the difference is probably the largest. The nervous system has no separate power supply at all. Instead, each neuron generates energy for itself. Nevertheless the On/Off switch is necessary. It comes in the form of the sleep/awakeness cycle. In contrast to computer electronics, neurons are heavily dependent on biochemistry which produces a lot of waste and requires regeneration of stock substances. That's why we sleep every day and the reticular formation controls this cycle. Moreover, it can fine-tune its output so as not just to turn the whole brain on or off, but to do it separately for different structures. The reticular formation not only sends output, but also receives input from other parts. This supports the principle of a dynamic processor discussed previously in connection with the amygdala.
That's all about the core functional architecture of the brain. Now let's formalize it in a concise description.
Formal neurocomputer
This machine is created from blocks of associative memory linked in certain order. Each block has input and output in the format of 2D or 3D image plus a control signal that switches the mode of operation – read out, write in, or inactivate. In the first mode, the block uses the input image as a key to retrieve an output one. In the second – memory remembers the association between 2 images.
The machine is connected to the external world by input and output. Both have hierarchical structure with several tiers and maybe several parallel channels which converge from periphery to the centre. The main difference is the direction of information flow. The sensory system gathers data from various receptors and creates an integral picture of the world in the central representation. The motor system gets a general idea of the action in its central representation, then implements it in concrete actions of various effectors.
Operation of the machine is determined by interaction between central images of sensory and motor systems. A sensory image serves as a key to the motor block. Associative memory generates an idea and while this idea is being realized, the associative pathways between the upper levels are inactivated. As soon as the action is completed, they are activated again, but now the content of sensory operative memory is already changed. Two sources are possible. The action may change something in the environment or data may be written directly into the sensory memory in the course of internal exchange. Hence the key will be different and associative memory will generate the next insight. The cycle repeats. So the work of this machine is based on chain reactions.
Fig. 7.
This scheme is compiled from the most reliably described links between various brain regions. How to understand its operation? The main problem here is multiple feedbacks. The common method in such cases is to break the loop. Then, operation of the open circuit will become obvious. So we need to remove some links from this scheme, but which specifically? The red nucleus is a clearly supplementary structure because the motor cortex already has direct output to motoneurons. So we will throw it out together with the cerebellum. Functions of the latter are well established. It provides muscular coordination and fine-tuning. If it is damaged, patients still cam move, but their movements become more primitive, rough, and non-coordinated. The cerebellum is a typical "improvement".
Fig. 8.
This picture still may be further simplified.
Fig. 9.
The upper link on the picture is a well known reflex. The link through the basal ganglia is a bit more complicated. It combines the previous principle of stimulus-reaction with sensorimotor coordination. The last detail comes from the fact that muscular actions change the environment and these changes will be perceived by external sensors.
Fig. 10.
This device is not so weird as it may seem at the first glance. For illustration, let's demonstrate how to implement common linear programming here. Suppose you have some algorithm, that is a definite sequence of actions. Just associate each one with the corresponding number in the sequence and remember these associations. By the way, memorizing may be performed in any order. Next, you will need a small system program. Namely, you should have a sensory image which keeps the number of the current operation. That is you emulate the register of a common processor. Increment it after each action. Now use this image as a key for sensorimotor associations and you will be able to run common programs on this computer.
Processor of the brain
Fig. 11.
Limbic system.
1 – Hippocampus. 2 – Amygdala. 3 – Mammillary body. 4 – Fornix.
Basal ganglia.
5 – Caudate nucleus. 6 – Lentiform nucleus.
Other.
7 – Pituitary gland.
Like a personal computer, the human brain also has memory and a processor, although they operate and interact quite differently. Neurocomputing does not use linear memory containing bytes or machine words which may be accessed one by one. Instead, it relies upon associative memory that keeps bytes in synapses, but they can't be immediately read out. Humans use a different principle. Memory bytes form a distributed hologram which is only an instrument to retrieve data. The output comes in the form of an image encoded by a current spike rate distributed over some neural net. Again, in contrast to a computer, each neuron outputs not binary data of 1 or 0, but a real value in some range, say [0, 1]. This principle is used both in memory and a processor; they are different only by the architecture. Human memory is located in the neocortex which is a large 2D structure packed into the skull. It has 6 layers of neurons of different types which are basically the same throughout the whole of its area, but this area is genetically partitioned according to the functional principle. The frontal half of the neocortex works mainly for output, the rear – for input. Many different cortical fields are interconnected by direct axonal projections. This system alone already can work like computer software. When some portion of data comes to a sensory input, this is 1:1 as an event in a computer. This event produces activation which spreads to a different field, undergoes various processing, and finally may generate a muscular action. The problem is that this flow is not synchronized, no single clock generator. Also, brain connections tend to be organized by the principle of all-to-all or one-to-many. The uncontrollable spread of activity will lead to informational overflow. Obviously, for a given event, only a fraction of all fields and their links should be used. All the rest need to be inactivated. This is the main function of the brain processor – the limbic system.
In a computer, the processor has a definite set of instructions and handles data loaded from memory. In the brain, data are processed immediately in memory. The processor only starts and stops this process. Nevertheless, it has some instruction set too. The limbic system generates emotions. Motions are activation of muscles, emotions – similar activation of internal brain structures. Like the neocortex, parts of the limbic system also keep various images, but these images encode patterns of activity for other parts of the brain. Some of them need to be boosted, others – suppressed. Such emotional patterns may be learned, but also may be inborn basic distributions of strong reactions.
Another important function of the limbic system is related to operative memory. Associative memory does not keep the image itself – only the ability to evoke it. Nevertheless, neural nets can retain images too. For this purpose, they build a sort of a trigger system using positive biofeedback. In this case, the output of some internal block is supplied to its input starting recirculation in the closed loop. As soon as some pattern gets into this system, it becomes locked and can exist for a prolonged time even if the source was turned off. The limbic system was initially called the Papez circuit. It and the adjacent basal ganglia have many such loops which may be employed in various cases. Usually, some cortical field sends output to the limbic system and also receives input from it. Obviously, when some image is reverberating through the limbic structures, they can control it. That's how this system manages operative memory.
A well known function of emotions is assessment. In fact, most of them are broadly divided into 2 categories – positive and negative. The strength of emotion provides a value for this sign. Suppose you need to write a program for monitoring of some object. How would you do that? Probably you will define an appropriate variable, write the evaluation there, and use it for decision making when necessary. Seemingly the limbic system operates differently. The assessment is not kept separately but appears immediately at the output as the aforementioned control signal. That is, if the results of some actions are negative, the aversive emotion will immediately suppress that activity.
2 key blocks of the limbic system are the amygdala and the hippocampus. They are very different and seemingly complement each other. The amygdala is a typical subcortical structure. It is linked to generation of very basic states of the organism as a whole – such as fear and aggression. The hippocampus is the archicortex. This is a small piece of cortical matter which emerged early in evolution and has just 3 layers. Thus it should perform some important functions. An interesting theory of the hippocampus is as follows. What you usually see in illustrations (CA1, CA3, etc.) is a cross-section. The hippocampus as a whole is a tube or more precisely – a long cone in correspondence to its name Cornu Ammonis. Functionally, the hippocampus generates a 1D image that is a vector. The index of this vector goes along the tube while its cross-section provides a pathway for data rotation. Input comes from the entorhinal cortex, output returns back to it. That is, the entorhinal cortex may be a memory buffer which can receive an image from a wide variety of cortical regions, then keep it for temporary dynamical storage via reverberation through the hippocampus. Thus, the hippocampus may work as famous operative memory. Note that it can't store data because it is 1D, while cortical images are usually 2D. Only plays some important role in this storage. In addition to that loop, another major output of the hippocampus goes via the fornix to several small (0D that is scalar) areas of the brain. Through the mammillary bodies, the hippocampus can control the thalamus. It also sends output to the septum (a pleasure zone). There are several such spots in the brain (the substantia nigra, ventral tegmental area). They use the neuromediator dopamine which can control formation of long-term memory.