Computational Aspects of the Human Brain

As part of my lectures on the neurobiology of addiction - I digress briefly to discuss the computational aspects of the brain.  A lot of that discussion is focused on on the above graphic showing that overlaps in capacity with a list of the world's ten fastest supercomputers.  At least that is the estimate of the AI Impacts group.  It is basically a computation based on edges and nodes. I include power estimates for a brain from existing hardware to the actual power estimate of the human brain that I would guess every physical chemistry student from my era had to contemplate at one time.  And then I try to stimulate some discussion of supercomputers versus the human brain and it generally falls flat.  My Socratic process goes something like this:

"OK so we know that humans can't really beat computers on straightforward calculations so what advantages do we have?"

"I will give you a hint - why do we all go thorough residency training? Why can't you learn your specialty by reading about it in a book?"

The first lesson is pattern matching.  The human brain is designed not only to match patterns but to be trained to match a lot of them.  Some research article suggest about 88,000, but when  you consider what has to be matched that has be very a very low estimate.  I quote references from 15-20 years ago and a course I used to teach on diagnostics and diagnostic decision making.  Ophthalmologists correctly diagnosing diabetic retinopathy at a much higher rate than nonspecialists.  Dermatologists diagnosing rashes faster and correctly classifying ambiguous rashes with greater precision than nonspecialists. If I am really on a roll I might digress to talk about Infection Disease rounds at the Milwaukee VA sometime during 1982.  I was the medical student on a team of residents and fellows doing a consult for possible subacute bacterial peritonitis.  As the attending listening to the presentation he was also looking at a rash on the patient's shin.  By the time we were done he had also diagnosed a strep infection in addition to the peritonitis.  When you have significant pattern matching capacity, and you have been exposed to relevant patterns you can recognize them quickly and improve the speed and accuracy of the diagnosis.

I move on at that point to illustrate that the computers are catching up.  The simple captcha is less robust in discriminating machines from humans.  Opening an account may take more that checking the "I am not a computer" box. Now you might have to look at 8 pictures and check the one that contains an automobile or a stop sign.  Some of these photos are often difficult for humans to decipher.

At that point I touch on human consciousness - both the unique aspects and computational power it takes to generate.   About a decade ago I started saying that if there are 8 billion people on the planet - there are 8 billion unique conscious states. It makes sense at a number of levels especially when I put up hard numbers on cell types, protein types, the genetic information represented, and the typical stream of consciousness that every person experiences every day.  What is the content and flow of that activity? How does it get biased in psychiatric disorders and addictions?  How much computational power does it take to generate all of this information?

My latest step is what I like to consider The Matrix observation.  If I am standing in front of a room of 15-20 residents - what does it take to generate the physical representation of all of the people and all of the objects in that room? What does it take to make all of those representations unique? There can be a general consensus about what is happening - but just looking around it is clear that there are obvious different experiences.  One person looks very interested and one semi-interested.  One person is more focused on her Smartphone and is indifferent to my presentation.  Some people look sleepy.  Others look irritated.    They also appear to be indifferent to the context.  I know that my job is to try to get this information across and make is semi-interesting.  There is no real expectation on the residents.  It is clear from the questions I ask that they really don't know too much about the brain.  There are parallel streams of information processing that allow us all to evaluate what is occurring on the fly both the information content and emotion.  In some case there are pre-existing heuristics and in other cases associative memories and biases.  All of this represents a tremendous amount of information or computational power depending on how you may want to discuss it.

I have been preoccupied myself with the computational power and estimating it accurately. I used to try to model it in terms of electrical buses and neuronal firing rates - but the numbers I got were far too low.  There really are no good equivalents in the physical world with the possible exception of the Transversed Edges per Second (TEPS) metric used by the AI Impacts group for the above graphic.  You can't really use estimates of typical audio or visual information and concluding that is what is being processed by the brain.  I have never really seen an accurate estimate of all of the sensory information that the brain is handling in real time.

I went to bed last night and waited for sleep reverie or that period of time where you stream of thinking is jumbled and illogical just before you fall asleep.  As a chronic insomniac it is one of the few reliable cues that I am probably getting some sleep.  It happened when I had a sudden image of a baby high up on a brick wall, followed immediately by a person who seemed to be me sitting in a single seat futuristic car.  The salesperson was describing it to me and suddenly the car and everything else was being swept down what appeared to be a very sophisticated hydraulic roadway. The roadway was bright orange and the salesman shifted his pitch to tell me the advantages of this kind of a roadway with this car.  The roadway was moving at about 20 miles per hour.

I shifted briefly and remembered it was 2018 and I was in my bedroom in Minnesota.

And for a minute I thought about being able to estimate the information necessary to generate that brief full color science fiction scene and the three or four more I would encounter that night.


George Dawson, MD, DFAPA


Some additional examples as they happened:


1. Dream of 11/22/2018:  I am back on my old inpatient unit.  The layout is exactly the way it was 20 years ago (the building has since been razed).  I am working with the same staff.  I walk into the examination room to look at the templates for the day.  In those pre-EHR days I had designed a template with all of the relevant features necessary for the billing and coding requirements.  At the time we were all threatened with legal action if we did not comply with these regulations even though they were totally subjective.  In those days I worked with a physicians assistant who prepared the templates ahead of time before we started interviewing patients and completing the subjective aspects of the evaluation and documenting the progress.

The templates were all stacked in two circular patterns - ten templates in each circle.  They appeared to be the exact temples that we used right down to the blurred fonts from being photocopied too many times.  The precise handwriting of my physicians assistant in the diagnostic section was exactly the way he wrote things down.  The placement of the exam table and crash cart were exactly where they were in reality.  The table we used was circular and about 6 feet in diameter with a laminated blonde wood finish and it was also exactly the way it was in that now 20 year old reality.

I looked at the templates and asked myself: "Why are they all face down?  I can't see the patient's name or identifying data.  I will have to go through them all to find the correct template when I start interviewing patients."

I felt somewhat irritated.

And then I woke up. 

2. Dream of 11/23/2018: I am in a large modern, multi-floor medical facility. It is not one that I specifically recognize, but it seem like there are elements of many that I have been in.  I am rushing around on the ground floor. The impression I have is that I am late for a lecture. It doesn't seem to be an explicit CME lecture but everyone else there (including myself) is too old to be a medical student or resident. I run into the elevator just beating the door as it closes.

I make to to the lecture.  It is basically a large room - maybe 50' x 50' and for some reason I burst through the door running at full speed.  Just before the crash into the back wall, a guy standing on the side wall grabs my arm to slow me down and stop me.

I ask myself if that was really necessary because my plan was just to stop myself by reaching out and planting my hand on the back wall.  I notice that there are several people who I assume are physicians that are standing and sitting near the back wall and they seem a little alarmed about something.

Then I am back in the elevator and headed to the ground floor.  I am walking out of the building and realize that I am chewing something metallic.  I realize that is is a collection of machine screws, nuts, and ball bearings. I realize that is purchased them on the ground floor of this building and that they are sold for that purpose.  I also know that I cannot really chew them or I will break my teeth.  I have to cautiously move them around in my mouth.  They remind me of a chap stick product that is applied with a ball bearing device at the end of the dispenser.

I wake up with a metallic taste in my mouth.

3.  Dream of 11/24/2018:  I am back in my home town. The streets and buildings are identical to the way they look in reality.  I am with a friend of mine and we are looking at a 1960s vintage Buick.  It is large and chalky white.  He tells me that his sister recently bought it and she wants to take everyone for a ride.  He thinks I should come along, but just then I remember something that his sister said to me in the last 15 years that would make me not want to go with them. He is talking about the car as though it is a great buy, but as I walk past the tail end of the vehicle, I notice that it has a new paint job and that whoever did it just painted over the decals of the previous dealers.  You can see them faintly through the paint.

I tell my friend that I can't stay around because I have to go grocery shopping. Just then one of his friends comes out and tells me that he has a lot of groceries he can just give me so that I will not have to go to the store.  I decline but he continues to insist. I reluctantly accept free groceries and sling them over my shoulder in a large garbage bag and start to walk home.

The real path home is just 6 blocks - 4 blocks south and 2 blocks east. It is all residential. But in the dream I encounter a large modern baseball park right next to the street. The game is just completed and they are interviewing the winning pitcher. She is in her mid 20s and short and compactly built.  Her uniform and short brown hair are drenched with sweat.  Just then I notice that it is hot. The announcer asks her if the heat was a factor in the game and she says:

"The hot was so hot that when my hot fingers touched the hot ball - I could barely feel it." 

The ballpark looks real.  There are thousands of cheering fans and the announcer and the pitchers statements are amplified over the PA system.  Everything is in color.

I wake up and feel hot and flushed.

Human Brain Performance Compared To Supercomputers

For about the past year, I have been using transversed edges per second (TEPS) in my lectures about neurobiology to give a rough estimate of the computing power of the human brain and a rougher estimate of where brain power compares with artificial intelligence (AI).  I finally found the detailed information on the AI Impacts web site and wanted to post it here, both for future reference and to possibly generate more interest in this topic for psychiatrists.

I have been interested in human computer comparisons since I gave a Grand Rounds on the topic back in the 1990s.  Back then I was very interested in bandwidth in the human brain and trying to calculate it.  My basic approach was to look at the major bus systems in the brain and their fiber counts and try to estimate how much information was passing down that bus.  In engineering terms a bus is a path that the computer or processors use to communicate with other devices or processors.  The rate at which that communication passes down that pathway is a major limitation in terms of computing speed to the rate at which tasks are transmitter to peripheral devices.  Engineers typically specify the characteristics of these communication paths.  A good example are the standard USB connectors on your computer.  Today there are USB 2.0 and USB 3.0 connectors.  The USB 2.0 devices can support a data transfer rate of 480 Mbps or 60 MBs.  The USB 3.0 connection supports 5 gbps or 640 MBs.

In the work I was doing in the 1990s, I looked at the major structures in the brain that I considered to be bus-like the fascicles and the corpus callosum.  Unfortunately there were not many fiber count estimates for these structures.  It turns out that very few neuroanatomists count fibers or neurons.  The ones who do are very exacting.  The second issue was the information transfer rate.  If fiber counts could be established was there any reliable estimate of the information contained in spikes.  I was fortunate at the time that a book came out that was somewhat acclaimed at the time called Spikes.  In it the authors, attempted to calculate the exact amount of information in these spikes.  They used a fast Fourier transform (FFT) methodology that I was familiar with from quantitative EEG (QEEG).  From available data t the time I was limited to calculating the bandwidth of the corpus callosum.  I used a fiber (axon) count of 200 million.  It turns out that the corpus callosum is a heterogeneous bus with about 160,000 very large fibers.  Using a bit rate of 300 bits/sec for each spiking neuron multiplied by the entire bus results in a total of 60 Gbs.  I had a preliminary calculation but realized I had about another 11 white matter fiber tracts connecting lobes, hemispheres and the limbic system.  I did not have the fiber counts for any of these structures and the top neuroanatomist in the world could not help me.

Then I found an interesting question posted in a coffee shop.  In the process of investigating it, I found some preliminary data about a group that was using a calculation called tranversed edges per second (TEPS) and showing at least on a preliminary basis that the human brain is currently calculating at a rate that is currently on par with supercomputers.  I found additional papers from the group, just this week.  The articles can be read and understood by anyone.  They are interesting to read to look at the authors basic assumptions as well as how they might be wrong.  They give rough estimates in some cases about how large the error might be if their assumptions are wrong.  They provide detailed references and footnotes for their assumptions and calculations.  

Their basic model assumes that the human brain is comprised of interconnected nodes in the same way that a supercomputer connects with processors or clusters of processors.  This basic pattern has been described in some situations in the brain but the details are hard to find.  There is also a question about the level for analysis of the nodes.  For example are large structures the best choice and if not how many smaller networks and nodes are relevant for the analysis.  In high performance computing (HPC) several bottlenecks are anticipated as nodes try to connect with one another including bus latency, bus length in some cases, and the smaller scale of any circuity delays on the processor.  The ability to scale or divide the signal without losing the signal across several pathways is also relevant.  For the purpose of their analysis, these authors use one of the estimated numbers of neurons in the brain (2 x 10¹¹).  The authors use a figure of 1.8-3.2 x 10¹⁴ synapses.  Division yields synaptic connections for each neuron at 3,600-6,400.

The TEPS benchmark is discussed in detail on the Graph 500 web site under 8.2 Performance Metrics (TEPS).  Reference 1 contains a more basic accessible definition as the "time required to perform a breadth first search of a large random graph requiring propagating information across every edge of the graph."  The information propagation is between nodes or nodes and memory locations.  The Graph 500 site also contains a listing of top performing supercomputer system and a description of their total number of processors and cores.  The rankings are all in billions of TEPS or GTEPS in terms of the performance benchmark with 216 systems ranked ranging from 0.0214748 to 38621.4 GTEPS.

For the human brain calculation, the authors use the conversion of TEPS = synapse-spikes/second = number of synapses in the brain x average spikes/second in neurons = 1.8-3.2 x 10¹⁴ x 0.1-2 = 0.18 - 6.4 x 10¹⁴ TEPS or 18 - 640 trillion TEPS.

What are the implications of these calculations?  If accurate, they do illustrate that human brain performance is limited by node to node communication like computers.  The AI researchers are not physicians, but it it obvious that taking more nodes or buses off line will progressively impact the computational aspects of the human brain.  We already know that happens at the microscopic level with progressive brain diseases and at the functional level with processes that directly affect brain metabolism but leave the neurons and synapses intact.  The original research in this area with early estimates was performed by researchers interested specifically in when computers would get to the computational level of the human brain.  Several of these researchers discuss the implications of this level of artificial intelligence and what it implies for the future.

For the purpose of my neurobiology lecture, my emphasis in on the fact that most people don't know that they have such a robust computational device in their head.  We tend to think that a robust memory is the mark of computation performance and ignore the fact that is why humans can match patterns faster than computers and comprehend context faster than computers.  We also have a green model that is more cost effective.

These are all great reasons for taking care of it.

George Dawson, MD, DFAPA



References:

1:  AI Impacts:  Brain performance in TEPS:  http://aiimpacts.org/brain-performance-in-teps/

2:  AI Impacts:  Human level hardware:  http://aiimpacts.org/category/ai-timelines/hardware-and-ai-timelines/human-level-hardware/

3:  AI Impacts:  Brain Performance in FLOPS:  http://aiimpacts.org/brain-performance-in-flops/

4:  Rieke F, Warland D, de Ruyter van Steveninck, Bialek W.  Spikes: Exploring the neural code.  The MIT Press, Cambridge, MA 1997, 395 pp.


Attribution:

Slides below are from my original 1997 presentation (scanned from Ektachrome).  Click to enlarge any slide.  I am currently working on a better slide to incorporate the work of the AI Impacts and Graph 500 groups on a single slide with an additional explanatory slide.

Additional reference:

My copy of Spikes:

Thoroughly Read: