Stephen Smith's Blog

Musings on Machine Learning…

Posts Tagged ‘AI

The Road to Strong AI

with one comment


There have been a great many strides in the field of Artificial Intelligence (AI) lately, with self driving cars becoming a reality, computers now routinely beating human masters at chess and go, computers accurately recognizing speech and even providing real time translation between languages. We have digital assistants like Alexa or Siri.

This article expands on some ideas in my previous article: “Some Thoughts on Artificial Intelligence”. This article is a little over a year old and I thought I might want to expand on some of the ideas here. Mostly since I’ve been reading quite a few articles and books recently that claim this is all impossible and true machine intelligence will never happen. I think there is always a large number of people that argue that anything that hasn’t happened is impossible, after all there were a large number of people still believed human flight was impossible after the Wright brothers actually did fly and for that matter it’s amazing how many people still believe the world is flat. Covering this all in one article is too much, so I’ll start with an overview this time and then expand on some of the topics in future articles.

The Quest for Strong AI

Strong AI or Artificial General Intelligence usually refers to the goal of producing a true intelligence with consciousness, self awareness and any other cognitive functions that a human processes. This is the form of AI you typically see in Science Fiction movies. Weak AI refers to solving narrow tasks and to appear intelligent at doing them. Weak AI was what you you are typically seeing with computers playing Go or Chess, self driving cars or machine pattern recognition. For practical purposes weak AI research is proving to solve all sorts of common problems and there are a great many algorithms that contribute to making this work well.

At this point Strong AI tends to be more a topic for research, but at the same time many companies are working hard on this, but often we suspect in highly secretive labs.

Where is AI Today?

A lot of AI researchers and practitioners today consider themselves working on modules that will later be connected to build a much larger whole. Perhaps a good model for this are the current self driving cars where people are working on all sorts of individual components, like vision recognition, radar interpretation, choice of what to do next, interpreting feedback from the last action. All of these modules are then connected up to form the whole. A self driving car makes a good model of what could be accomplished this way, but note that I don’t think anyone would say a self driving car has any sort of self awareness or consciousness, even to the level of say a cat or dog.

Researchers today in strong AI are building individual components, for instance good visual pattern recognition that use algorithms very similar to how neurologists have determined the visual cortex in the brain work. Then they are putting these components together on a “bus” and getting them to work together. At this point they are developing more and more modules, but they are still really working in the weak AI world and haven’t figured out quite how to make the jump to strong AI.

The Case Against Strong AI

There have been quite a few books recently about why strong AI is impossible, usually arguing that the brain isn’t a computer, that it is something else. Let’s have a look at some of these arguments.

This argument takes a few different forms. One compares the brain to a typical von Neumann architecture computer, and I think it’s clear to everyone that this isn’t the architecture of the brain. But the von Neumann architecture was just a convenient way for us poor humans to build computers in a fairly structured way that weren’t too hard to program. Brains are clearly highly parallel and distributed. However there is Turing’s completeness theorem which does say all computers are equivalent, so that means a von Neuman computer could be programmed for intelligence (if the brain is some sort of computer). But like all theoretical results, this says nothing about performance or practicality.

I recently read “Beyond Zero and One” by Andrew Smart which seems to infer that machines can never hallucinate or do LSD and hence must somehow be fundamentally different than the brain. The book doesn’t say what the brain is instead of being a computer, just that it can’t be a computer.

I don’t buy this argument. I tend to believe that machine intelligence doesn’t need to fail the same way human brains fail when damaged, but at the same time we learn an awful lot about the brain when studying it when it malfunctions. It may turn turn out that hallucinations are a major driver in creativity and that once we achieve a higher level of AI, that AIs in fact hallucinate, have dreams and exhibit the same creativity as humans. One theory is that LSD removes the filters through which we perceive the world and opens us up to greater possibilities, if this is the case, removing or changing filters is probably easier for AIs than for biological brains.

Another common argument is that the brain is more than a current digital computer, and is in fact a quantum computer of far greater complexity than we currently imagine. That in fact it isn’t chemical reactions that drive intelligence, but quantum reactions and that in fact every neuron is really a quantum computer in its own right. I don’t buy this argument at all, since the scale and speed of the brain exactly match that of the general chemical reactions we understand in biology and that the scale of the brain is much larger than the electronic circuits where we start to see quantum phenomena.

A very good book on modern Physics is “The Big Picture” by Sean Carroll. This book shreds a lot of the weird quantum brain model theories and also shows how a lot of the other more flaky theories (usually involving souls and such) are impossible under modern Physics.

The book is interesting, in that it explains very well the areas we don’t understand, but also shows how much of what happens on our scale (the Earth, Solar System, etc.) are mathematically provable to be completely understood to a very high accuracy. For instance if there is an unknown force that interacts with the brain, then we must be able to see its force carrier particle when we crash either antiprotons with protons or positrons with electrons. And since we haven’t seen these to very high energies, it means if something unknown exists then it would operate at the energy of a nuclear explosion.

Consciousness and Intelligence in Animals

I recently read “Are We Smart Enough To Know How Smart Animals Are?” by Frans de Waal. This was an excellent book highlighting how we (humans) often use our own prejudices and sense of self-importance to denigrate or deny the ability of the “lesser” animals. The book contains many examples of intelligent behaviour in animals including acts of reasoning, memory, communication and emotion.

I think the modern study of animal intelligence is showing that intelligence and self-awareness isn’t just an on/off attribute. That in fact there are levels and degrees. I think this bodes very well for machine intelligence, since it shows that many facets of intelligence can be achieved at a complexity far less than that inherent in a human brain.


I don’t recommend the book “Beyond Zero and One”, however I strongly recommend the books: “Are We Smart Enough to Know How Smart Animals Are?” and “The Big Picture”. I don’t think intelligence will turn out to be unique to humans and as we are recognizing more and more intelligence in animals, so we will start to see more and more intelligence emerging in computers. In future articles we will look at how the brain is a computer and how we are starting to copy its operations in electronic computers.




Written by smist08

May 16, 2017 at 7:49 pm

Posted in Artificial Intelligence

Tagged with ,

Some Thoughts on Artificial Intelligence

with 2 comments


A few years ago I posted a blog post on the Singularity. This is the point where machine intelligence surpasses human intelligence and all predictions past that point are out the window. Just recently we’ve seen a number of notable advances in AI as well as a number of instances where it has gone wrong. On the notable side we have Google’s DeepMind AlphaGo program beat the world champion at Go. This is remarkable since only recently did IBM’s Deep Blue program beat the world champion at Chess and the prevailing wisdom was that Go would be much harder than Chess. On the downside we have Microsoft’s recent Tay chat bot which quickly became a racist rather tainting Microsoft’s vision as presented at their Build Conference.

So this begs the question, are computers getting smarter? Or are they just becoming more computationally better without any real intelligence? For instance, you can imagine that Chess or Go just require sufficient computational resources to overcome a poor old human. Are chat bot’s like Tay really learning? Or are they just blindly mimicking back what is fed to them? From the mimicking side they are getting lots of help from big data which is now providing huge storehouses or all our accumulated knowledge to draw on.

In this article I’ll look at a few comparisons to the brain and then what are some of the stumbling blocks and where might true intelligence emerge.



Let’s compare a few interesting statistics of humans to computers. Let’s start with initialization, the human genome contains about 3.2Gigabytes of information. This is all the information required to build a human body including the heart, liver, skin, and then the brain. That means there is very little information in the genome that could be dedicated to providing say an operating system for the brain. An ISO image of Windows 10 is about 3.8Gigabytes, so clearly the brain doesn’t have something like Windows running at its core.

The human brain contains about 86 billion neurons. The Intel i7 processor contains about 1.7 billion transistors. Intel further predicts that their processor will have as many transistors as the brain has neurons by 2026. The neuron is the basic computational logic gate in the brain and the transistor is the basic computational logic gate in a computer. Let’s set aside the differences. A neuron is quite a bit more complicated than a transistor, it has many more interconnections and works in a slightly analog fashion rather than being purely digital. However, these differences probably only account for one order of magnitude in the size (so perhaps the computer needs 860 billion transistors to be comparable). Ultimately though these are both Turing machines, and hence can solve the same problems as proved by Alan Turing.

To compare memory is a bit more difficult since the brain doesn’t separate memory from computation like a computer. The neurons also hold memories as well as performing computations.  Estimates on the brains memory capacity seem to range from a few gigabytes to 2.5petabytes. I suspect its unlike to be anywhere close to 1petabyte (100Gigabytes). Regardless it seems that computers currently can exceed the memory of the brain (especially when networked together).

From a speed point of view, it would appear that computers are much faster than the brain. A neuron can fire about 200 times per second, which is glacial compared to a 3GHz processor. However, the brain makes up for it through parallel processing. Modern computers are limited by the Von Neumann architecture. In this architecture the computer does one thing at a time, unlike the brain where all (or many) neurons are all doing things at the same time. Computers are limited to Von Neumann architectures because these make it easier to program. Its hard enough to program a computer today, let alone if it didn’t have the structure this architecture imposes. Generally, computer parallel processing is very simple either through multiple cores or through very specific algorithms.


Learning Versus Inherited Intelligence

From the previous comparisons, one striking data point is the size of the human genome. In fact, the genome is quite small and doesn’t have enough information to seed the brain with so called inherited intelligence. Plus, if we did have inherited intelligence it would be more aligned to what humans needed to survive hundreds of thousands of years ago and wouldn’t say tell you how to work your mobile phone. What it appears is that the genome defines the structure of the brain and the formulae for neurons, but doesn’t pre-program them with knowledge, perhaps with just some really basic things like when you feel hungry, you should eat and to be afraid of snakes. This means nearly all our intelligence is learned in our earliest years.

This means a brain is programmed quite differently from a computer. The brain has a number of sensory inputs, namely touch, sight, hearing, smell and taste and the with the help of adult humans, it learns everything through these senses. Whereas a computer is mostly pre-programmed and the amount of learning its capable of is very limited.

It takes many years for a human to develop, learn language, basic education, physical co-ordination, visual recognition, geography, etc. Say we want a computer with the level of intelligence of a ten-year-old human; then, do we need to train a computer for ten years to become comparable? If so this would be very hard on AI researchers needing ten years to test each AI to see if it works.

Complexity Theory

It seems that both computers and the brains are both Turing machines. All Turing machines can solve the same problems, though this says nothing about how long they may take. Computer’s logic elements are far faster than neurons, but suffer from being organized in a von Neumann architecture and thus operate very serially as opposed to the brain that does everything in parallel. But as such both are programmed from very simple logic elements with a certain small amount of initial programming. So where does self-aware intelligence arise from?

I believe the answer comes from complexity and chaos theory. When you study dynamic systems with increasing complexity like studying transitions to turbulence in fluid mechanics or studying more and more complicated systems like cellular automation or fractals, you find there are emergent stable solutions (sometimes called strange attractors) that appear that couldn’t be predicted from the initial conditions. With brains having billions of neurons all performing simple logic operations, but in parallel this is a very complex system. There is guaranteed to be emergent stable behaviours that evolution has adjusted into becoming our intelligence.

What’s Needed

Our computers aren’t quite at a truly self aware intelligent state yet (at least that I know of, who knows what might be happening in a secret research lab somewhere). So what is needed to get over the hump and to create a true artificial intelligence? I believe we need two things, one on the hardware side and the other on the software side.

First we need the software algorithm that the brain uses to learn from its environment. This must be fairly simple and it must apply to a wide range of inputs. There isn’t enough data in the human genome for anything else. I think we are getting closer to this with algorithms around the Hidden Markov Model that are currently being used in machine learning. One key part of this algorithm will be how it can be adapted to scale by running millions of copies in parallel.

Second we need the hardware to run it. This is a bit controversial, since one school of thought is that once we have the correct algorithm then we can run it on standard hardware, since its raw processing speed will overcome its lack of parallel processing. Even hardware like GPUs with hundreds of cores aren’t anywhere as parallel as the brain. Until we figure out this ideal learning algorithm, we won’t know the exact computer architecture to build. There are people building computer hardware that are very parallel and more precisely model neurons, but others feel that this is like building an aeroplane by exactly simulating birds flapping their wings.


We’ve solved a lot of difficult problems with Artificial Intelligence computer algorithms. We now have self-driving cars, robots that can walk over rugged terrain, computer world chess and go champions and really good voice and picture recognition systems. As these come together, we just need a couple more breakthroughs to achieve true intelligence. Now it seems every now and then we predict this is just around the corner and then we get stuck for a decade or so. Right now we are making great progress and hopefully we won’t hit another major roadblock. We are certainly seeing a lot of exciting advances right now.

Written by smist08

April 1, 2016 at 9:09 pm

Posted in Artificial Intelligence

Tagged with , , ,