I was interviewed by the excellent Allen Saakyan for his Simulation YouTube channel. Check it out if you want to know about my research… and also my take on the question of Life, the Universe, and Everything. 😛
There’s also a shorter excerpt where we speculate about schizophrenia and “hyperrealities”:
An answer I wrote to the Quora question Does the human brain work solely by pattern recognition?:
Great question! Broadly speaking, the brain does two things: it processes ‘inputs’ from the world and from the body, and generates ‘outputs’ to the muscles and internal organs.
Pattern recognition shows up most clearly during the processing of inputs. Recognition allows us to navigate the world, seeking beneficial/pleasurable experiences and avoiding harmful/negative experiences.* So pattern recognition must also be supplemented by associative learning: humans and animals must learn how patterns relate to each other, and to their positive and negative consequences.
And patterns must not simply be recognized: they must also be categorized. We are bombarded by patterns all the time. The only way to make sense of them is to categorize them into classes that can all be treated similarly. We have one big category for ‘snake’, even though the sensory patterns produced by specific snakes can be quite different. Pattern recognition and classification are closely intertwined, so in what follows I’m really talking about both.
Creativity does have a connection with pattern recognition. One of the most complex and fascinating manifestations of pattern recognition is the process of analogy and metaphor. People often draw analogies between seemingly disparate topics: this requires creative use of the faculty of pattern recognition. Flexible intelligence depends on the ability to recognize patterns of similarity between phenomena. This is a particularly useful skill for scientists, teachers, artists, writers, poets and public thinkers, but it shows up all over the place. Many internet memes, for example, involve drawing analogies: seeing the structural connections between unrelated things.
One of my favourites is a meme on twitter called #sameguy. It started as a game of uploading pictures of two celebrities that resemble each other, followed by the hashtag #sameguy. But it evolved to include abstract ideas and phenomena that are the “same” in some respect. Making cultural metaphors like this requires creativity, as does understanding them. One has to free one’s mind of literal-mindedness in order to temporarily ignore the ever-present differences between things and focus on the similarities.
Here’s a blog that collects #sameguy submissions: Same Guy
On twitter you sometimes come across more imaginative, analogical #sameguy posts: #sameguy – Twitter Search
The topic of metaphor and analogy is one of the most fascinating aspects of intelligence, in my opinion. I think it’s far more important that coming up with theories about ‘consciousness’. 🙂 Check out this answer:
Why are metaphors and allusions used while writing?
(This Quora answer is a cross-post of a blog post I wrote: Metaphor: the Alchemy of Thought)
In one sense metaphor and analogy are central to scientific research. I’ve written about this here:
What are some of the most important problems in computational neuroscience?
Science: the Quest for Symmetry
This essay is tangentially related to the topic of creativity and patterns:
From Cell Membranes to Computational Aesthetics: On the Importance of Boundaries in Life and Art
* The brain’s outputs — commands to muscles and glands — are closely linked with pattern recognition too. What you choose to do depends on what you can do given your intentions, circumstances, and bodily configuration. The state that you and the universe happen to be in constrains what you can do, and so it is useful for the brain to recognize and categorize the state in order to mediate decision-making, or even non-conscious behavior.When you’re walking on a busy street, you rapidly process pathways that are available to you. even if you stumble, you can quickly and unconsciously act to minimize damage to yourself and others. Abilities of this sort suggest that pattern recognition is not purely a way to create am ‘image’ of the world, but also a central part of our ability to navigate it.
My latest 3QD essay is a bit of a wild one. I start by talking about Goodhart’s Law, a quirk of economics that I think has implications elsewhere. I try to link it with neuroscience, but in order to do so I first construct an analogy between the brain and an economy. We might not understand economic networks any better than we do neural networks, but this analogy is a fun way to re-frame matters of neuroscience and society.
Plan of a Citadel (from Wikipedia)
Here’s an excerpt:
The Neural Citadel
Nowadays we routinely encounter descriptions of the brain as a computer, especially in the pop science world. Just like computers, brains accept inputs (sensations from the world) and produce outputs (speech, action, and influence on internal organs). Within the world of neuroscience there is a widespread belief that the computer metaphor becomes unhelpful very quickly, and that new analogies must be sought. So you can also come across conceptions of the brain as a dynamical system, or as a network. One of the purposes of a metaphor is to link things we understand (like computers) with thing we are still stymied by (like brains). Since the educated public has plenty of experience with computers, but at best nebulous conceptions of dynamical systems and networks, it makes sense that the computer metaphor is the most popular one. In fact, outside of a relatively small group of mathematically-minded thinkers, even scientists often feel most comfortable thinking of the brain as a elaborate biological computer. [3]
However, there is another metaphor for the brain that most human beings will be able to relate to. The brain can be thought of as an economy: as a biological social network, in which the manufacturers, marketers, consumers, government officials and investors are neurons. Before going any further, let me declare up front that this analogy has a fundamental flaw. The purpose of metaphor is to understand the unknown — in this case the brain — in terms of the known. But with all due respect to economists and other social scientists, we still don’t actually understand socio-economic networks all that well. Not nearly as well as computer scientists understand computers. Nevertheless, we are all embedded in economies and social networks, and therefore have intuitions, suspicions, ideologies, and conspiracy theories about how they work.
Because of its fundamental flaw, the brain-as-economy metaphor isn’t really going to make my fellow neuroscientists’ jobs any easier, which is why I am writing about it on 3 Quarks Daily rather than in a peer-reviewed academic journal. What the brain-as-economy metaphor does do is allow us to translate neural or mental phenomena into the language of social cooperation and competition, and vice versa. Even though brains and economies seem equally mysterious and unpredictable, perhaps in attempting to bridge the two domains something can be gained in translation. If nothing else, we can expect some amusing raw material for armchair philosophizing about life, the universe, and everything. [4]
So let’s paint a picture of the neural economy. Imagine that the brain is a city — the capital of the vast country that is the body. The neural citadel is a fortress; the blood-brain barrier serves as its defensive wall, protecting it from the goings-on in the countryside, and only allowing certain raw materials through its heavily guarded gates — oxygen and nutrients, for the most part. Fuel for the crucial work carried out by the city’s residents: the neurons and their helper cells. The citadel needs all this fuel to deal with its main task: the industrial scale transformation of raw data into refined information. The unprocessed data pours into the citadel through the various axonal highways. The trucks carrying the data are dispatched by the nervous system’s network of spies and informants. Their job is to inform the citadel of the goings-on outside its walls. The external sense organs — the eyes, ears, nose, tongue and skin — are the body’s border patrols, coast guards, observatories, and foreign intelligence agencies. The muscles and internal organs, meanwhile, are monitored by the home ministry’s police and bureaucrats, always on the look-out for any domestic turbulence. (The stomach, for instance, is known to be a hotbed of labor unrest.)
The neural citadel enables an information economy — a marketplace of ideas, as it were. Most of this information is manufactured within the brain and internally traded, but some of it — perhaps the most important information — is exported from the brain in the form of executive orders, requests and the occasional plaintive plea from the citadel to the sense organs, muscles, glands and viscera. The purpose of the brain is definitely subject to debate — even within the citadel — but one thing most people can agree on is that it must serve as an effective and just ruler of the body: a government that marries a harmonious domestic policy — unstressed stomach cells, unblackened lung cells, radiant skin cells and resilient muscle cells — with a peaceful and profitable foreign policy. (The country is frustratingly dependent on foreign countries, over which it has limited control, for its energy and construction material.)
The citadel is divided into various neighborhoods, according to the types of information being processed. There are neighborhoods subject to strict zoning requirements that process only one sort of information: visions, sounds, smells, tastes, or textures. Then there are mixed use neighborhoods where different kinds of information are assembled into more complex packages, endlessly remixed and recontextualized. These neighborhoods are not arranged in a strict hierarchy. Allegiances can form and dissolve. Each is trying to do something useful with the information that is fed to it: to use older information to predict future trends, or to stay on the look-out for a particular pattern that might arise in the body, the outside world, or some other part of the citadel. Each neighborhood has an assortment of manufacturing strategies, polling systems, research groups, and experimental start-up incubators. Though they are all working for the welfare of the country, they sometimes compete for the privilege of contributing to governmental policies. These policies seem to be formulated at the centers of planning and coordination in the prefrontal cortex — an ivory tower (or a corporate skyscraper, if you prefer merchant princes to philosopher kings) that has a panoramic view of the citadel. The prefrontal tower then dispatches its decisions to the motor control areas of the citadel, which notify the body of governmental marching orders.
~
The essay is not just about the metaphor though. There are bits about dopamine, and addiction, and also some wide-eyed idealism. 🙂 Check the whole thing out at 3 Quarks Daily.
For the record, there is a major problem with personifying neurons. It doesn’t actually explain anything, since we are just as baffled by persons as we are by neurons. Personifying neurons creates billions of microscopic homunculi. The Neural Citadel metaphor was devised in a spirit of play, rather than as serious science or philosophy.
There are quite a few problems that computational neuroscientists need to solve in order to achieve a true theoretical understanding of biological intelligence. But I’d like to talk about one problem that I think is the holy grail of computational neuroscience and artificial intelligence: the quest for invariance. From a purely scientific and technological perspective I think this is a far more important and interesting problem than anything to do with the “C-word”: Consciousness. 🙂
Human (and animal) perception has an extraordinary feature that we still can’t fully emulate with artificial devices. Our brains somehow create and/or discover invariances in the world. Let me start with a few examples and then explain what invariance is.
Invariance in vision
Think about squares. You can recognize a square irrespective of it’s size, color, and position. You can even recognize a square with reasonable accuracy when viewing it from an oblique angle. This ability is something we take for granted, but we haven’t really figured it out yet.
Now think about human faces. You can recognize a familiar face in various lighting conditions, and under changes of facial hair, make-up, age, and context. How does the brain allow you to do things like this?
Invariance in hearing
Think about a musical tune you know well. You will probably be able to recognize it even if it is slowed down, sped up, hummed, whistled, or even sung wordlessly by someone who is tone-deaf. In some special cases, you can even recognize a piece of music from its rhythmic pattern alone, without any melody. How do you manage to do this?
Think about octave equivalence. A sound at a particular frequency sounds like the same note as a sound at double the frequency. In other words, notes an octave apart sound similar. What is happening here?
What is invariance?
How does your brain discover similarity in the midst of so much dissimilarity? The answer is that the brain somehow creates invariant representations of objects and patterns. Many computational neuroscientists are working on this problem, but there are no unifying theoretical frameworks yet.
So what does “invariance” mean? It means “immunity to a possible change”. It’s related to the formal concept of symmetry. According to mathematics and theoretical physics, an object has a symmetry if it looks the same even after a change. a square looks exactly the same if you rotate it by 90 degrees around the center. We say it is invariant (or symmetrical) with respect to a 90 degree rotation.
Our neural representations of sensory patterns somehow allow us to discover symmetries and using them for recognition and flexible behavior. And we manage to do this implicitly, without any conscious effort. This type of ability is limited and it varies from person to person, but all people have it to some extent.
Back to the examples
We can redefine our examples using the language of invariance.
All these invariances are partial and limited in scope, but they are still extremely useful, and far more sophisticated than anything we can do with artificial systems.
Invariance of thought patterns?
The power of invariance is particularly striking when we enter the domain of abstract ideas — particularly metaphors and analogies.
Consider perceptual metaphors. We can touch a surface and describe it as smooth. But we can also use the word “smooth” to describe sounds. How is it that we can use texture words for things that we do not literally touch?
Now consider analogies, which are the more formal cousins of metaphors. Think of analogy questions in tests like the GRE and the SATs. Here’s an example
Army: Soldier :: Navy : _____
The answer is “Sailor”.
These questions take the form “A:B::C:D”, which we normally read as “A is to B as C is to D”. The test questions normally ask you to specify what D should be.
To make an analogy more explicit, we can re-write it this way: “R(x,y) for all (x,y) = (A,B) or (C,D)”. The relation “R” holds for pairs of words (x,y), and in particular, for pairs (A,B) as well as (C,D).
In this example, the analogical relationship R can be captured in the phrase “is made up of”. An army is made up of soldiers and a navy is made up of sailors. In any analogy, we are able to pick out an abstract relationship between things or concepts.
Here’s another example discussed in the Wikipedia page on analogy:
Hand: Palm :: Foot: _____
The answer most people give is “Sole”. What’s interesting about this example is that many people can understand the analogy without necessarily being able to explain the relationship R in words. This is true of various analogies. We can see implicit relationships without necessarily being able to describe them.
We can translate metaphors and analogies into the language or invariance.
The words “metaphor” and “analogy” are essentially synonyms for the word “invariant” in the domains of percepts and concepts. Science, mathematics and philosophy often involve trying to make explicit our implicit analogies and metaphors.
Neuroscience, psychology and cognitive science aim to understand how we form these invariant representations in the first place. In my opinion doing so will revolutionize artificial intelligence.
Further reading:
I’ve only scratched the surface of the topic of invariance and symmetry.
I talk about symmetry and invariance in this answer too:
Mathematics: What are some small but effective theses or ideas in mathematics that you have came across? [Quora link. Sign-up required]
I talk about the importance of metaphors in this blog post:
Metaphor: the Alchemy of Thought
I was introduced to many of these ideas through a book by physicist Joe Rosen called Symmetry Rules: How Science and Nature Are Founded on Symmetry. It’s closer to a textbook that a popular treatment, but for people interested in the mathematics of symmetry and group theory, and how it relates to science, this is an excellent introduction. Here is a summary of the book: [pdf]
Relatively recent techniques such as deep learning have helped artificial systems form invariant representations. This is how facial recognition software used by Google and Facebook work. But these algorithms still don’t have the accuracy and generality of human skills, and the way they work, despite being inspired by real neural networks, is sufficiently unlike real neural processes that these algorithms may not shed much light on how human intelligence works.
Notes:
This post is a slightly edited form of a Quora answer I wrote recently.
In the comments section someone brought up the idea that some invariants can be easily extracted using Fourier decomposition. This is what I said is response:
Good point. Fourier decomposition is definitely part of the story (for sound at the very least), but it seems there is a lot more.
Some people think that the auditory system is just doing a Fourier transform. But this was actually shown to be partially false a century ago. The idea that pitch corresponds to the frequencies of sinusoids is called Ohm’s acoustic law.
From the wiki page:
For years musicians have been told that the ear is able to separate any complex signal into a series of sinusoidal signals – that it acts as a Fourier analyzer. This quarter-truth, known as Ohm’s Other Law, has served to increase the distrust with which perceptive musicians regard scientists, since it is readily apparent to them that the ear acts in this way only under very restricted conditions.
—W. Dixon Ward (1970)
This web page discusses some of the dimensions other that frequency that contribute to pitch:
Introduction to Psychoacoustics – Module 05
There are interesting aspects of pitch perception that render the Fourier picture problematic. For example, there is the Phenomenon of the missing fundamental: “the observation that the pitch of a complex harmonic tone matches the frequency of its fundamental spectral component, even if this component is missing from the tone’s spectrum.”
Evidence suggests that the human auditory system uses both frequency and time/phase coding.
Missing fundamental: “The brain perceives the pitch of a tone not only by its fundamental frequency, but also by the periodicity of the waveform; we may perceive the same pitch (perhaps with a different timbre) even if the fundamental frequency is missing from a tone.”
This book chapter also covers some of the evidence: [pdf]
” One of the most remarkable properties of the human auditory system is its ability to extract pitch from complex tones. If a group of pure tones, equally spaced in freque ncy are presented together, a pitch corresponding to the common frequency distance between the individual components will be heard. For example, if the pure tones with frequencies of 700, 800, and 900 Hz ar e presented together, the result is a complex sound with an underlying pitch corresponding to that of a 100 Hz tone. Since there is no physical energy at the frequency of 100 Hz in the complex, such a pitch sensation is called residual pitch or virtual pitch (Schouten 1940; Schouten, Ritsma and Cardozo, 1961). Licklider (1954) demonstrated that both the plac e (spectral) pitch and the residual (virtual) pitch have the same properties and cannot be auditorally differentiated.”
The status of Fourier decomposition in vision might be more controversial. Spatial frequency based models have their adherents, but also plenty of critics. One of my professors says that claiming the visual system does spatial Fourier amounts to confusing the object of study with the tools of study. 🙂 We still don’t whether and how the brain performs spatial Fourier decomposition.
A very recent paper reviews this issue:
The neural bases of spatial frequency processing during scene perception
“how and where spatial frequencies are processed within the brain remain unresolved questions.”
Vision scientists I know often talk about how the time domain cannot be ignored in visual processing.
A general point to be made is that even if we have mathematical solutions that are invariant, computational neuroscientists haven’t quite figured out how neural networks achieve such invariant representations. The quest for invariance is more about plausible neural implementation than mathematical description per se.
Neurologism 