Minds and Markets

These are heady days for cognitive scientists. Arcana such as “The Stuff of Thought” or “Musicophilia” top bestselling lists. Steven Pinker’s article on the moral sense in the New York Times Magazine is instructive. He explains that moral reasoning is not to be thought of as a set of beliefs or feelings about the world but rather a lens to view the world through, very much as our more salient senses are. Moral reasoning is typically performed automatically and justified retroactively. What’s striking about this article is that it represents a new approach for a timeless and elusive aspect of cognition. The computational approach to cognition and the dance of neuroscience, evolutionary biology and economics has been tremendously productive in generating theories and predictions by recognizing the best approach asks not only what brains do but what they are for. “Why Choose This Book” is a wonderful contribution to this tradition. Psychologists tend to suffer from physics envy, but Montague has happily avoided this because he is by training a biophysicist and now a neuroscientist. His energy and enthusiasm for physics, computer science, economics and psychology gives him a quite original approach to thinking about what the mind is for, and he does an excellent job of guiding us step by step to this new conception that is driving research today.

Until our language and reasoning changes a great deal, it will still be most convenient to use Cartesian dualism, even for scientists (”here we see a picture of my brain” is an odd thing for a brain to say) to informally discuss the mind. Most attempts at explaining cognition have attempted to dissect it without reference to its function. The functional approach in science, ushered in by William Harvey’s question of what valves in veins are for, is one of the great triumphs of human reasoning. Functional approaches to the mind are much younger. Charles Darwin provided the foundation for the modern conception of mind by showing that evolution is the method for imbuing matter with goals. Alan Turing provided the conceptual revolution that showed how mechanistic processes, including biological ones, are about manipulating information. Evolution is an algorithm for producing propagating entities working on their own survival.

With these two innovations, cognitive science found a solid foothold. But somehow this is not enough to say what the mind is for. Is cognition about learning what is true about the world? Hard to say, but Donald Hoffman, a vision scientist who’s put much thought into this thinks not:

“It used to be hard to imagine how perceptions could possibly be useful if they were not true. Now, thanks to technology, we have a metaphor that makes it clear — the windows interface of the personal computer. This interface sports colorful geometric icons on a two-dimensional screen. The colors, shapes and positions of the icons on the screen are not true depictions of what they represent inside the computer. And that is why the interface is useful. It hides the complexity of the diodes, resistors, voltages and magnetic fields inside the computer. It allows us to effectively interact with the truth because it hides the truth.”

In “On Intelligence”, Jeff Hawkins organizes intelligence around prediction. While certainly a useful explanation for explaining lots of properties of information processing and collection, this misses some key aspects of cognition. Unless prediction as a concept is so loose as to be meaningless, in what sense are our social interactions about prediction? How do we model other intelligent beings? It’s reasonable to conclude the primary reason I wish to know other minds is so I can take advantage of their experience, but to what end? The reason why intelligence is a property of the intelligent creatures we observe is that they are products of evolution.

Cognition is not some quirky preference of the algorithm, but rather the consequence of creatures with limited lifespans seeking to navigate the world and obtain resources. Creatures do not simply compute, they care about survival. Because life and resources are necessarily limited, organisms must make choices. As Montague points out, this means decisions are not disembodied from the constraints of the flesh, as Descartes would have it, but rather are inextricable linked to the goals of the organism. The key to navigating the world then, is more than just prediction. It’s valuation. Prediction, in the sense I am trying to malign, resembles a gentleman-scholar pinning butterflies in his study. Valuation comes from the hungry kid on the floor of the Merc - his nervous hands, his eye on the clock, the elbows all around. Evolution looks through his eyes.

Our gentleman-scholar is free to pursue his goals without consideration of cost. If there is a rare butterfly, cost is no object to mounting an expedition, even if this rare butterfly may add little knowledge of its genus. Montague considers computation without consideration of cost to be an odd and perhaps unfortunate legacy of the Bletchley Park lads (including Turing) who poured as much energy as they could to crack Enigma, the encryption system used by the German U-boats strangling Britain. Our young trader is acutely aware of the opportunity costs of his actions (even taking a bathroom break). Further, knowledge-gathering is not arbitrary. Our gentlemen scholar could just as easily study weather-formations at his cabin. The information the trader gathers directly impacts whether he will be able to return the next day, and indeed, he is rewarded for extracting extra information others have not considered. Traders and poker players alike seek “tells”, tell-tale signs of another’s behavior.

Montague refers to these properties of evolved creatures as “efficient computation”, where “instead of just the computation, there’s the computation plus ’something else’ and that ’something else’ is a measure of the value of that computation to the overall success of the organism”. Organisms possess goals, and guidance signals are feedback mechanisms used to provide suggestions on approaches and provide updates on progress. Valuations not only dynamically represent the expectations of achieving a goal but also organize the algorithms available to accomplish them. How does an organism compare algorithms? One way familiar to computer scientists of evaluating the efficiency of an algorithm is to compare it to other algorithms capable of producing an equivalent result. Evolution then places a premium on the ability to simulate algorithms, to navigate counterfactuals. Underlying this capacity is the ability to model other modules (individual components or steps that comprise cognition as a whole), which may be the happy by-product of modules that operate too fast for feedback and must simply provide related modules with a model of it’s actions. Its important to pay attention to the absence of the humonculus from here on out: to every module, *everything* else is “out there”, whether originating in the brain or in the outside world.

Valuation is necessary in an uncertain world. In order to build models, modules have to come to be able to extract information in spite of the noise inherent to the world. Reverend Bayes asked how, from a limited sampling of data, could we infer if there was a rule producing the data we observed. We’re all Bayesians now, and modelers must be able subtract noise from signal. Modules must recognize when there is opportunity to refine models (efficiently, of course) and the limitations inherent in a noisy world. Montague permits a illuminating aside: this noise may be appropriated for an excellent purpose. Organisms can use this noise to vary behavior, as Montague’s rabbit rehearses slightly random escape routes. Organisms engaging in exchange may wish to broadcast that they are unable to possess full models of their own behavior, eliminating any tells to be exploited. I wonder if organisms might actually prefer to trade with such partners - smart opponents last, and trade is beneficial.

Back to valuation. How to compare the effectiveness of one algorithm or another? Montague uses the example of trying to figure out how to access fruit on the ledge of a rock face. One can imagine various ways to attempt to grab the fruit. But none of the successful ones will involve flying DeLoreans or stretching limbs. Physical reality, and the invariance inherent to it is necessary for successful modelling. We already know we possess intuitive theories (that is, bodies of information) of physics involving object constancy, persistence throughout time and invariance to points of view. Montague makes a breathtaking observation: each of these concepts translates into the three laws of conservation of linear momentum, energy and angular momentum - the foundations of classical physics! That details about the physical world that corroborate with our scientific knowledge and is instantiated in the mind is both shocking and obvious. Another implication of invariance is by definition it holds true for what other organisms possess, so social learning is an effective way of gathering information about the world - model others as if they have similar goals and perceptions and you put yourself in their shoes - it could have happened to you. Empathy then, just makes sense.

Efficient computation requires guidance signals to provide information not only about how close we are to the goal (”getting warmer…”) but also about expectations about future rewards (”statistically, having this warmth means a forty percent likelihood of 30 units of…”). To help us choose efficient procedures, guidance signal is available for simulated models. And these guidance signals are not just a hypothetical construct, they match the behavior of the dopamine system. A mere 15-20,000 neurons (compared to a hundred billion in the entire brain) in the midbrain match the computational and physiological requirements of a system capable of acting on brain systems processing goals, notably the prefrontal cortex. We can interpret increases in activity corresponding to “reward is larger than expected”, pauses as “reward is less than expected” and no change as “reward is just as expected”.

Rewards do not come at once, they are distributed over time. Resources also do not come with labels such as “eat me”. As our trader, we are seeking information that predicts rewards. Dopamine helps us to this. When a light flashes, without any impact on our valuation of the world this is treated as irrelevant to our goals and dopamine produces a “things are as expected” signal. But if this light begins to consistently precede a reward, such as juice, when the light comes on dopamine releases a “better than expected” signal. In a sense, the future value of the reward is transferred to the light. Suddenly, we know something about the distribution of rewards in the world. Not only does this act of valuation help us predict rewards in the world, but it gives us a common currency for comparing them. As currency, these signals are not only stores of value but a means of exchange. Actions can be compared and simulated. Just as barter is inefficient if I do not want a whole goat or bale of cotton, neural currency smoothes out the potential comparisons to be made. Just as money can be used anywhere, proxies of value can be plugged into any model to compare potential returns.

But helping us choose actions by predicting rewards is only part of what dopamine does. What is rewarding is also constrained by what our goals are. According to the dopamine gating hypothesis, dopamine does indeed help us select goals. And it does this through the same mechanism it uses for stimuli in the world (remember, no humonculus): it learns that certain goals are predictive of rewards. Thus, a twenty dollar bill on the ground I see while walking to the bank is only predictive of twenty dollars extra in my pocket when my goal changes to “pick up twenty dollar bill”. Montague argues that the ability for humans to pursue abstract goals is by the hijacking of rewards by ideas: ideas become intrinsically valuable. Of course, the ability to do so can be extraordinarily positive, such as running a marathon for charity, or totally debilitating, as in drug addiction.

Montague makes exploring this “superpower” a focal element of his book, which is somewhat difficult to make compelling since as he acknowledges he cannot account for how certain ideas can hijack or why we are not continuously debilitated by alphabetizing our underwear or scrubbing our toenails. I would like a better account of how the content of something as a biological primitive such as goals of homeostasis can be equivalent to (and thus substitute for) an idea. This book is an odd mix of insightful stepwise building of a model with gaps in the details of the mechanisms involved. The gaps are not problematic because it is evident these are problems, not mysteries, and perhaps soon we will know much more about what other powerful and efficient computations we perform.

I was particularly intrigued by the connections to concepts in economics and finance. When Montague discussed the problem organisms face in allocating energy to either updating models or exercising them, I was reminded of the question of corporate dividend policy (reinvest extra profits or reward shareholders with it?). Regarding trade, only organisms with random and thus unpredictable behavior remain because predictable partners are exploited out of the market very much resembles the efficient markets hypothesis. I’d like some clarifications on the proxies of value the dopamine circuits produce. What is the connection between the valuation of the proxies (derivatives) and the underlying signals (underlying asset)? Many other cognitive science books have capitalized on developments in evolutionary biology, economics and computer science but few have as original insights as this book.

This entry was posted on Monday, January 14th, 2008 at 3:16 pm and is filed under Book Reviews. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.