Learning to Trade: The Psychology of Expertise

This article is written by Dr. Brett N. Steenbarger and published on www.brettsteenbarger.com . He is one of the best trading psychologist and I really enjoy his articles.

 Learning to Trade:  The Psychology of Expertise             

When people hear that I am an active trader and a professional psychologist, they naturally want to hear about techniques for mastering emotions in trading.  That is an important topic to be sure, and later in this article I will even have a few things to say about it.  But there is much more to psychology and trading than “trading psychology”, and that is the ground I hope to cover here.  Specifically, I would like to address a surprisingly neglected issue:  How does one gain expertise as a trader? 

It turns out that there are two broad answers to this question, focusing upon quantitative and qualitative insights into the markets.  We can dub these research expertise and pattern-recognition expertise, respectively.  These perspectives are much more than academic, theoretical issues.  How we view knowledge and learning in the markets will shape the strategies we employ and—quite likely—the results we will obtain.  In this article, I will summarize these two positions and then offer a third, unique perspective that draws upon recent research in the psychology of learning.  I believe this third perspective, based on implicit learning, has important, practical implications for our development as traders.

Developing Expertise Through Research

The research answer to our question says that we gain trading expertise by performing superior research.  We collect a database of market behavior and then we research variables (or combinations of variables) that are significantly associated with future price trends.  This is the way of mechanical trading systems, as in the trading strategies developed with TradeStation and the systems featured on the FuturesTruth.com site.  We become expert, the mechanical system trader would argue, by building a better mousetrap—finding the system with the lowest drawdown, least risk, greatest profit, etc.

A variation of the research answer can be seen in traders who rely on data-mining strategies.  The data-miner questions whether there can be a single system appropriate for all markets or appropriate for all time frames.  To use a phrase popularized by Victor Niederhoffer, the market embodies “ever-changing cycles”.  The combination of predictors that worked in the bull market of 2000 may be disastrous a year later.  The data-miner, therefore, engages in continuous research: modeling and remodeling the markets to capture the changing cycles.  Tools for data mining can be found at kdnuggets.com. 

There are hybrid strategies of research, in which an array of prefabricated mechanical systems are defined and then applied, data-mining style, to individual stocks to see which ones have predictive value at present.  This is the approach of “scanning” software, such as Nirvana Systems’ OmniTrader.  By scanning a universe of stocks and indices across an array of systems, it is possible to determine which systems are working best for which instruments.

As most traders are aware, the risk of research-based strategies is that of overfitting.  If you define enough parameters and time periods, eventually you’ll find a combination that predicts the past very well—by complete chance.  It is not at all unusual to find an optimized research strategy that performs poorly going forward.  Reputable researchers develop and test their systems on independent data sets, so as to demonstrate the reliability of their findings.

Can quantitative, research-based strategies capture market expertise?  I believe the answer is an unequivocal “Yes!”  A perusal of the most successful hedge funds reveals a predominance of “quant shops”.  Several research-based stock selection strategies, such as Jon Markman’s seasonal patterns (MoneyCentral.com) and the Value Line system, exhibit long-term track records that defy mere chance occurrence. 

And yet it is also true that many successful traders neither rely upon mechanical systems nor data-mining.  Indeed, one of Jack Schwager’s most interesting findings in his Market Wizards interviews was that the expert traders employed a wide range of strategies.  Some were highly quantitative; others relied solely upon discretionary judgment.  Several of the most famous market participants—Warren Buffet and Peter Lynch, for example—employed research in their work, but ultimately based their decisions upon their judgment: their personal synthesis of this research.   Quantitative strategies can capture market expertise, but it would appear that all market expertise cannot be reduced to numbers.

Developing Expertise Through Pattern Recognition
 

The second major answer to the question of trading expertise is that of pattern recognition.  The markets display patterns that repeat over time, across various time-scales.  Traders gain expertise by acquiring information about these patterns and then learning to recognize the patterns for themselves.  An analogy would be a medical student learning to diagnose a disease, such as pneumonia.  Each disease is defined by a discrete set of signs and symptoms.  By running appropriate tests and making proper observations of the patient, the medical student can gather the information needed to recognize pneumonia.  Becoming an expert doctor requires seeing many patients and gaining practice in putting the pieces of information together rapidly and accurately.

The clearest example of gaining trading expertise through pattern recognition is the large literature on technical analysis.  Most technical analysis books are like the books carried by medical students.  They attempt to group market “signs” and “symptoms” into identifiable patterns that help the trader “diagnose” the market.  Some of the patterns may be chart patterns; others may be based upon the identification of cycles, configurations of oscillators, etc.  Like the doctor, the technical analyst cultivates expertise by seeing many markets and learning to identify the patterns in real time. 

Note how the pattern recognition answer to the question of expertise leads to a very different approach to the training of traders.  In the research perspective, traders learn to improve their trading by conducting better research.  This means learning to use more sophisticated tools, gather more data, uncover better predictors, etc.  From a pattern recognition vantage point, however, trading success will not come from doing more research.  Rather, direct instruction from experts and massed practice leads to the development of competence (again, like medical school, where the dictum is “See one, do one, teach one”).

Another way of stating this is that the research answer treats trading as a science.  We gain knowledge by uncovering new observations and patterns.  The pattern recognition answer treats trading as a performance activity.  We gain proficiency through mentoring and constant practice.  This is the way of the athlete, the musician, and the craftsperson.

Can expertise be acquired by learning patterns from others and then gaining experience identifying them on one’s own?  It would seem so: this is traditionally how chess champions and Olympic athletes develop.  There are also examples of such expertise development in trading: Linda Raschke’s chatroom (www.mrci.com/lbr) is an excellent example of a learning device that takes the pattern recognition approach.  Users of the site can “listen in” as Linda—a Market Wizard trader herself—identifies market patterns in real time.  My conversations with traders who have enrolled in this service leave me with little doubt that they have acquired profitable skills, eventually moving on to becoming successful independent traders.  Richard Dennis’ experiment with the “Turtles” is perhaps the most famous example of how expertise (in this case, a pattern-based trading system) can be modeled for people with little market background and yield winning results.

And yet there are nagging doubts about the actual value of the patterns typically described in market books and tapes.  A comprehensive investigation of technical analysis strategies ____ found very little evidence for their effectiveness.  An attempt to quantify technical analysis patterns by Andrew Lo at MIT found that they did, indeed, contain information about future market moves, but hardly as much as is generally portrayed.  Because pattern recognition entails a healthy measure of judgment, it is very difficult to demonstrate its efficacy outside of the expert’s hands.  In other words, the expert trader may be utilizing more information in trading than he or she can verbalize.  This is certainly the case for chess experts and athletes.  While they can describe what they are doing, it is clear that their proficiency extends well beyond the application of a limited set of rules or patterns.

This phenomenon has been the subject of extensive study in psychotherapy research.  It turns out that there really is a difference in results between expert therapists and novices.  But it also turns out that there is a difference between what expert therapists say they do and what they actually do in their sessions.  This was noted as far back as the days of Freud.  While he advocated a set of strict therapeutic procedures to be followed, his own published cases deviated from these significantly.  What appears to work in therapy is not what the therapists focus on—their behavioral techniques, psychoanalytic methods, etc.—but the ways in which these are employed.  Using any techniques in a sensitive way that gains the client’s trust and fits with the client’s understandings is more important than the procedures embodied by any of the techniques.

So it may well be with trading.  Expert traders may describe their work in terms of price-volatility patterns, momentum divergences, short-skirt patterns, or a nesting of cycles, but it might be the ways in which these patterns are employed that makes for the expertise.  Great traders may be able to identify patterns in their work, but it is not clear that their greatness lies in their patterns.

Implicit Learning: A New Perspective

The term implicit learning began with the research of Brooklyn College’s Arthur Reber in the mid 1960s.  Since that time, it has been an active area of investigation, producing numerous journal articles and books.

Implicit learning can be contrasted with the research and pattern recognition perspectives described above, in that the latter are examples of explicit learning.  By conducting research or by receiving instruction in market patterns, we are learning in a conscious, intentional fashion.  The implicit learning research suggests that much of the expertise we acquire is the result of processes that are neither conscious nor intentional.

A simple example drawn from Reber’s work will illustrate the idea.  Suppose I invent an artificial “grammar”.  In this grammar, there are rules that determine which letters can follow given letters and which cannot.  If I use a very simple grammar such as
MQTXG, then every time I show a subject the letter M, it should be followed by a Q; every time I flash a T, it should be followed by an X. 

The key in the research is that subjects are not told the rules behind the grammar in advance.  They are simply shown a letter string (QT, for example) and asked whether it is “grammatical” or not.  If they get the answer wrong, they are given the correct answer and then shown another string.  This continues for many trials.

Interestingly, the subjects eventually become quite proficient at distinguishing the grammatical strings from the ungrammatical ones.  If they are shown a TX, they know this is right, but that TG is not.  Nevertheless, if you ask the subjects to describe how they know the string is grammatical or not, they cannot verbalize any set of cogent rules.  Indeed, many subjects insist that the letter arrangements are random—even as they sort out the grammatical ones from the ungrammatical ones with great skill.

Reber referred to this as implicit learning, because it appeared that the subjects had truly learned something about the patterns presented to them, but that this learning was not conscious and self-directed.  Reber and subsequent researchers in the field, such as Axel Cleeremans in Brussels, suggest that many performance skills, such as riding a bicycle and learning a language, are acquired in just this way.  We learn what to do, even with great proficiency, but cannot fully verbalize what we know or reduce our knowledge to a set of patterns or principles. 

Such implicit learning has been demonstrated in the laboratory across a variety of tasks.  Cleeremans and McClelland, for example, flashed lights on a computer screen for subjects, with the lights appearing at six different places on the screen.  The subjects had to press a keyboard button corresponding to the location of the light on the screen.  There were complex rules determining where the light would flash, but these rules were not known by the subjects.  After thousands of trials, the subjects became very good at anticipating the location of the light, as demonstrated by reduced response times.  Significantly, when the lights were flashed on the screen in a random pattern, no such reduction in response time was observed.  This was a meaningful finding, since the patterns picked up by the subjects were not only outside their conscious awareness—they were also mathematically complex and beyond the subjects’ computational abilities!  (Like the markets, the patterns were actually “noisy”—a mixture of patterns and random events.)

It appears that much repetition is needed before implicit learning can occur.  The thousands of trials in the Cleeremans and McClelland study are not unusual for this research.  Moreover, it appears that the state of the subjects’ attention is crucial to the results.  In a research review, Cleeremans, Destrebeckqz, and Boyer report that, when subjects perform the learning tasks with divided attention, the implicit learning suffers greatly.  (Interestingly, conscious efforts to abstract the rules from the stream of trials also interfere with learning).  This has led Cleeremans to speculate that implicit learning is akin to the learning demonstrated by neural networks, in which complex patterns can be abstracted from material through the presentation of numerous examples.

The implicit learning research suggests a provocative hypothesis:  Perhaps expertise in trading is akin to expertise in psychotherapy.  While therapists say their work is grounded in research and makes use of theory-based techniques, the actual factors that account for positive results are implicit, and acquired over the course of years of working with patients.  Similarly, traders may attribute their results to the research or patterns they are trading.  In reality, however, the research and the patterns are simply “cover stories” that legitimize seeing many markets over the period of years.  It is the implicit learning of markets over thousands of “trials” that makes for expertise, not the conscious strategies that traders profess.

Implications for Developing Expertise in the Markets

Such an implicit learning perspective helps to make sense of Schwager’s findings.  There are many ways of becoming immersed in the markets: through research, observation of charts, tape reading, etc.  The specific activity is less important than the immersion.  We become experts in trading in the same way that subjects learned Reber’s artificial grammars.  We see enough examples under sufficient conditions of attention and concentration that we become able to intuit the underlying patterns.  In an important sense, we learn to feel our market knowledge before we become able to verbalize it.   While simply “going with your feelings” is generally a recipe for trading disaster, I believe it is also the case that our emotions and “gut” feelings can be an important source of market information.

The reason for this is tied up in the neurobiology of the brain.  In his excellent text The Executive Brain: Frontal Lobes and the Civilized Mind, New York University’s Elkhonon Goldberg summarizes evidence that suggests a division of labor for the hemispheres of our brains.  Our right, nonverbal hemispheres become activated when we encounter novel stimuli and information.  Our left, verbal hemispheres are more active in processing routine knowledge and situations.  When we first encounter new situations, as in the markets, we will tend to process the information non-verbally—which means implicitly.  Only when we have made these patterns highly familiar will there be a transfer to left hemisphere processing and an ability to capture, in words, some of the complexity of one’s understandings.  As we know from studies of regional cerebral blood flow, the right hemisphere is also activated under emotional conditions.  It is not surprising that our awareness of novel patterns, whether in artificial grammars or in markets, would appear as felt tendencies rather than as verbalized rules.

So now we get to the traditional domain of the trading psychologist!  How do we know when our feelings are conveying real information for trading and when they are interference from our conflicts over success/failure, risk/safety, etc.?  It is not so simple as “tune out your emotions when you are trading”.  Much of what you might know about the markets may take the form of implicit knowledge that is encoded nonverbally.

This is an area that I am currently researching, and I welcome readers to stay in touch with me about the results.  I will make sure updated information is posted in a timely way to my personal page at www.greatspeculations.com.  I also hope to have my own book out on the topic early in 2003; my page will also keep readers abreast of that development.  But in the remainder of this article, allow me to engage in a few speculations of my own regarding the implications of implicit learning for trading success.

1. Many are called, few are chosen – I believe the implicit learning perspective helps to explain why so few traders ultimately succeed at their craft.  Quite simply, they cannot outlast their learning curves.  If, indeed, it takes thousands of trials to generate successful implicit learning, a great number of traders would have been bankrupted by then.  Many others might not survive that number of trials simply due to the time and energy required.  It is impossible to hold a full-time job and generate the degree of immersion in the markets needed for implicit learning.  On the other hand, it is impossible to obtain a full-time income from trading without developing the mastery conferred by years of experience.  Part-time traders never develop expertise for the same reason that part-time chess players or athletes are unlikely to succeed.  For purely practical reasons associated with raising a family, making a living, etc., few people can undergo the “starving artist” phase of skill-building.

2. Emotions interfere with trading – This is a near-universal observation among full-time traders and captures an important understanding.  Fear, greed, overconfidence, self-blame—all of these can undercut even the most mechanical trading.  Indeed, when Linda Raschke and I surveyed 64 traders for their personality and coping patterns, the factor of neuroticism—the tendency to experience negative emotions—emerged as a major factor associated with trading difficulties.  This makes sense from an implicit learning perspective.  To the degree that a trader is focused on his or her fears, self-esteem, fantasies, etc., attention is drawn away from the learning process.  The problem may not be emotionalism per se; there are many highly emotional, but successful traders.  Rather, the issue may be the degree to which emotions interfere with one’s cognitive processing by competing for attention.  Focusing on negative emotions may be a much larger problem than actually experiencing them.  Many outstanding traders “explode” when they make a rookie error.  For them, however, the storm blows over quickly; less successful traders appear to be less able to let the issue go.  As a result, they become caught in a cycle of blame, increasing self-consciousness, and further blame.  As a psychologist, my leaning is to help traders experience their frustration and get over it quickly, rather than “overcome” it altogether.  (In my chatroom session with Linda Raschke, I will be addressing how to accomplish this).

3. The advantages of learning trading vs. investing – If the internalization of complex patterns requires many thousands of observations across different market conditions, the challenge for the trader is making this process as efficient as possible.  My sense is that there may be an advantage to learning trading, as opposed to investing, simply because short-term traders are apt to observe many patterns in the course of a single day or week.  The investor, conversely, may note a pattern every few months or years, greatly extending the amount of time needed for implicit learning.  This dynamic would help to explain why many of the most successful traders I have met have had experience working on the exchange floors.  In the fast-paced environment of the floors, a trade may last seconds to minutes, with many trades placed per day.  Complex research strategies and chart analyses fly out the window when time frames are compressed to that degree.  Instead, traders become so immersed in the markets that they acquire the (implicit) ability to read moment-to-moment patterns of momentum and price change.  This creates an ideal implicit learning environment—having so many patterns to read per day makes the development of expertise much more efficient—but it also might help account for difficulties floor traders often experience when they attempt to trade away from the floor.  Without the contextual cues that help them process those price and momentum shifts, floor traders lose their edge—even though they may think they are employing their same, successful trading methods.  

4. Developing technologies for training traders – If we look at how experts are trained in other fields, we notice a common factor:  an intensive period of apprenticeship in which the student works under a master and obtains continuous instruction and practice.  Consider, for example, the cultivation of expertise in the martial arts.  Many years will be spent in the dojo studying under a sensei before the black belt is conferred.  Instruction alternates with practice; rehearsal of techniques alternates with the application of techniques in real-life (tournament) conditions.  The online medium has created a variety of promising strategies for training traders, such as Linda’s chatroom, real-time market commentary via weblog, and services that allow simulated online trading.  My sense is that we will see an accelerated shift from services that emphasize trading techniques to comprehensive trading “dojos” that incorporate real-time instruction, practice, and coaching.  Already we are seeing expert instruction modules built into conventional software programs such as Metastock.  This move toward immersive implicit learning environments strikes me as a most promising application for peer-to-peer networks, as traders share research resources and trading experiences and learn from each other.  (See www.limewire.org for more information on Gnutella and P2P networking). 

5. Developing technologies for facilitating learning – This is my primary research interest in trading psychology.  A broad array of research suggests that learning is mediated through the brain’s prefrontal cortex, which also controls attention, concentration, planning, and other executive functions.  We also know that children with learning disabilities are significantly more likely than others to possess neurological deficits associated with the frontal lobes, including attention deficit hyperactivity disorder (ADHD).  Elkhonon Goldberg cites considerable research that indicates we can improve the functioning of our frontal cortex through structured exercises, much as we can build our muscles in the gym.  Such exercises have been used, for example, in delaying the onset and progression of Alzheimer’s disease.  Is it possible, however, to develop super-states of concentration and learning in a mental gym the way that bodybuilders can hone their physiques in a weight room?  I believe we can.  I am currently working with Dr. Jeffrey Carmen on biofeedback strategies that directly measure regional cerebral blood flow to the prefrontal cortex.  Utilizing infrared sensors to detect heat changes in the forehead (reflecting increased frontal blood flow), it is possible for traders to know exactly how much of their mental processing power is available to them at all times.  Moreover, it is possible for them to learn strategies for increasing their blood flow and maximizing their optimal learning states.  This would allow traders to process each trading day (or lesson) as thoroughly as possible, creating more efficient learning.

My research to date suggests that the state of mind induced by the biofeedback work is not unlike the state that people enter during hypnotic induction or meditation.  It is a state of relaxed and focused concentration.  Such a mind frame minimizes the impact of emotional interference at the same time that it quiets the verbal, internal dialogue that permeates much of our cognitive lives.  Following Goldberg’s hypothesis, I believe that the capacity to enter such states of consciousness may allow us to efficiently process novel information by facilitating right hemispheric activation, even as it dampens emotional arousal and the interference of critical, verbal thinking.  This very much fits with psychologist Mihalyi Csikszentmihalyi’s observations of “flow” states among highly creative and successful individuals.  The learning of expertise may depend as much upon the mind state of the learner as the quality of the instructional materials.

Conclusion

I began this article with a straightforward question: How does one gain expertise as a trader?  We have seen that expertise is often described as the outcome of an explicit research process or as an explicit process of acquiring knowledge about recurrent patterns.  Much skill-based learning, however, is acquired implicitly, as the result of processing thousands of examples.  Small children learn language, for example, long before they can verbalize rules of grammar and syntax; we learn complex motor skills, such as hitting a baseball, without ever being able to capture our knowledge in a way that could be transferred to another person.

While immersion in research and in pattern recognition can indeed produce trading expertise—a finding made clear by Schwager—the key ingredient in trading development may be the immersion, not the research or the patterns per se.  If this is true, efforts to find the best trading system or the most promising chart pattern are off the mark.  The what of learning trading is less important than the how.  If you want to become a proficient trader, the most promising strategy is to immerse yourself in the markets under the tutelage of a master trader.  You need to process example after example under real trading conditions, with full concentration, to develop your own neural network. 

I believe the most exciting frontier for trading psychology is the development of tools and techniques for maximizing implicit learning processes.  Such techniques would assist both in the acquisition and utilization of expertise by training individuals to sustain states of consciousness in which they are open to implicit processing.  As I hope to demonstrate more thoroughly in my forthcoming book, there are reasons for believing that experienced traders possess greater expertise than they are aware of.   This tacit knowledge, to use Michael Polanyi’s memorable term, reveals itself during “hot streaks” in trading and those wonderful experiences where we just “know” what the market is doing and place winning trades accordingly.  Too many traders look to emulate others.  The secret to success, conversely, might just well be to gain greater access to the expertise we have already acquired implicitly and learn to become the traders we already are when we’re at our best.

Well, if you’ve followed me thus far through a lengthy article you no doubt have much of capacity for attention and concentration needed to become a master trader!  I have purposely made no effort to “dumb down” this article for the mass audience, even as I’ve tried to steer clear of technical jargon and the intricacies of research studies.  In the coming months, I hope to elaborate many of the ideas and techniques alluded to in this article, and I encourage you to stay in touch regarding new directions and developments.

With that, I will part with a last research finding from Reber.  Remember those artificial grammars that people had to learn, such as MQTXG?  Letters were displayed to subjects that either followed the grammar (i.e., Q could only follow M; T could only follow Q, etc.) or that did not.  The subjects did not know the rules of the grammar, but over many trials could figure out which combinations of letters were right and which were wrong.   Suppose, however, that the grammar is changed in the middle of the experiment, so that the new constructions follow the rules of NRSYF instead of MQTXG.  Will subjects continue to display implicit learning?

The answer is enlightening.  After many trials with the initial grammar, without knowing the rules, subjects will choose “MQ”, “TX”, and “QT as grammatical constructions while rejecting “QM”, “XT”, and “TQ”.  Once the grammar is switched, the subjects’ learning goes out the window and their guesses retreat to chance levels.  But with enough new trials, subjects pick up the new grammar and are able to recognize “NR”, “SY”, and “RS” as grammatical and reject “RN”, “YS”, and “SR”.  In other words, people not only learn complex patterns implicitly; they continue their implicit learning when the patterns shift.  This has major implications for the development of market expertise.  The markets are always changing, but as long as we stay in our learning modes, we can adapt with them.