Everyone following markets wants to see order, even if the data present only chaos. Humans reject random data, imposing explanations instead. They see patterns that reflect normal probabilities. They look at zillions of cases and celebrate a single instance that confirms a preconception.

Because behavioral economics has been so widely celebrated, Nobel Prizes awarded, and proponents publicized, it seems like we should have learned something.

Not so!

People are schooled to buy when others are fearful, but it is easier said than done.Similarly, and despite the warnings, it is easy to fall into the trap of imposing order on chaos.Larry Sessions has great examples from nature.

This imposition of patterns on chaotic data is correlated with intelligence. One of my early classes in methods (one that seems to have been omitted by many current Wall Street experts) emphasized the need to start with a hypothesis. I wrote the story almost ten years ago.I encourage you to laugh and me and my classmates by reading the original (not very long), before the spoiler below.

If you read the link, you are laughing.If you did not here is the key point. The professor started with conclusions and then asked for explanations. He got some persuasive ideas. Then he surprised the class by revealing that the relationships were all the opposite of what he had originally told them!

This is an extremely important lesson.Analyzing lots of data, with hundreds of possible relationships, will always yield some findings — statistically significant!Fertile minds can figure out some logic to explain these findings.

That approach is backwards.Good research begins with theory and hypotheses and then moves to testing.

In one sense it is a shame that Wall Street researchers did not get this kind of training.If one looks carefully at their reports, it is pretty obvious when a researcher is “data mining” and when there is some theory behind the work. A key question is: Which came first?