This next stretch of posts turns from Simons' own story toward something more immediately useful: how to actually apply this thinking to our own trading, concretely, not just philosophically.
Start with the most foundational discipline of all: knowing whether a
pattern you've noticed is real, or just noise dressed up as a discovery.
Why This Is Harder Than It Sounds
As covered earlier in this series, Simons' team never trusted a pattern
because it "made sense" or "felt right" — they trusted it
only once it held up statistically across large amounts of data, repeatedly,
across different conditions. Most traders do something close to the opposite
without realizing it: they notice a handful of instances where a setup worked,
feel a jolt of pattern-recognition satisfaction, and start trading it as though
it's proven — when what they've actually observed is a small, cherry-picked
sample that hasn't been tested against anything.
The human brain is exceptionally good at finding patterns in randomness,
and exceptionally bad at estimating how much evidence is actually needed before
a pattern deserves trust. That gap is where most false edges are born.
A Practical Checklist
Before trusting a setup with real size, ask honestly:
- How
many independent instances have I actually seen this work, not just
recently, but across meaningfully different market conditions? A
handful of examples from the same trending month tells you far less than
the same setup tested across trending, choppy, and volatile regimes.
- Did I
find this pattern by testing a specific hypothesis, or by scanning charts
until something jumped out? A powerful enough search
will find "patterns" almost anywhere you look — including pure
coincidence, as covered earlier in this series. Patterns found through a
directed hypothesis carry more weight than patterns noticed retroactively.
- Have I
checked how it performed on data or periods I wasn't already familiar with
when I formed the idea? Testing only on the
examples that inspired the idea in the first place is the exact
overfitting trap covered earlier — it flatters the very data that created
the hypothesis.
- Am I
able to state a rule for this pattern precisely enough that someone else
could apply it without my judgment calls filling the gaps? If the
"pattern" only works when you eyeball it a certain way, it isn't
yet a tested edge — it's an intuition wearing the costume of one.
- What
would prove this pattern wrong, and have I actually looked for those
counterexamples, or only remembered the times it worked?
Selective memory is one of the most common ways a false pattern survives
far longer than it should.
Why This Matters More at Small Sample Sizes
Medallion's roughly 50.75% win rate, covered earlier in this series,
only means anything because of the enormous volume behind it — a coin flip
repeated a handful of times tells you almost nothing, while the same coin
flipped a million times reveals its true odds with real precision. A pattern
you've traded five or ten times, however good it feels, simply hasn't
accumulated enough evidence to separate genuine edge from ordinary variance yet
— and treating it as proven before it has is one of the most common, avoidable
mistakes in discretionary trading.
Apply This to Your Own Trading
Take one setup you currently trade with real conviction — the one that
feels most "obviously" reliable to you. Write down, honestly, how
many genuinely independent instances you've actually tracked, across how many
different market conditions, and whether you tested it against periods you
weren't already familiar with. If you can't answer those questions with real
numbers, that's not a failure — it's simply the next piece of work. Build the
tracking habit for that one setup before adding a second, and let the evidence,
not the feeling, decide how much size it deserves going forward.