In practice
Understanding overfitting protects traders from their own cleverness. Every added rule and tuned setting can improve past results simply by fitting the noise in that data.
The usual defence is to build the strategy on one part of the history and check it on another part it has never seen. Simple rules with few settings tend to hold up best.
What to watch for
A smooth, almost perfect result in testing is a warning sign. If small changes to a setting ruin the results, the strategy was fitted to the past and will not repeat.