How to properly compare a trading signal to a random strategy.
Harry Georgakopoulos wrote:
Set up ^^^^ Let's say i have an evenly spaced discrete time-series of bid-ask prices. Let's also say that the total number of such bid-ask pairs is N for a given day. Given a signal that generates a buy opportunity on "n" such times (where n << N), how can I reliably say that these n signals have a mean profit that is statistically significant? For example, assume I get 35 buy signals throughout the day where I buy the offer, wait 5 minutes and then sell the bid. This will generate a vector of 35 price-differences. These price differences will have a particular distribution. Thoughts ^^^^^ 1. I can compare the distribution of the 35 price-differences generated from the signal against the distribution of 35 randomly chosen entry points throughout the day. (maybe some kind of t-test on the difference of the means of these distributions) 2. I can compare the distribution of the 35 price-differences to a rolling window of all possible buys throughout the day and selling after 5 mins. (more data-points to compare against) 3. I can compare the distribution of the 35 price-differences against an absolute value of 0. Any ideas on quantifying the significance of such a signal would be appreciated. Is one method preferred over another? Am I inadvertently introducing bias in the analysis? I realize that the distribution of the price-differences might not be normally distributed. This might make any analysis based on a t-test invalid. Thank you in advance. H.
Pat Burns has a paper on this topic on his website.
Regards,
- Brian
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock