KS Test question (2)

The warning (with an error you would not see any results) means that there are ties in your data, the theory behind the ks test says that the probability of seeing ties is 0, so your data and the theory do not match, therefore the p-value is suspect (though an ok approximation for some uses).

These types of tests are useful for showing differences (often in a non meaningful way), not similarities.  You really need to decide what you mean by similar.

Consider two population distributions, the first is the standard uniform with density height equal to 1 between 0 and 1 (0 elsewhere), the 2nd distribution has height 1 from 0 to 0.99 and from 99.99 to 100 (0 elsewhere), are these 2 populations similar?  By some measures they are (the ks statistic for one), by other measures they are not (comparing mean and variance as an example).  Whether they are similar or not really depends on what you want to do with them.

One additional "test" you might consider is use the vis.test function in the TeachingDemos package, write a function that will either draw a standard qqplot of your 2 datasets, or pools them together then splits them randomly and creates the qqplot.  Use this with vis.test, if you cannot pick out the real dataset then it is less likely to matter if you interchange them.  (this assumes 2 random samples from the respective populations, if there is something more going on then you will need to come up with a different comparison that accounts for any structure).

Hope this helps,

KS Test question (2)

Thread (4 messages)