Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

I want to use matlab to perform a statistical test to find if there's a significantly higher mean return in any of n different sample populations. However, I can only find post hoc (although they can be used separately the input in matlab requires that ANOVA is run first) tests for the ANOVA, which I can't apply since the assumptions do not hold (normaly distributed, equal variance, independence between samples (do they mean between observations within the samples or between the sample populations?)). What I have is different strategies and I want to find out if a significantly higher mean return can be achieved by changing a parameter (sort of). Since the strategies are evaluated using a rolling window the returns will be dependend. Also the variance most likely differs between the samples.

Details: I'm evaluating a strategy using different stocks. I want to see if I can find significantly higher mean returns by using these different groups of stocks (7 groups). The strategy is evaluated over a ~16 year period using 4 month trading periods and a 2 month rolling window. Since the trading periods are overlapping the returns will most likely be dependent (the returns depend on where the trading period starts so the returns in the overlap will not be exactly equal). However, the different samples are independent of eachother as the stocks are mutually exclusive.

In the second case the stocks are not mutually exclusive. Hence the (in this case) 3 groups will be slightly dependent. Otherwise the same conditions holds here.

What is the most appropriate test to test for significant different means under these conditions? How can I implement it in matlab as a standalone test?

share|improve this question
Tukey's name is associated with many tests, including more than one test relating to equality of location. It's probably better to be more specific than 'Tukey's test' (though your tags seem to clarify it here, it's best not to rely on that to convey your meaning). Why do you specify that particular test? Simply because it's for multiple comparisons? Note that Tukey's HSD relies on the same assumptions as ANOVA. If the assumptions of the test you're asking about don't hold, you may want to consider a different title, since it doesn't seem to be directly relevant to your ultimate question. – Glen_b May 1 at 0:48
Okey. I'm very novice in this particular area. From reading at wikipedia I got the impression that it was the same test with several names. Yes, I should change the topic. I guess the problem is that I don't know what tests could be used. I've looked at Dunnett's, ANOVA, Hotteling's and Tukey's so far. To me it seemed like Dunnett's and Tukey's had the best fit. I only found that the assumptions of Tukey's range test are; the observations being tested are independent, there is equal within-group variance across the groups associated with each mean in the test (wikipedia). – Good Guy Mike May 1 at 1:23
Although the independence do not hold for me, I assumed that maybe this was something that could be corrected for using Newey-Wests standard errors or something similar. – Good Guy Mike May 1 at 1:26
It would probably make sense to describe what you want to achieve in relatively basic terms, and what your circumstances are - how many groups you have, for example, what you're comparing (what is being measured; is it a count, a measurement of size, a 0-1 variable, a categorical variable ...) and exactly how you can tell the assumptions of ANOVA aren't satisfied. If you're not knowledgeable about the right approach, looking some names up and then specifying them as if you know what you want may lead to a poorer outcome than if you just tell us the problem you want to solve in plainer terms. – Glen_b May 1 at 1:27
Ok, makes sense. I will update with some more details. – Good Guy Mike May 1 at 1:30

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.