Online Stats Help for Friends

5/26/2010

Welcome to the online statistics and research methods help page. If you are a student or researcher looking for help with statistical and methods questions, post your question below and I will do my best to answer your question. If you would like to answer someone else's question or just discuss research issues, feel free to post your comments. If you find my advice helpful, say a friendly hello to the next Mormon missionary you see on the street.

124 Comments

Roy

6/7/2010 11:57:57 pm

I am looking for help from anyone who has worked with Poisson regression in GPower

Dave C.

6/8/2010 06:42:40 am

Roy,
Explaining poisson regression in Gpower is next on my list. I will try to get to it soon. You may try http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/user-guide-by-distribution/z/poisson_regression

The Gpower authors recently posted a tutorial on poisson regression power analysis.

Dave C.

6/11/2010 06:17:26 am

Roy,

I updated the G*Power files. Instructions on Poisson regression with binary and continuous predictors are now available.

Best wishes.

Elaine

8/13/2010 12:40:31 pm

Hi,

Thank you for your web site, it is the only guide I have found on using G*Power for ANOVA, repeated measures within-between interactions. However I don't understand how to find out "correlations among repeated measures' value. I would like to perform a post hoc analysis, and would be so grateful for any help you can offer.
Kindest Regards,
Elaine

Dave C.

8/14/2010 05:56:08 am

Elaine,

Thank you for dropping by. I hope you find the G*Power resources useful.

The "correlation among repeated measures" value refers to the expected correlation among the variable that is measured more than once. For instance, if you measure test performance on day1, day2, and day3, think about the level of correlation between the test scores across the three days and enter that value (usually a value between 0 and 1).

In most cases we expect a moderate to high correlation among the variable that we repeatedly measure. So, continuing with the previous example, if you administer the same test on 3 consecutive days and expect a high correlation between day1, day2, and day3 scores, you might enter .70 as the "correlations among repeated measures" value.

Let me know if you have more questions.

Best Wishes,

Dave

Alli link

11/20/2013 03:08:45 am

Dave,

I truly appreciate your website -- it is so helpful!

I was wondering if you can recommend a citation for the information you provided above about the "correlation among repeated measures" value in G*Power for a repeated measures ANOVA. Is there a textbook or journal article that discusses .70 as a reasonable expectation for this value?

Thank you,
Alli

Dave

11/20/2013 04:39:11 am

Alli,

I am not aware of an article stating that something close to 0.70 is a reasonable expectation for the "corr among repeated measures" value. Because lowering this value reduces the statistical power and raises the required sample size, you can experiment with different levels of this value. For example, if entering 0.70 results in 50 subjects to achieve 90% power, but you have access to 80 subjects, you could enter a more conservative "corr among rep measures" value like 0.50 which may result in 90% power with 75 subjects. In any event, I've never had an IRB or granting agency request information on the "corr among rep measures" value. They are usually more concerned with the expected effect size, power, and sample size.

I am glad you've found the G*Power guide helpful. Best wishes.

Dave

Eliane92

8/14/2010 06:02:15 am

Dear Dave,

Thank you so much, I honestly can't tell you how grateful I am. That makes complete sense. I will go ahead and give it a go. Fingers crossed!

Best Wishes,
Elaine

Nicolas

12/27/2010 10:17:44 pm

Hi,

I'm really grateful for this G*Power guide, because it makes power calculations really easy for all students and researchers who have to deal with statistics but arent statisticians. My question is about the partial eta squared that is needed to determine effect size in "MANOVA, repeated measures, between factors". How does one determines this? If, for example, one has a pilot group and runs the MANOVA for the pilot data then would it be correct to use the partial eta squared received in SPSS in the table "multivariate tests" for the effect of group? Or should it be calculated elsewhise?
Kind Regards,
Nicolas

Dave C.

12/28/2010 06:21:09 am

Greetings Nicolas,

I am glad you’ve found the G*Power guide useful. I hope I am able to answer your question.
Unfortunately G*Power can be somewhat confusing when it comes to MANOVA.

One problem is that some people call a design MANOVA when referring to univariate repeated measures designs. In this case, the M in MANOVA refers to “multiple” as is multiple/repeated measures. While others, such as myself and G*Power included, call a design MANOVA when referring to “multivariate” designs with more than one outcome variable (DV).

Another problem is that under G*Power’s drop down list for F tests it says that it can run calculations for “MANOVA: Repeated measures”. No doubt people have interpreted this to mean that the program can run power and sample size calculations for repeated measures, multivariate analysis of variance (RMMANOVA), sometimes called doubly multivariate ANOVA. RMMANOVA involves measuring multiple outcome variables (DVs) on the same group of people multiple times (e.g., time1, time2, time3). If this is what you’re trying to accomplish then I am sorry to say that G*Power cannot do this. I even contacted one of G*Power’s creators and he confirm that G*Power cannot handle RMMANOVA, which is unfortunate because it is a common research design.

So what kind of analysis does G*Power’s “MANOVA: Repeated measures” refer to? It is the multivariate approach to analyzing univariate (one DV) repeated measures data. This can be seen more clearly when you mouse click on Tests along the top, then go to means, and then go to “Repeated measures: between factors, MANOVA-approach” in the drop down list. The MANOVA approach is commonly used in univariate repeated measures when the Sphericity assumption does not hold in univariate analyses. The multivariate approach to univariate repeated measures (RMANOVA) treats the outcomes as being comprised of multiple DVs obtained on a single group of people.

If you are running a true repeated measures multivariate ANOVA (RMMANOVA) then I recommend generating random samples of data (using expected means and standard deviations) for each group of the within subjects factor (e.g., time1, time2, time3) and then run these through SPSS. Ask SPSS to give you the power calculation each time you run the RMMANOVA. Do it multiple times (with a new set of generated means and SDs) and average the power calculations that SPSS gives you. If you need help on generating random data for specific expected means and SDs, I can give you a site that does this nicely. You only need to copy and paste the randomly generated data into SPSS.

If you are looking at powering a univariate repeated measures design then I recommend using G*Power’s “ANOVA: repeated measures” function. Instructions on how to use this function are included on my website. You should only use the multivariate equivalent if you expected that the sphericity assumption will not be met, which is unlikely in most cases. For this reason I think that adding the MANOVA equivalent test for univariate repeated measures in G*Power was superfluous at best, and confusing at worst.

Please let me know if you have any more questions.

Nicolas

12/28/2010 09:05:43 pm

Thank you very very much for your time to answer, things got much more clear now. Repeated MANOVA was a trap after all to me! So, what i am trying to do is to see if two groups of subjects differ regarding a variable measured 5 sequential times in each subject. In this case I assume the correct test is a univariate repeated ANOVA testing for between-subjects effect (setting group as the between-subjects factor).
I read the instructions as you told me but I still have some questions: What is the mean in each group that should be entered in G*Power (ANOVA, repeated measures, between factors) for effect size calculation? The mean of the means for the 5 measurements in each group or the mean of all data pooled for each group? I am confused because the variable was measured 5 times in each subject. And what if the SDs (for the overall means)differ between the 2 groups? I should use the mean of the two SDs in the G*Power?
Instead of this, could I use the partial eta squared for the pilot data in the SPSS derived from repeated-ANOVA for the effect of group? It seems much more comfortable to me!
I know this is all too much, but I cant stand also to ask: If the power calculation is done with normally distributed pilot data, but the final data turn out not to be(and therefore analysed with a non-parametric test), is there a problem with the validity of power calculation? And finally, what is done when the pilot data for power calculation are non-parametric or someone wishes to calculate observed power for not-normally distributed data? I havent found any answer to this case anywhere(I dont know if i am so unlucky, but my final data are always eager for non-parametric tests!!)
Thank you so much!
Nicolas

Dave C.

12/29/2010 05:39:59 am

Nicolas,

I've included some replies to your questions.

So, what i am trying to do is to see if two groups of subjects differ regarding a variable measured 5 sequential times in each subject. In this case I assume the correct test is a univariate repeated ANOVA testing for between-subjects effect (setting group as the between-subjects factor).

- I agree. So to calculate power for the between subjects effect in a univariate, repeated measures ANOVA you use the G*Power “ANOVA: Repeated Measures, between factors” analysis. On the other hand, if you wanted to know power for testing the within subjects factor (5 sequential times), then you would use the “ANOVA: Repeated Measures, within factors” analysis.

What is the mean in each group that should be entered in G*Power (ANOVA, repeated measures, between factors) for effect size calculation? The mean of the means for the 5 measurements in each group or the mean of all data pooled for each group?

- The means being asked for are those for each group across all 5 measurements (the mean of all data pooled for each group). Calculate the means for all data points for group 1 and group 2 while ignoring the times when those data were collected, then enter these 2 means into the appropriate cells.

I am confused because the variable was measured 5 times in each subject. And what if the SDs (for the overall means)differ between the 2 groups? I should use the mean of the two SDs in the G*Power?

- Yes, use the mean of the two estimated SDs.

Instead of this, could I use the partial eta squared for the pilot data in the SPSS derived from repeated-ANOVA for the effect of group? It seems much more comfortable to me!

- Yes. Under the “Select procedure” tab, chose “effect size from variance” and enter the SPSS partial eta squared in the appropriate box. This is the direct approach. Eta squared conventions are small=.01, medium=.06, large=.14. I would compare the results from using eta squared and using the means. They should be similar. If you have preliminary data in SPSS, then ask SPSS to give you an estimate of power in your output file. This should also be similar to the G*Power calculations given that they use the same data.

I know this is all too much, but I cant stand also to ask: If the power calculation is done with normally distributed pilot data, but the final data turn out not to be(and therefore analysed with a non-parametric test), is there a problem with the validity of power calculation?

- No. Power and sample size calculations are simply a best guess analyses of your chances of correctly rejecting the null hypothesis if your data turn out the way you expected them.

And finally, what is done when the pilot data for power calculation are non-parametric or someone wishes to calculate observed power for not-normally distributed data? I havent found any answer to this case anywhere(I dont know if i am so unlucky, but my final data are always eager for non-parametric tests!!)

- If your preliminary data are not normal, you may consider doing a natural log transformation to bring them within normal parameters, that way you could stick with ANOVA. If you did this you could just enter the power calculation parameters into G*Power for the log transformed data. On the other hand, you could analyze your data using the nonparametric equivalent of the RMANOVA, called the Friedman test, although, as you point out, it is difficult finding a power calculation tool for this test. Anyway, here is what I would do if the preliminary data were not normal. I would power everything for a RMANOVA and add extra participants to the study if the expected power is not at least 90% or higher to compensate for the possibility of having to resort to a less powerful nonparametric test in the event that the final data turn out to violate the normalcy assumption.

I hope this helps.

Nicolas

12/29/2010 06:23:10 pm

Dear Dave,
everything is clear now! Thank you so very much!
Wishes for the best!!
Nicolas

Haider link

3/27/2011 07:00:37 am

Thank you for this wonderful guide.

In the guide:
42. MANOVA – Special Effects and Interactions

The number of groups that you mentioned is 4.
But the example was:
We want to compare 4 groups of patients getting therapies A, B, C, and D based on 5 outcome variables Y1, Y2, Y3, Y4, and Y5, with sex (M vs. F) as a between subjects factor.

This example if we apply it as univariate ANOVA with just 1 DV, according to your example in univariate ANOVA (ANOVA: Fixed effects, special, main effects and interactions) the Number of groups = found by multiplying the levels in both factors (in this case 2x4=8) .

Shouldn't be the same way for MANOVA?

There are 4 levels in the 1st factor * 2 levels in the other (Male and Female).

So, is it right that the number of groups should be 8 instead of 4?

The other thing is that in Gpower 3.1 I have found that there is another parameter that you did not mentioned in MANOVA – Special Effects and Interactions.

It is the number of predictors.
What should be in your example?

And why this number of predictors is not available in MANOVA – Global Effects in Gpower?

By the way, What is the best software (commercial or free) in your opinion in power analysis? Is it the PASS?

I found that it has many parametric and nonparametric tests can be power tested in PASS (even more than GPower 3).

Strangely, the SPSS powersample has less statistical tests than GPower (e.g., MANOVA is not available). Although that IBM's SPSS is a big player in statistical software industry.

Also aperantly NQuery's list of tests is less than GPower.

GPower is really good, but just curious about the best software that may be you have encountered.

Thanks
Haider

Haider link

3/27/2011 09:51:16 am

Dave,

I have found an interesting java application on power analysis.

Probably you are interested to see here other power analysis tests that is not available in GPower (or even the other commercial softwares that you mentioned like the ANOVA with 'replicates').

It is here:
http://www.stat.uiowa.edu/~rlenth/Power/index.html

He also has some conservative view on the Cohen's rules of thumb (small , med., large Effect size conventions), which I agree with him completely. But it is pretty nice from Cohen that he provide us with these rules for whom they do not have any previous pilot study nor any previous literature published on a novel study design.

ozaz

8/18/2011 06:32:28 am

Hello,

Thank you for posting your helpful G*Power guide. I have one question.

I understand from this and other guides how to calculate power for a within-between ANOVA interaction effect but I don't understand how to calculate power for a within-within ANOVA interaction effect.

For example, the power for the interaction between 2 within subject factors: music listened to (rock,country,rap) x day listened on (day1,day2)

Dr. Virender Verma

12/6/2011 05:14:40 am

Dear Dave,

Thanks for providing an excellent resourse on Gpower, it is really very very helpful.

I am an MD in Pediatrics and we are plamming a study where we will assingn children with severe pneumonia(meeting some defined criteria) randomly to two groups ie group I and group II. Children are of 2months to 5yr of age. Group A will receive standard therapy (injectable antibiotics) + oral placebo and group II will receive standard therapy (injectable sntibiotics) + oral test drug. we will look for following out come variables:

1) time in hours for Clinical improvement in terms of fever and tachypnea

2) Total hours of supplimentary oxygen therapy

3)Total days of hospital stay

4) Need for change of antibiots

5) Complications (empyema, pneumothorax, need for mechanical ventlation)

6) death

We want to do "a priori" analysis for sample size calculation

can you suggest me which test to be used (MANOVA, RMNOVA etc) and how to calculate the sample size (using gpower) taking in consideration age and sex as the confounding covarient.

I agree that my question is a bit childish but I accept I have very little knowledge of stats and I hope you must guide me.

Excuse me if i asked any foolish question.

Sincerely Yours

Virender Verma

Dave C.

6/5/2012 06:33:48 am

Everyone,

For the past year I had been wondering why this page was not getting anymore traffic (questions). It turns out that I forgot to update my email address so I was not recieving anymore notifications of activity. Add to this the fact that the comments settings were set to "approve first". So Weebly was trying to notify me of new messages but those were going to my old email address, and because I was not receiving and approving the messages, they were not being approved. My apologies to everyone who posted comments/questions that seemed to fall into an abyss without a trace. I've fixed the problem.

Dave C.

Jim Howe

6/22/2012 03:15:35 am

Dave:

Please email me. I want to discuss quantum physics, entanglement, and prayer. I found you on mormonsandscience.com, but this stats help site is the only way I can find to contact you. I am LDS, and I am intrigued by some comments you made about your book, "Truth and Science."

Thank you,
Jim

Han

11/13/2012 08:38:52 pm

Hi Dave,

I need to estimate sample size using G*Power 3.1.5.

The pilot study experiment design was like this, 3 treatments, 8 subjects per treatment group and each undergo observation of 60 minutes, data collected every 5 minutes. Therefore the time(min) would be the repeated measures.

In G*power 3.1.5, I'll need to key in the corr among rep measures, how do I calculate this parameter?

Dave C.

11/14/2012 05:42:45 am

Han,
The correlation value between repeated measures is probably going to be fairly high if you have reason to believe that the variability between participants' outcomes will be fairly constant across time. Imagine taking the outcomes for eight subjects in treatment 1 at time 1 and correlating those outcomes with their scores at time 2, and so on. If you expect the magnitude of the differences between those eight subjects to remain fairly constant across time, even though the individual scores may increase or decrease across time, then the correlation will be fairly high. Example of data at first measure: 2,4,3,5,8,7,9,9. Example of their data at second measure: 3,3,4,6,8,8,10,9. This shows a fairly high correlation and constant variability between measures.

Pieter

1/18/2013 09:44:39 am

Hi Dave,

I have a question about calculating the right sample size to get an power of 0.95. I have looked all over t
he web but I just can't get to find an definitive answer, you're my last hope!

I have an project in which I have 4 groups, each group starts the research by reading an different instruction, which makes for the 4 groups. Then they get to see 36 faces which they'll judge on a 7-point scale after seeing each face, but each face has a different presentation time, so they'll see 36 faces, of which 12 will be presented for 8 seconds, 12 for 10 seconds and 12 for 12 seconds.

So an between-subjects factor with 4 levels, and an within-subjects factor with 3 levels, correct?

I tried to do power analysis, but I am unsure whether I am doing it right or not.

F-tests --> ANOVA: repeated measures, within-between interaction --> A priori:

Input parameters:

Effect size f: 0,25
alpha err prob: 0,05
Power: 0.95
Number of groups: 4
Number of measurements: 3
Corr among repeated measures: 0.5 (how do I know!?)
Nonspherecity correction: 1

Then I need a sample size of 60, is this correct?

Dave C.

1/18/2013 05:50:54 pm

Hi Pieter,

It sounds like the between subjects factor is 4 levels of instruction, and the within subjects variable has 3 levels (viewing faces for 8, 10 and 12 seconds). I am wondering if the subjects' 7-point ratings for each group of 12 faces will be reduced to single outcomes. If so then it sounds like a univariate repeated measures design as you state. Note that a potential problem with this design is that type of face is confounded with presentation time. In other words, it could be argued that we can’t tell whether judgment was influenced by type of face or duration of presentation, or a combination of both.

You should consider powering the between subjects effect of the repeated measures, and/or the within subjects effect. You would only power the interaction if you are truly interested in such an effect (which may be the case). The between subjects effect for a repeated measures would be
Alpha = 0.05, power = .95, number of groups = 4 (between factors), repetitions = 3 (within levels), correlation among repeated measures = fairly high (e.g., .70 to .90) if you expect each groups’ scores to be consistent across the 3 repetitions, but it could be moderate to low if the 12 faces in one repetition differ considerably from the faces in other repetitions. Also, don’t forget to determine the effect size.

Does that help?

Pieter

1/18/2013 08:51:28 pm

Yes, the 12 faces for each group will de reduced to single outcomes, but are randomly given different presentation times. So every participant gets to see the same faces, but each face can be presented in an different order, and can also have a different presentation time. Sometimes 10 sec, 10 sec, 12 sec, 8 sec, 12 sec, etc. this is randomly altered and unique for every participant to reduce the confound of face type. In the end I will have data from 4 conditions and for each condition a mean score for 8, 10 and 12 seconds. So, it would be correct to say 4 groups and 3 measures?

The only thing I am wondering about is the correlation. Because the same 36 faces are used, but they're randomly 'shuffled' between the within groups. Their order and presentation time differs. Has that any influence on the expected correlation?

Pieter

1/18/2013 08:52:49 pm

Sorry for Triple posting, but the 'submit' button said there was an error and I should try again, but it seems the reactions were posted anyway. My apologies :) You can delete the double ones and this one :D

Pieter

1/18/2013 09:46:04 pm

Oh, and I do expect the condition to influence the outcomes of the 8 sec, 10 sec and 12 sec presentation times in an specific way :) So I think the between-within interaction is correct :)

Dave C.

1/21/2013 07:52:27 am

Pieter,

Yes, I would say 4 groups (between subjects factor) and 3 repeated measure conditions (8, 10, and 12 seconds). Because the faces are randomly shuffled between the 3 interval settings it is difficult to determine the correlation. I would probably start with 0.50 and check the sample size for .95% power. You can always make the power estimate more conservative by lowering the correlation to say .40 if you have enough participants. For example, if 0.50 correlation says you need 30 participants but you can easily find 40 participants, you can select a lower, more conservative correlation value (e.g., 0.45) that gets you closer to 40 participants. This sort of adjustment is fine when you have to justify a sample size to an IRB or some other group, and it is fine as long as you are sure that the other estimates are fairly accurate.
Regards,
DC

Pieter

2/8/2013 01:27:29 am

Thank you very much Dave! You've been a great help! I had forgotten to thank you before, so my apologies :) You're an statistical life saver! ;)

virender verma

1/23/2013 06:20:56 am

Dave C.

1/24/2013 06:42:51 am

Virender,

Here is one way to approach your research situation. There are 3 continuous outcome variables. These are hours to fever improvement, hours to tachypnea improvement, and hospital LOS. There are also 3 categorical outcomes which appear to be binary (change antibiotics [yes vs. no], complications [yes vs. no], and mortality [yes vs. no]). I advise analyzing each group of outcome variables separately.

You could compare the 3 continuous outcomes variables between group 1 and group 2 with a Hotelling’s T-squared test. The Hotelling’s T-squared test will first evaluate the multivariate difference by considering whether the 2 groups differ when all 3 continuous outcomes are taken into account. If the multivariate test is significant, it will then test each continuous outcome by checking for univariate differences between the 2 groups. Hotelling’s T-squared can be powered using G*Power’s Hotellings T2: two group mean vectors power analysis in the F-test group. My G*Power user guide has instructions on how to power this test.

You could then analyze each of the binary outcomes (change antibiotics, complications, mortality) using separate binary logistic regressions. For example, this could be done by setting up death (yes [1] vs. no [0]) as an outcome variable to be regressed onto group (placebo [0] vs. test drug [1]), age, and sex (male [1], female [0]). This analysis would tell us the Odds Ratio for death among test drug patients compared to placebo patients while controlling for age and gender. The same analysis can then be used for the other two dichotomous outcomes, change antibiotics (yes vs. no) and complications (yes vs. no). You can power logistic regression with G*Power. My user guide shows how to do this.

Consider powering one analysis, preferably the one with the most important outcome. You could power either the Hotelling’s T-squared or one of the logistic regressions. IRBs and granting agencies usually require one power analysis, usually for the test with the most important outcome variable.

How does this sound?

TY, Ph.D.

1/26/2013 01:18:38 pm

I'm wondering if you've ever done rank ordered logistic regression with grouped data in R. I've been reading about mlogit. It's quite easy in STATA. Any suggestions?

Dave C.

1/28/2013 09:22:53 am

TY,

Sorry. I have not done ordinal regression in R.

DC

Corey link

2/20/2013 04:56:15 am

"There are 2,000 members of the National Academy of Sciences..., of these 2,000 people, 95% of them are atheist and the percentage for the physicists is even higher." I'm looking for an analysis for this second part about physicists. I suspect the author of the quote is dead wrong.

Jeremy

8/5/2013 11:46:26 am

I want to thank you for your page on using G*Power software. I have recently spent over a FULL DAY trying to understand, find examples for, pour through the G*Power manual, read theoretical papers on, and otherwise decipher G*Power's user interface for inputting values when doing sample size calculations for logistic regression. After a lengthy Google search, your website came up on the 6th page of results. All G*Power had to do was give intelligent, step-by-step examples and better explanations of what their input fields are. I realize it's creators are German, and English is not their first language, but still . . . Again, THANK YOU!

Dave C.

8/6/2013 04:21:32 pm

You are welcome, Jeremy. I hope you find the guide useful.

Tam O

8/15/2013 09:25:36 am

I am trying to determine sample size for a two are RCT where the outcome is fall rate (number of falls / 1000 patient days). For the control group, the fall rate is 5.0. I projecting a 50% decrease in the fall rate.

I put in the following:
base rate exp(bo)=0.05
exp(b1)=0.5
mean exposure=1000
Binomial distribution with parameter=0.5 (equal prob).

The total sample size comes up as 3? I must not be specifying the base rate correctly. What am I doing wrong.

Dave C.

8/15/2013 10:15:11 am

Tam,

I think your problem is with "mean exposure". You entered 1000 patient days. The "mean exposure" value reflects the approximate number of days you track the patients in your study, or the average number of days that they are considered subjects in your study. You are probably not following each patient for 1000 days in an RCT. However, if, for example, you tracked falls among inpatients randomized to treatment or control conditions for 30 days, then the "mean exposure" value would be 30 resulting in a total sample size of 84 patients with all other values staying the same.

Tam

8/16/2013 08:21:49 am

Thank you. That make perfect sense. I can use the average length of stay because we are only tracking inpatient falls.

Just to verify, am I interpretting how to specify the baseline rate correctly, i.e. the fall rate for the control group is 5 falls/ 1000 patient days so I am putting in 0.05. Where does the fact that this is the rate relative to 1000 days get taken into consideration in the calcuations?

Dave C.

8/16/2013 09:43:17 am

Tam,

You don't take the denominator (e.g., 1000) into account when entering the base rate. You just enter the percentage. If you expect the base rate falls to be .05 (5%) that is what you enter for Exp(B0).

Tam

9/11/2013 02:47:52 am

Dave,

I am still contemplating the required sample size for the study I am designing looking at in-hospital fall rates. Here is what I know:

Patients have varying lengths of stay.
Fall (yes/no) is a dichotomous or polychotomous (number of falls) outcome.
Research question: After controlling for known covariates, compared to the usual care group, the intervention will have a lower fall rate.
Preliminary data: Fall rates are traditionally reported as the number of falls / 1000 patient days. Suppose that at the facility used in this study, the fall rate is 5 falls/ 1000 patient days, i.e. a very rare event. I also know that 1000 patient days, translates to 100 patients. Hence the proportion of patients falling is 5/100 or 0.05.

To handle this outcome, Poisson Regression seems like the method of choice. I could consider using proportional hazards regression.

My confusion and questions:

In G*Power, is the base rate that I enter the fall rate (5/1000 =0.005) or is the base rate that I enter the proportion of patients falling (5/100 = 0.05)? This makes a big difference in the sample size required. Can you explain based on the statistical model, your answer. Specifically, isn't the dependent variable (fall (0,1) / length of stay) which is usually handled in software by specifying a dichotomous outcome (fall (0,1)) and a lag function (ln(length of stay)). Given this, I can see both definitions for the base rate.

Thank you very much for your thoughts.

Dave C.

9/13/2013 09:44:30 am

If you are interested in the number of falls per case during a period of observation and falls are relatively rare, then it is best to go with poisson. If you are interested in checking for a single fall event and finding out how the risk changes over time for a specific group, then something like survival analysis or survival analysis with covariates (i.e., cox regression) is more appropriate. If you want to run a regression with a binary outcome variable and you are not concerned about survival over time, then binary logistic regression is ideal.

Based on what you told me, I think you have the following values for powering a poisson regression with covariates.

Tail: 1 if you expect the treatment to lower fall rates. 2 if you don't know if the treatment will lower or raise fall rates. 2 tail is often used when people want a slightly more rigorous analysis because it is a more stringent test (i.e., more difficult to reject the null hypothesis).
Exp(B1): increase or decrease in fall rate beyond the base rate. If you expect a 50% decrease, then enter 0.50.
Alpha: 0.05
Power: 80%-90%
Exp(Bo): base rate for falls without treatment. Based on your previous message, it looks like this is 0.05 (5%).
Mean exposure: average number of days patients will be tracked for the study.
R-squared other x: enter the expected R-squared value between the other covariates you want to control or account for in the model and your outcome (falls). R-squared is the coefficient of determination. It is best found by estimating the R value (multiple correlation coefficient) and then squaring. The R value reflects how strongly the other covariates are correlated with falls. Do you expect a small (0.10), medium (0.30), or large (0.50) correlation/relationship between falls and the other covariates (predictor variables)?
X-distribution: binomial
X param pi: enter proportion of cases in the study that are in the treatment condition. E.g., if half are treatment cases, enter 0.50.

Good luck!

Mahmoud

10/28/2013 10:57:42 pm

I'd like to conduct power analysis global effects (to determine sample size) using MANOVA in GPower. The tutorials give info about how to determine effect size. Any suggestions would be appreciated.

Many thanks

Dave C.

10/31/2013 04:50:46 pm

See item #41 in the GPower guide under the GPower tab.

Stephen A.

4/1/2014 08:15:27 am

Hello Dave: Thank you so much for this discussion and the helps provided. Could you clarify your response to Nicholas dated 12/29/2010 re: using the eta-squared value from an analysis conducted in SPSS? Daniel Lakens indicates in his blog (https://sites.google.com/site/lakens2/blog/thefirstruleofnotunderstandingeffectsizesisyoudon%E2%80%99ttalkaboutnotunderstandingeffectsizes) that SPSS uses a different formula for eta-squared than that used by G*Power, and that SPSS values for eta-squared must first be transformed before using in G*Power. I haven't seen this elsewhere but he indicates that he confirmed this with the authors of G*Power.

Thanks,

Stephen A.

Dave

4/1/2014 09:39:45 am

Stephen,

Looking back at the comments, my response to Nicholas (shown below) was for an ANOVA repeated measures - between factors.

"Under the “Select procedure” tab, chose “effect size from variance” and enter the SPSS partial eta squared in the appropriate box. This is the direct approach. Eta squared conventions are small=.01, medium=.06, large=.14. I would compare the results from using eta squared and using the means. They should be similar. If you have preliminary data in SPSS, then ask SPSS to give you an estimate of power in your output file. This should also be similar to the G*Power calculations given that they use the same data."

I have not heard about SPSS eta squared calculations differing from G*Power calculations. I will look into this some more beginning with the link you provided. I've always been somewhat suspicious of power calculations based on effect size estimates. I personally prefer to use estimated means etc. from preliminary data or researcher's expectations. This preference came through in my response to Nicholas where I encouraged him to estimate power using both means and effect size and then compare the results.

Anyway, thanks for the heads up on eta squared!

Stephen A.

4/1/2014 10:21:45 am

Thanks Dave for the lightening-quick response:

In randomized controlled trials the hypothesis of greatest interest for continuous-scale outcomes is usually associated with the group x time interaction effect from a repeated measures ANOVA. Because powering a study for the interaction effect in a trial with one within-subjects factor and one between-subjects factor is complicated (stats texts I have don't cover it), I was delighted to see a sample size estimation algorithm for this in G*Power - ANOVA: Repeated measures, within-between interaction. The program requires an input of the "Effect size f" which can be "Determined" in one of two ways: either from variances or by directly inputting partial eta-squared. Either option can't be estimated intuitively (at least not with my limited intuition) so I've typically used a random number generator to create a dummy data set with estimated variance, baseline means that seem reasonable for the primary outcome variable, changes in means over time that represent a between-group post-treatment minimal clinically important difference, and then with sorting and unsorting of values until I can get a correlation among repeated measures that approximates a best guess of what that will be once data are collected. Running a GLM repeated measures ANOVA in SPSS on these dummy data will yield a partial eta-squared value for the interaction effect which I used to input directly into the field for partial eta-squared in G*Power. However since reading the warning against doing this from Lakens, I've been using his formula for transforming eta-squared before inputting in G*Power. Because the transformation yields a smaller value for eta-squared and thus a larger sample size estimation, I'm hesitant to continue with this if it's not really necessary. I'm wondering if you have any insight on this, or any suggestions for a better approach to estimating sample size for a design where a within-between interaction effect is of greatest interest in the study.

All the best,

Stephen A.

Stephen Allison

4/1/2014 10:46:08 am

So sorry for the multiple replies. I kept getting an error message that my message wasn't submitted and to try again. So I trimmed more and more language thinking perhaps I exceeded a word limit. Then I saw all these replies cluttering the blog (hope you can delete all but one).

SA

Ingrid

2/11/2015 11:33:40 pm

Hi Dave (and others), this is truly a very helpful site and chat. I am hoping you can help me with an effect size question. I am trying to determine was the effect size in cluster randomised trial with the incidence of school exclusion - whether it occurred or not (yes/no). Is there a formula I can use to calculate the appropriate effect size in the treatment compared to control group on a dichotomous variable? Many thanks, Ingrid

Dave

2/12/2015 05:35:13 am

Ingrid,

If your outcome variable is dichotomous (yes/no) you have a few options. You can plan for a chi-square test or better yet a test of independent proportions. There is also binary logistic regression but this is typically used when there are covariates (other variables) that you need to control for.

To power a test of independent proportions see item 5 in the G*Power guide (Poportions: Inequality, 2 Independent Groups (Fisher’s Exact). Enter the proportion of expected "yes" responses for the treatment group (Prop 1) and the proportion of expected "yes" responses for the control group (Prop 2). Or you could compare expected proportion of expected "no" responses instead.

Anna

3/25/2015 04:19:32 am

Hi Dave
Thank you for this website, here has clear explanation for use G*power.
I am a fresh user, it took me lot time to familiar with this software.
After studying the contents of ANOVA, I was confused.
The example for ANOVA: repeated measures, betweens factors,
The number of group were 2 genders, but why the sample size can enter 45??
It should enter 30, isn't it?
Total sample size is calculate with "number of groups" or "number of measurements"?
Excuse me if i asked stupid questions.
Many thanks,
Anna

Dave C

3/25/2015 08:15:23 am

Anna,

You are correct. Thanks for pointing out this error! I have updated the instructions for the repeated measures, between subjects ANOVA. Let me know if you need more help with your analysis.

Arnon

5/19/2015 02:05:44 pm

I am a veterinarian (with a limited understanding in advanced stats) who is looking for help in calculating the effect of size f for the calculation of sample size. I am conducting a prospective, repeated measure, interventional-clinical study. I am looking at two dependent variables in 3 groups of dogs. Dogs will be sampled 9 times over a period of 72 h. For one of the dependent variables there is some data from a pilot study and for the other there is none. In the pilot study, the high and low over a period of 48 h was 360 and 120 in one group and 2500 and 1000 in the other group (the third group that I'm having did not exist in that study). If someone can please explain to me in simple words how to calculate the effect size f either from variance or direct (via the determine button) than I'll appreciate it greatly. I've tried to follow the explanations in this website but unfortunately it is not enough...
Thanks
Arnon

Dave C.

5/20/2015 04:53:26 am

Arnon,

If you are taking 9 repeated measures over a 3 day period and there are two outcome variables that you are recording at each measurement then it sounds like you will be running a repeated measures multivariate ANOVA (sometimes called doubly multivariate ANOVA). G*Power does not have a test for this which is why I included my own instructions on how to run a power and sample size calculation in the G*Power guide. Those instructions are found in item 50 in the guide. The process involves randomly generating fictional data based on expected means and standard deviations or variances. This can be done in a free program like R or at a website called Motulsky's Graphpad. You then analyze the fictional data with a software program and ask the program to give you estimated power.

5/31/2015 06:07:36 am

Hi, I am running an a priori mixed model ANOVA study using gPower and I need to indicate the sample size, however, I am not sure what correlation among repeating elements value to input. I can't find any information from past research over what this correlation value might be as my study hasn't really be done before. Should I just leave it at the default of 0.5 with power at 0.8? Thanks

5/31/2015 06:09:41 am

*need to find out the sample size for my dissertation. Thanks

Dave C

5/31/2015 08:44:25 am

N,

Correlation among repeated measures are usually high. Values that are higher during one time, relative to other measures, are usually higher at other times. Similarly values that are lower, relative to other people's meaures, are usually lower at other times. Thus if the measures are expected to show the same higher versus lower realtionships across time then the expected correlation will be high. A correlation of 0.50 is the bottom end of the high range and would be a reasonable, conservative estimate.

6/1/2015 04:43:05 am

Thanks but I'm assuming the values that you input are based upon similar past research on the topic of investigation - supporting your point that if values are found to be large in past research you would enter a higher correlation among repeating elements compared with if the values found were lower in past research. I was asking what if you're running a study where we cannot make a direct comparison with past research over what the correlation among repeating elements would be. Would you just keep it at 0.5 as you have no past research/justification to input a higher/lower value? Thanks again!

Dave C.

6/1/2015 05:43:48 am

As I understand it the correlation among repeated measures refers to change in scores over time, comparing one person's scores over time to another person's scores over time. This is what I meant by "relative to other people's measures" in my previous email although I probably did not explain it very well.

Let's say that person 1 has the following six measures over six days (one measure taken per day): 3, 4, 5, 6, 5, 6. Now let's say that person 2 has the following six scores over 6 days: 2, 3, 2, 6, 5, 7. (Notice that there is an increasing trend which bodes well for finding a within subjects effect across time.)

To run the power analysis we need the correlation among repeated measures. The correlation between these two sets of data is 0.81. As one goes up so does the other. A higher positive correlation among repeated measures increases the power which makes sense because as both sets of scores go up so does the chances of finding a significant within subjects effect. If researchers have no idea of what to enter, the default 0.50 is a safe bet.

Let me know if you find a different interpretation of correlation among repeated measures. Thanks.

6/1/2015 09:30:16 am

Thanks, it all makes sense now.

7/17/2015 07:35:03 am

Hi Dave, I need your help to get the sample size for poisson regression. My study is to find out if gender is a factor of a disease. No treatment is involved in the study. The information I have:

a)Based on literature, the infection rate of male, Exp(Bo)=0.88
b)The samples I collected: Male=84%, female=16%, total=447
Can I know how if sample size is sufficient? Thanks.

Dave C.

7/17/2015 03:57:56 pm

SM,

The instructions on powering a Poisson regression in GPower are in the GPower guide, item 30A or 30B. Good luck.

7/18/2015 05:31:13 pm

Yes, I've read it & used the following for my calculation:
poisson regression, A priori
a)Tail=1, alpha=0.05, power=0.8, mean exposure=1 & R sq=0
b)Exp(Bo)=0.88
c)X=binomial, X pi=0.84

I would like to know if my X pi=0.84 is correct (84% of the data collected are male)? And, if Exp(B1)=0.7 means I want to know the sample size needed to detect a 30% decrease in male of the disease?
Thank you

Dave

7/21/2015 11:26:35 am

Oops. I was thinking absolute % reduction like there is in a test of proportions. Exp(B1) represents the amount of increase or decrease in response rate, not the actual rate. So if the decrease is 30% then the Exp(B1) value would be 0.70, or if the expected decrease is 15% then the Exp(B1) would be 0.85 (subtract the expected decrease from 1.0). I only recommend using division to get the Exp(B1) if you know the unit of exposure that you will be using. I am not sure that you mentioned the unit of exposure in your emails.

Dave

7/20/2015 07:19:53 am

Based on my limited understanding of the study it sounds like you have Exp(B1) = 0.58 representing a 30% decrease for females over the base rate of 0.88 for males.

If Exp(B0) = 0.88 for males then it sounds like males are considered your base rate group. The "X parm pi" value is supposed to represent the proportion of cases belonging to the treatment or Exp(B1) group which, from what you've described, appears to be females. So the X parm pi value should likely be 0.16 rather than 0.84.

7/20/2015 05:32:27 pm

Thanks for your reply. Can I know how did you get the Exp(B1)=0.58? Shouldn't it be Exp(B1)= infection rate of female / infection rate of male = 0.12/0.88 = 0.14?
Thanks

7/21/2015 08:10:36 pm

Thanks, hope you can clear my final doubt:
Exp(B1)=0.7 represents the 30% decrease in response rate of female (X=1) as compared to male (X=0)?
Thanks.

Dave

7/22/2015 04:30:22 am

I can't confirm whether your Exp(B1) is correct as I don't know the exact details of your study and I don't know your unit of exposure. I recommend consulting with a statistician if you have doubts. In any case, Exp(B1) represents the increase or decrease beyond the baseline Exp(B0) response rate. Good luck.

7/22/2015 06:23:54 pm

Thanks a lot, Dave. You have helped me to understand it better now. Thank you.

Amber Ward

10/29/2015 03:29:00 pm

Hi there
I have a question about the "ANOVA repeated measures, within-between interaction" power analysis. The instructions are very clear on your website. My question is about filling the "correlation among measures" and "Nonsphericity correction boxes". You have advised that the correlation among measures will probably be high, so I am leaving that at .5. Can you offer any advice about estimating the nonsphericity? Thank you.

Amber Ward

10/29/2015 03:36:48 pm

Me again :-) Also, I wondered if the "Total sample size" value yielded truly reflects the total sample deemed necessary, or rather the number of participants in each group (between subjects levels). Thank you!

Dave C.

10/30/2015 03:31:10 pm

"Total sample size" likely refers to the total number of participants needed in your repeated measures study, not the number needed in each level of the between subjects factor.

Dave C.

10/30/2015 03:28:09 pm

Hi Amber,

I've added a clearer explanation of the nonsphericity correction value in the GPower guide. Sphericity refers to the variances of the differences between levels of your repeated measures variable. If you expect the variances to be relatively the same then you can assume sphericity and can leave the default value at 1.0. You could change the value to something less than 1.0 if you have reason to believe that the sphericity assumption will not be met. Most people will have no idea so the value can probably be put at 1.0. When sphericity is not met the Type I error rate increases, so if you want to protect against this error then go ahead and enter a value like 0.90. All it will probably do is slightly increase your sample size requirement.

Amber Ward

10/30/2015 03:33:25 pm

Awesome - thank you for answering all my questions (no need to answer the last one about post hoc tests - I think it's a bit of a no-brainer really!). All the best, and again, thank you - this site is pretty invaluable! Amber.

Amber Ward

10/29/2015 03:50:43 pm

Oh and a final question (goodness knows why I can't think of them all together)... can I assume that following the calculation for the ANOVA omnibus test, can I assume that any posthoc comparisons are going to be adequately powered? Thank you so much.

Dave C.

10/30/2015 03:35:52 pm

Generally speaking post hoc comparisons of levels of a between subjects factor will be significant if the omnibus F test is significant, so I think that in most cases an adequately powered Omnibus test will also adequately power post hoc tests, but one cannot be 100% sure.

Amber Ward

10/30/2015 03:49:36 pm

I agree. And I also think that since the omnibus test is intended in part to protect against Type I errors, it probably requires more power than a t-test anyway. So if my sample is large enough for ANOVA, it should be large enough for the t-tests. Thanks again. :-)

Nadine

6/17/2016 06:49:26 pm

Hallo, I am currently writing my Bachelor-thesis in Psychology and need to calculate the necessary sample size using G*Power. I am very glad, that I came across your tutorial, but still I have doubts that I calculate it all correctly.
I have a 2x2 design (both factors have two levels) and I have 7 dependent variables:

Is it correctly to use MANOVA: Special effects and interactions

Is it correctly to write into G*Power the following:

number of groups: 4
number of predictors: 2
response variables: 7

What do I need to write in at effect size, when I assume (according to previous studies) a medium effect size? 0.0625 (I found this on the net somewhere, but without an explanation and it's what G*Power offers as default setting.

I really hope you can help me and understand what I wrote (my English is not the best).

Thank you,
best wishes, Nadine

Dave C.

6/19/2016 09:20:08 pm

Greetings Nadine,

I understand your research design. Should you use MANOVA: Global Effects of MANOVA: Special Effects and Interactions? The answer depends on what you are most interested in testing. Are you mostly interested in the main effect of one of your factors? If so then you should use manova: global effects. Are you mostly interested in the interaction between your two factors? If so then you should use manova: special effects and interactions.

If you are running a "manova: special effects and interactions" power calculation, the values you mentioned look correct.

If you are running a "manova: global effects" the number of groups will be 2 (one factor has 2 levels). If you have a main independent variable that you are most interested in (like treatment), you should run the power calculation for that factor. If the other factor is something less important like gender you could skip running a power calculation for that factor. If both factors are equally important you could run a separate power calculation for both.

The most difficult part may be estimating Pillai's V. I suggest that you do this by creating some fictional data that represents the data you expect to get then run a manova and use that estimate of Pillai's V in your G*Power calculation. You might also find Pillai's V conventions online.

How does this sound?

Nadine

6/24/2016 01:35:57 am

Thank you for your quick reply. The mail was put in Spam-Folder :-( so I only realised today, that there was a reply.

Sounds good :-)

I will try to find a Pillai`s V convention online, as I have no idea how to create the fictional data.

Thanks again for your help!

best wishes,
Nadine

Simon

7/27/2016 09:22:52 am

Hi. I wondered if you were aware of any resources for power analysis of ANCOVA or if there was some kind of clever work-around that could be applied in GPower. Thanks, Simon

Dave

7/29/2016 01:20:10 pm

Simon,

I suggest running your analysis as a regression with a main predictor variable (probably a between groups factor in your case) and covariates. ANOVA and ANCOVA are basically equivalent to regression, just different mathematical techniques to reach the same conclusion.

Celia

10/17/2016 10:53:35 pm

Hello,

I was wondering if you could direct me to the type of G*Power a priori test that can estimate sample size based on number of predictors, power, p level, BUT (and here might be the tricky piece) outcome base rates.

Is there a way to estimate power using outcome base rates as an estimate parameter?

Kind regards,
Celia

Dave C

10/18/2016 10:02:33 am

Hello Celia,

If by base rate you mean the number of observed events over a period of time, you might want to look at the poisson regression. The poisson power calculations are #30A and #30B. Use 30A if your main predictor variable is a continuous variable and use 30B if it is a categorical predictor. Let me know how this goes.

Michael link

11/22/2016 03:23:13 am

Hello,

I am trying to calculate "a priori" the appropriate number of samples to use for a MANOVA study.

So my question is about item #42 "MANOVA: Special Effects and Interactions" in your documentation.

For "MANOVA: Special effects and interactions", is G*Power's "Total sample size" output parameter the recommended sample size for **each** group, or is it the total size of all samples added together for **all** groups?

QUICK BACKGROUND:
---------------------------------------------

For example, I have a factorial MANOVA study, with 4 independent treatment variables (call them 'A', 'B', 'C', and 'D' for simplicity), and I have 2 dependent variables.

First, I know I need to use MANOVA because I have more than 1 dependent output variable, the independent variables are categorical, and my dependent variables are quantitative.

My independent variable 'A' has 4 levels.
'B' has 3 levels
'C' has 2 levels
and 'D' has 2 levels

Multiplied together, A * B * C * D, this yields: 4 * 3 * 2 * 2 = 48 treatment groups in my factorial MANOVA study.

My confusion is with G*Power's recommended sample size.

STEPS TO REPLICATE:
------------------------------------------------

Here's what I did (following item #42 in your documentation):

1) Select "F Tests"
2) Select "MANOVA: Special effects and interactions"
3) Select "A priori: Compute required sample size"

Then enter the following data:
Effect Size: 0.0625
alpha err prob: 0.05
Power: 0.95
Number of groups: 48 (i.e., my 4 * 3 * 2 * 2 = 48)
Number of predictors: 4 (i.e., my 4 independent variables)
Response variables: 2 (i.e., my 2 dependent variables)

Results: "Total Sample Size" output = 188

QUESTION AND CONFUSION:
----------------------------------------------------

Does this mean 188 samples are recommended for **each** of my 48 groups? If so, that means 48 * 188 = 9024 total.

Or, does that mean 188 samples **in total**, thus 188 / 48 groups = 3.9167 samples per group? (which rounds up to 4)

Here's why I'm confused, and this is the freaky part (I wonder if it's a bug in G*Power):

If I change the input "Number of Groups" value from 48 to 70, while keeping the rest of the inputs the same, I still get the same "Total sample size" output result of 188.

It seems strange... I can increase the input number of groups by almost 50%, yet I still get the same "Total sample size" output?!?

This made me wonder if the recommended "Total sample size" was per group, instead of in total.

So which is it?

Does "Total sample size" in this MANOVA case mean "per group", or is it the total count of all samples used, for the groups combined? There is quite a difference (as shown above).

Thank you for your help, and for your excellent G*Power documentation.

Kind regards,
Michael

Michael link

11/22/2016 03:48:22 am

Hello,

Another MANOVA question, this one similar to the question posed by Nadine on 6/17/2016 06:49:26 pm:

"MANOVA: Global effects" vs. "MANOVA: Special effects and interactions".

---------------------------------------------------

For my first question I posed earlier with 4 independent variables, and 48 treatment groups (recall, 4 * 3 * 2 * 2 = 48), what value for "Number of Groups" would I use in a "MANOVA: Global effects" study instead?

Recall, the 4 independent variables are:
"A" (with 4 levels),
"B" (with 3 levels),
"C" (with 2 levels),
and "D" (with 2 levels).

I have 2 dependent variables.

For Nadine, she had a 2 x 2 factorial study (i.e., 2 * 2 = 4 treatment groups in total), where each predictor had 2 levels.

You suggested that she use 2 groups. Was this because she had 2 predictor variables?

If so, then in my case I have 4 predictor variables (i.e., independent variables A, B, C and D).

So if I were to do a "MANOVA: Global effects" statistical test in G*Power, would I use "4 groups" (1 per predictor variable)?

Or would I use 48 groups (i.e., A * B * C * D = 4 * 3 * 2 * 2 = 48), like I did for "MANOVA: Special effects and interactions" ?

Or something else? Please advise, thank you.

Kindest regards,

Michael

Dave C

11/23/2016 12:51:25 pm

Michael,

Power analysis #42 assumes that you are interested in interactions. If you have 4 between groups factors with 4, 3, 2, and 2 levels then you will generate 4*3*2*2 = 48 interactions. Powering 48 interactions will require a lot of cases so it is safe to say that the 188 cases you got refers to the number required per group for your moderate effect size of 0.065.

If you are not interested in powering 48 interactions I suggest using power analysis #41, MANOVA global effects. With this analysis you would power each between groups variable separately. In other words you would power variable A (4 levels) by itself, then power variables B (3 levels), C (2 levels), and D (2 levels) separately. Actually, when it comes to data analysis you might consider analyzing each grouping variable separately because a MANOVA with 4 grouping variables will be difficult to interpret.

So if you used method #41 for variable A (4 levels) you would enter Number of Groups = 4 because there are 4 levels in your grouping variable, and you would enter Response Variables = 2 because there are 2 outcome/dependent variables. This answers your follow up question about number of groups to enter.

Variables C and D could also be powered with method #40, Hotellings T-squared for independent groups because there are only 2 levels in each grouping variable.

Michael link

11/28/2016 12:42:34 am

Thank you, Dave!
I will reassess my plan, based on your input. Hope you had a happy Thanksgiving, and thank you again for your help.

-Michael

Sonia

2/15/2017 03:32:49 am

How does the no of measurements in Repeated Measures affect the sample size number?

Dave

2/15/2017 11:48:41 am

As the number of repeated measures goes up, the sample size requirement usually drops.

Joy

5/16/2017 07:50:15 pm

Is there a way to do an a priori power analysis for a doubly MANOVA? I think that is the most appropriate test for what I want to do...

I have 1 between subjects independent variable (treatment with 3 levels) and 1 within subjects independent variable (time with pre and post). I then have 6 dependent variables that will be measured at pre and post. I also have subject variables that may need to be added as a covariate.

Is there a way to determine the necessary sample size to detect a medium effect with a power of .80 and alpha of .05?

Dave C

5/22/2017 12:20:41 pm

Unfortunately GPower does not run doubly multivariate RMANOVA (i.e., repeated measures with multiple outcome variables).

Joy

5/22/2017 01:21:47 pm

Thanks for the reply. Do you know of another software to do an a priori power analysis for RMANOVA or can power analysis only be done post hoc for RMANOVA?

Dave C.

5/22/2017 01:54:38 pm

I think that NCSS PASS can power a repeated measures multivariate ANOVA (RMMANOVA). The software license is fairly expensive but you can download a trial version to use for free for 1 week. Another option is to create some fictional data based on your expected outcome and put the data through RMMANOVA, using a program that will estimate power when it runs the analysis. SPSS will estimate power in a RMMANOVA. You can also use pilot data if it is available.

Tom A

9/11/2017 05:29:11 pm

Hi Dave,

I would like to echo a question above from years ago that I did not see answered.

How do you calculate power for a repeated-measures ANOVA with multiple within-subjects factors--I want to do a calculation for a within-within interaction, not within-between.

Thanks for any advice you have!

Best,

Tom

Dave C.

9/11/2017 05:56:52 pm

Tom,

When I created the guide the makers or G*Power had not published a way to power 2 or more within subjects factors. There are other power calculations I wish they would add to their program. Anyway, there might be a method available in R software, although I am not aware of a package for that.

Tom

4/30/2018 01:05:25 pm

Thanks for the reply. I came across the link below, which suggests that you can treat the within-subjects "difference between differences" (e.g., comparison of two change scores) as you would any other within-subjects comparison. In other words, you just compute the power for the within-subjects t-test.

http://core.ecu.edu/psyc/wuenschk/MV/RM-ANOVA/RM2x2Interaction.doc

John Xie

10/7/2017 07:33:53 am

Thanks Dave. As a statistics support officer, I am preparing a tutorial for using GPower3.1 for power analysis and found your website. It is very helpful with the information you have generously shared with us. Thank you again.
John

Malcolm Horn-Charnesky

4/29/2018 12:37:32 pm

I am doing a 2x2 Factorial ANOVA and have a question about my G*Power analysis.
When I select what I am going to run, I choose “ANOVA repeated measures between factors” and “A Priori compute required sample size given α, power & effect size.” I notice that it automatically selects a small effect size (.25). When I run it that way, I get a sample size of 176. When I run it with a medium sample size (.50) I get a sample size of 48. I have attached images of my screenshots of the calculation so you may see them.
My questions are:
• The sample size provided (either 176 or 48) is per group, correct? That would man I would need an N of 704 or 192, respectively. I just want to verify if that is per group or the total sample.
• Should I use the small or medium effect size? I always thought using a medium effect size was best so it threw me off when G*Power automatically selected a small sample size.

Dave C

4/30/2018 03:32:42 pm

Malcolm,

The sample size provided is the total (all subjects). For example, if the total sample size is 50 and your between subjects factor divides the subjects into two groups, that would be 25 subjects per group.
The effect size you select depends on your expectations. GPower probably defaults to a small effect size as it is a conservative estimate.

Joe Laffoon link

5/22/2018 01:18:29 am

Hi Dave,

Thanks so much for the site and youtube videos!

I'm designing a 2x3 repeated measures within-between interaction study. I believe this is a 2 way mixed design study. Specifically, I'm looking at the impact of training and distance between targets on task completion time a GUI.

IV: Training, Distance between targets
DV: Reaction time, Total time to completion, # of errors
Subjective DV: 10 item Likert scale on perceived usability (systems usability scale)

Between factor levels = Training (trained vs untrained)
Within factors levels = Distance (close, medium, far)

If I'm not wrong, my study is very much like the one you posted on youtube. I'd like to be sure that:

1) 28 is the total sample size need for my design (14 trained and 14 untrained)

Is that the case?

Best,
Joe

Dave C.

5/22/2018 12:52:49 pm

Joe,

The repeated measures within-between interaction method (#38) is the best approach to use if you are mostly interested in the interaction between training (yes/no) and distance (close, medium, far). I only bring this up because most people are not interested in the interaction. If you are interested in the effect of the between (training) variable then chose the repeated measures between factors method (#36). If you are most interested in the effect of the within variable (distance), use the repeated measures, within factors method (#37). If the between and within effects are equally important, you could run both power calculations separately.

It looks like you have multiple DVs. Methods 36, 37, and 38 are univariate in the sense that they are for one DV. So you could pick one DV to use in the power calculation or run a separate calculation for each DV. If you want to run a power calculation with all DVs, you could use one of the multivariate (MANOVA) approaches.

Whether N=28 is adequate for your study depends on a few things. When running a repeated measures within-between interaction power analysis, you need to specify the effect size for the interaction. This amount is called eta-squared. You also need to specify the correlation among repeated measures and the sphericity assumption. You are probably fine to use the default values for 'correlation among repeated measures' and 'sphericity'. The expected effect size (eta-squared) is more important. I cannot say whether N=28 is adequate without knowing the eta-squared.

Joe

5/22/2018 03:32:27 pm

Thanks for the quick reply!

This is incredibly helpful and much appreciated.

Regarding effect size: I've been using an effect size of 0.25 as the literature doesn't speak to the specific distances or degree of training that I'm looking at.

I'll explore running some of the measurements you mentioned (36 & 37) and see the potential differences in sample size. This is part of my MS and to be frank what is primary in the study may depend on how many participants I can afford to recruit as I'm using a specialized population. Additionally, I'll explore using a MANOVA.

I think number 45 is the MANOVA for me.

Thanks agian

Joey

7/23/2018 01:16:05 pm

Hello Dave,

Really great that people like you take time to compile information on G*Power and other science related topics!

I haven't read through everything, but I'm currently trying to understand Logistic Regressions better in G*Power and think that there might be a small mistake in your explanation of "R2 other X." The way you discuss it makes it sound like you interpret this to be "R2 increase from your variable of interest." But the way that G*Power Manual refers to it makes it sound like it's suppose to be the covariance between the variable of interest and all other predictors in the model.

I've copied the relevant text below:

In models with more than one covariate, the influence
of the other covariates X2, . . . , Xp on the power of the
test can be taken into account by using a correction factor.
This factor depends on the proportion of the variance of X1 explained by the regression relationship with X2, . . . , Xp. I

Just a small clarification that might be useful to update! Please correct me if I'm wrong in my understanding of the manual, which can be found in its full text here under "28.1" http://www.gpower.hhu.de/fileadmin/redaktion/Fakultaeten/Mathematisch-Naturwissenschaftliche_Fakultaet/Psychologie/AAP/gpower/GPowerManual.pdf

Best,
Joey

Dave C

7/24/2018 12:47:31 pm

Joey,

Thanks for your attention to the details. I've tried to express the "R2 other X" in the way you suggest it should be expressed, that is "the covariance between the variable of interest and all other predictors in the model." My Gpower instructions state "R-squared represents the amount of variability in the main predictor (BMI) that is accounted for by the covariates." The so called 'variable of interest' is the main predictor variable. This would be the same thing as "the proportion of the variance of X1 explained by the regression relationship with X2" as specified in the GPower manual. I would appreciate anymore insights you have. I have made corrections in the past.

Joey

7/24/2018 01:38:48 pm

Hello Dave,

I appreciate your reply! Based on your response I think I misinterpreted the G*Power manual. Let me see if I can elucidate what I think G*Power manual is actually saying and what I was thinking it said:

1. You (and G*Power manual) say that "R-squared other X" means the variance in the dependent variable that is explained by variables other than X1 which are included in the regression. For example, if your regression was simply:

y ~ X1 + X2

Then are you saying that "R-squared other X" means the partial R-squared of X2? Otherwise known as the amount of variance X2 explains in Y independent of X1.

2. What I interpreted G*Power as saying is that "R-squared other X" should capture the correlation between the predictor variables and your variable of interest (in this case X1). So I thought that this meant that you should correlate X1 and X2. I now think this is wrong, since it's possible that you could have two predictors that are perfectly correlated (r = 1) but this would mean that the partial R-squared of X1 and X2 would be 0.

If you could clarify whether my summary is correct or not that would be very beneficial to my understanding. Is it appropriate to summarize "R-squared other X" as the partial R-squared of all other predictor variables? Part of me thinks that this can't be true in cases of more than two predictors.

Finally, would the practical way to determine the "R-squared other X" be to simply run the regression without including the variable of interest (e.g., X1)?

This would look like:

y ~ X2 and using that R-squared of the model as the input parameter for "R-squared other X."

Again, I apologize for the essay and my initial confusion. G*Power tutorials are rare to come by and I wanted to make sure my understanding was correct before finalizing my power analysis.

Best,
Joey

Joey

7/24/2018 02:35:48 pm

After much head scratching and reading, I think I misunderstood your original post and G*Power's manual.

Everything in your original post is correct and the clarifications I wrong in my previous reply (7/24/2018 01:38:48 pm) are wrong.

Thank you again for the response! I now understand that the "R2 other X" refers to the coefficient of determination between your variable of interest (in your original post BMI) and other predictors.

Best,
Joey

Charlotte

8/20/2018 06:33:34 pm

Hi Dave,

thank you so much for taking the time to compile all this very useful information for the rest of us! A question on ANOVA with repeated measures, within-between interaction:

I am planning a study where stress is measured in two groups (treatment vs. control) on two occasions (in two different phases of the menstrual cycle - this is expected to make a difference with regard to the outcome variable). So my within-subjects variable is time (point in time in the menstrual cycle) and my between-subjects variable is treatment group (drug vs. control).

1. Can the sphericity assumption be assumed to be met because I only have two measurement points?
2. As I am expected the outcome variable to differ according to treatment group, that should give me a lower correlation among repeated measures, right? Any idea what type of value I should input?

Thank you so much for your time and effort, I really appreciate it!

Kind regards,
Charlotte

Dave C.

8/21/2018 11:12:24 am

Charlotte,

Yes, sphericity can be set to 1.0 in your power calculation for the reason that you mention.

The correlation value among repeated measures can probably be set to 0.50, indicating a moderate to strong correlation. If you expect the relationship among stress levels to be fairly constant from time1 to time2, then a strong value (0.50) is warranted. By constant I mean that if someone's score is fairly low at time1, it will be lower at time2 than the time2 score for someone who scored higher at time1. In other words, people who score lower at time1 tend to score lower at time2 relative to people who score higher at time1 and time2, even though their scores are changing due to a treatment variable. Personally I only enter a correlation value less than 0.50 when I expect erratic scores from my participants across time due to some known confounding variable I am unable to control.

Note that this power calculation assumes that you are mostly interested in the interaction between time and treatment. If you are largely interested in the treatment variable, the ANOVA Repeated Measures, Between Factors would be a better fit. If you are mostly interested in the within (time) effect, the Within Factors would be a better fit.

Charlotte

8/22/2018 03:45:10 pm

That makes perfect sense, thanks a million!

Julian

3/18/2019 04:45:52 am

Hello Dave,
Thanks for providing so helpful resourse on Gpower.
I wouldlike to ask a question concerning ANOVA repeated measures, within-between interaction. In the G power program there is a OPTIONS settings down below. I would like to ask, which one of effect size specification is correct (as in GPower - which is a default option, as in SPSS, as in Cohen - which is recommended). And what is the difference between them?
I will be grateful for your response, kind regards,
Julian

Dave C.

3/19/2019 05:13:52 pm

It appears that the Options tab you refer to is allowing you to choose different conventions for interpreting or computing eta squared levels.

Jody Finnegan

7/1/2019 11:29:56 pm

I cannot figure out how to conduct a G*power Post Hoc for a Spearman's rank order correlation. The result of Spearman’s rho analysis revealed a weak yet positive (.284) statistically significant (p = .007). There were 88 usable cases in my study.
Do I use test family= Exact
Correlation: Bivariate Normal
Post Hoc
Tails =Two. (I used two)
Correlation p H1 = .284 ??
a err prob= .05 or .01 or .007 ???
Total Sample =88
Corr p H0 = 0

I just cannot find out how to do this correctly
Jody

DaveC

7/3/2019 12:49:37 pm

Jody,

Good question. The spearman ranks correlation is computationally identical to pearson's correlation. The only difference is that spearman ranks uses data that are either ordinal (ranked) or non-normal. So this means that you can use the t-test: Correlation: point biserial method in GPower for powering ordinal data. It would also be fine to use the Exact: Correlation: bivariate method. Good luck.

tuzi

3/10/2020 08:56:11 am

Thank you so much for the excellent website!
I have a problem , really need to ask for your help.
My study is using the structural equation model, with one moderator.
would you please give me some suggestion about how to use g power to calculate sample size?

looking forward to hearing your response!

DaveC

3/10/2020 11:38:20 am

GPower will not power a SEM. I have never run a SEM so I can't help you either. However, SEM is an exploratory method. Exploratory methods usually do not require a power and sample size calculation. Just choose a reasonable number of participants.

Nikoletta

3/19/2020 01:23:55 pm

Hi Dave,

Thank you very much for your blog!
I just want some advice... I want to do an a priori analysis in G*power to determine the sample size. I am testing the effects of 4 different devices (between-factors) on a personality trait. The trait will be measured before (baseline trait), after the exposure to the devices, and 2 weeks after the exposure to the devices. My question is, should I use the "ANOVA: repeated measures, between factors test" or the "ANOVA: repeated measures, within-between interaction"? I can't understand the difference of these 2 options. Thank you very much,
Nikoletta

Dave C.

3/19/2020 02:33:32 pm

It depends on what you want to compare. If you have 4 different groups and each group receives a different device, and your purpose is to compare each group, then you would use ANOVA: repeated measures, between factors test. The "within-between interaction" analysis is not used very often. It is for powering the interaction between your repeated measures effect (change over time) and the between effect (difference between groups).

Nikoletta

4/6/2020 07:47:30 am

Hi Dave,

Thank you very much for the reply, I understand it now.
Also, have been searching how to do a sample size calculation for nested (hierarchical) ANOVA but I didn't find anything. Do you perhaps know how to do this with G*power? Thank you very much!

DaveC

4/8/2020 11:17:40 am

Nikoletta,
I don't think GPower has an option for nested variables in ANOVA. You could power each between groups factor in a separate power calculation.

Nikoletta

4/20/2020 07:58:43 am

Hi Dave,

Thank you for the advice!

Naomi

4/7/2020 10:30:18 am

Hi Dave,

Thank you very much for your blog!

I just want some advice... I want to do an a priori analysis in G*power to determine the sample size. I am looking for an interaction effect I found in another study, and am using the effect size of the interaction to compute my sample for my current study.

I am testing one between group factor that has 2 levels (e.g. group) and one within subject factor (e.g. pre and post treatment).
My question is, should I use the "ANOVA: repeated measures, between factors test" or the "ANOVA: repeated measures, within-between interaction"?
I am looking for an interaction effect, but these different tests yield huge differences.
If you can explain the usage of each and which is appropriate in this case that would be great :) I got super confused.
Thank you very much,

DaveC

4/8/2020 11:05:30 am

Naomi,
If you have one between groups factor (2 levels) and a within subjects factor (pre and post) and you are interested in powering the interaction, you should use "ANOVA: Repeated measures, within-between interaction." This is power analysis #38 in the GPower Guide.

Yamuna

5/31/2020 11:37:02 am

Hi Dave,

Need your advice on post-hoc power calculation for ANOVA: repeated measures, within-between interactions. I performed an experiment to test the longitudinal treatment effects by measuring the dependent variables (6 variables) at 5 time points using 4 groups (1 control, 1 placebo and 2 treatment groups). I applied mixed model ANOVA (Treatment; time; Interaction effect of treatment X time), SPSS to analyse the data set.

I have been advised to provide the power of the study in the PhD thesis. To use Gpower for post-hoc analysis (a) effect size; (b) Correlation among rep measures; and (c) Nonsphericity correction should be provided.

I am little confused about these values and want to clarify as follows:
(a) Can we choose partial Eta square of “Interaction effects” (Treatment*Time) from the table “Tests of within subject groups”?
(b) For variance explained by effect and within group for calculating the effect size, how to get those values from SPSS?
(c) For correlation among repeated measures: How to find this value? Should dependent variables from all treatment groups be cumulated for each visit, find correlation between each time points and then use the maximum or mean correlation value?
(d) For Nonsphericity correction: Should Mauchly’s W value from “Mauchly’s test of Sphericity” be used for this or provide 1 if p>0.05?
(e) Should the power analysis be calculated for each dependent variable?
(f) If any dependent variable is not significant, should that not be considered as variable for power analysis?

Kindly guide further!

Thanks in advance!
Yamuna

Dave C

6/2/2020 09:03:36 pm

Yamuna,

Because you have 6 dependent variables and repeated measures, you need a power calculation for repeated measures multivariate analyses (RMMANOVA). Unfortunately GPower does not have an option for running a power and sample size calculation on RMMANOVA. I suggest searching the web for options or asking your dissertation committee for guidance. I know that SPSS will provide a power estimate for RMMANOVA (also known as Doubly Multivariate ANOVA). You could just select the show power option in your SPSS analysis and report that to your committee.

Your comment will be posted after it is approved.

Online Stats Help for Friends

Leave a Reply.

About Me

Archives

Categories