Hosmer-Lemeshow statistic: susceptible to scale?

Hosmer-Lemeshow statistic: susceptible to scale?

Post by Diego » Wed, 11 Jun 2003 14:18:02



Sorry to cross post, but I need some help badly. I am currently
evaluating a predictive model that was built using logistic
regression, and using the HL test statistic to do this. I, however,
have found that the larger the sample size that I get, the smaller the
value of the statistic.

Has anyone experienced a similar problem? TO me, it does not make
sense that the larger the sample, and given similar patterns in
observed and expected values, we should observe a significantly
different HL statistic; however that is exactly what I am getting.

Any comments / suggestions on why could this be happening will be most
appreciated!!

Diego Villaveces

 
 
 

Hosmer-Lemeshow statistic: susceptible to scale?

Post by Rich Ulric » Wed, 11 Jun 2003 23:58:22



Quote:> Sorry to cross post, but I need some help badly. I am currently
> evaluating a predictive model that was built using logistic
> regression, and using the HL test statistic to do this. I, however,
> have found that the larger the sample size that I get, the smaller the
> value of the statistic.

> Has anyone experienced a similar problem? TO me, it does not make
> sense that the larger the sample, and given similar patterns in
> observed and expected values, we should observe a significantly
> different HL statistic; however that is exactly what I am getting.

Is that the HL  statistic for goodness of fit?
 - the one that compares the fit of the prediction to
the observed probabilities, by decile-group?
 - the one that warns of over-fitting?
 - the one that says, If this is BIG, you have a PROBLEM despite
what the significance test of regression (elsewhere) says?

Yes, I would hope that  if it starts out BIG, I should be so lucky
that it might be merely because the sample is too small;  
and that it should get better (smaller) when I add cases.

If that is not the test you refer to, then I don't know.

--

http://www.pitt.edu/~wpilib/index.html

 
 
 

Hosmer-Lemeshow statistic: susceptible to scale?

Post by Frank E Harrell J » Thu, 12 Jun 2003 03:26:29


On 9 Jun 2003 22:18:02 -0700


> Sorry to cross post, but I need some help badly. I am currently
> evaluating a predictive model that was built using logistic
> regression, and using the HL test statistic to do this. I, however,
> have found that the larger the sample size that I get, the smaller the
> value of the statistic.

> Has anyone experienced a similar problem? TO me, it does not make
> sense that the larger the sample, and given similar patterns in
> observed and expected values, we should observe a significantly
> different HL statistic; however that is exactly what I am getting.

> Any comments / suggestions on why could this be happening will be most
> appreciated!!

> Diego Villaveces


The H-L test is largely obsolete, having been replaced by specific directed tests of lack of fit and by the new test described in:

  author = {Hosmer, D. W. and Hosmer, T. and {le Cessie}, S. and Lemeshow, S.},
  year = 1997,
  title = {A comparison of goodness-of-fit tests for the logistic regression
          model},
  journal = Statistics in Medicine,
  volume = 16,
  pages = {965-980},
  annote = {goodness-of-fit for binary logistic model;difficulty with
           Hosmer-Lemeshow statistic being dependent on how groups are
           defined;sum of squares test;cumulative sum test;invalidity of naive
           test based on deviance;goodness-of-link function;simulation setup}

Quote:}

The new one degree of freedom test is implemented in R and S-Plus in the Design library's residuals.lrm function.
---
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. *ia School of Medicine  http://www.veryComputer.com/.*ia.edu/biostat
 
 
 

1. Hosmer-Lemeshow statistic: susceptible to scale?

Sorry to cross post, but I need some help badly. I am currently
evaluating a predictive model that was built using logistic
regression, and using the HL test statistic to do this. I, however,
have found that the larger the sample size that I get, the smaller the
value of the statistic.

Has anyone experienced a similar problem? TO me, it does not make
sense that the larger the sample, and given similar patterns in
observed and expected values, we should observe a significantly
different HL statistic; however that is exactly what I am getting.

Any comments / suggestions on why could this be happening will be most
appreciated!!

Diego Villaveces

2. Pocket Streets

3. hosmer-lemeshow statistic in hierarchical stepwise logistic regression

4. Fetching groups with PPP on 56k

5. Hosmer-Lemeshow

6. Looking for format of .WKS files (Lotus 1-2-3 format)

7. Logistic Regression (Hosmer-Lemeshow)

8. Fight Typographic Terrorism

9. What format is least susceptible to data corruption????

10. comp.periphs.scanners statistics (corrected) Sat, 16 Feb 2002 (last 15 days)

11. comp.periphs.scanners statistics for the last 7 days