## Hosmer-Lemeshow statistic: susceptible to scale?

### Hosmer-Lemeshow statistic: susceptible to scale?

Sorry to cross post, but I need some help badly. I am currently
evaluating a predictive model that was built using logistic
regression, and using the HL test statistic to do this. I, however,
have found that the larger the sample size that I get, the smaller the
value of the statistic.

Has anyone experienced a similar problem? TO me, it does not make
sense that the larger the sample, and given similar patterns in
observed and expected values, we should observe a significantly
different HL statistic; however that is exactly what I am getting.

Any comments / suggestions on why could this be happening will be most
appreciated!!

Diego Villaveces

### Hosmer-Lemeshow statistic: susceptible to scale?

Quote:> Sorry to cross post, but I need some help badly. I am currently
> evaluating a predictive model that was built using logistic
> regression, and using the HL test statistic to do this. I, however,
> have found that the larger the sample size that I get, the smaller the
> value of the statistic.

> Has anyone experienced a similar problem? TO me, it does not make
> sense that the larger the sample, and given similar patterns in
> observed and expected values, we should observe a significantly
> different HL statistic; however that is exactly what I am getting.

Is that the HL  statistic for goodness of fit?
- the one that compares the fit of the prediction to
the observed probabilities, by decile-group?
- the one that warns of over-fitting?
- the one that says, If this is BIG, you have a PROBLEM despite
what the significance test of regression (elsewhere) says?

Yes, I would hope that  if it starts out BIG, I should be so lucky
that it might be merely because the sample is too small;
and that it should get better (smaller) when I add cases.

If that is not the test you refer to, then I don't know.

--

http://www.pitt.edu/~wpilib/index.html

### Hosmer-Lemeshow statistic: susceptible to scale?

On 9 Jun 2003 22:18:02 -0700

> Sorry to cross post, but I need some help badly. I am currently
> evaluating a predictive model that was built using logistic
> regression, and using the HL test statistic to do this. I, however,
> have found that the larger the sample size that I get, the smaller the
> value of the statistic.

> Has anyone experienced a similar problem? TO me, it does not make
> sense that the larger the sample, and given similar patterns in
> observed and expected values, we should observe a significantly
> different HL statistic; however that is exactly what I am getting.

> Any comments / suggestions on why could this be happening will be most
> appreciated!!

> Diego Villaveces

The H-L test is largely obsolete, having been replaced by specific directed tests of lack of fit and by the new test described in:

author = {Hosmer, D. W. and Hosmer, T. and {le Cessie}, S. and Lemeshow, S.},
year = 1997,
title = {A comparison of goodness-of-fit tests for the logistic regression
model},
journal = Statistics in Medicine,
volume = 16,
pages = {965-980},
annote = {goodness-of-fit for binary logistic model;difficulty with
Hosmer-Lemeshow statistic being dependent on how groups are
defined;sum of squares test;cumulative sum test;invalidity of naive
test based on deviance;goodness-of-link function;simulation setup}

Quote:}

The new one degree of freedom test is implemented in R and S-Plus in the Design library's residuals.lrm function.
---
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. *ia School of Medicine  http://www.veryComputer.com/.*ia.edu/biostat

Sorry to cross post, but I need some help badly. I am currently
evaluating a predictive model that was built using logistic
regression, and using the HL test statistic to do this. I, however,
have found that the larger the sample size that I get, the smaller the
value of the statistic.

Has anyone experienced a similar problem? TO me, it does not make
sense that the larger the sample, and given similar patterns in
observed and expected values, we should observe a significantly
different HL statistic; however that is exactly what I am getting.

Any comments / suggestions on why could this be happening will be most
appreciated!!

Diego Villaveces