Skip to content

nls: different results if applied to normal or linearized data

9 messages · Wolfgang Waser, Gabor Grothendieck, Martin Elff +2 more

Wolfgang Waser
# 
Dear all,

I did a non-linear least square model fit

y ~ a * x^b 

(a) > nls(y ~ a * x^b, start=list(a=1,b=1))

to obtain the coefficients a & b.

I did the same with the linearized formula, including a linear model

log(y) ~ log(a) + b * log(x)

(b) > nls(log10(y) ~ log10(a) + b*log10(x), start=list(a=1,b=1))
(c) > lm(log10(y) ~ log10(x))

I expected coefficient b to be identical for all three cases. Hoever, using my 
dataset, coefficient b was:
(a) 0.912
(b) 0.9794
(c) 0.9794

Coefficient a also varied between option (a) and (b), 107.2 and 94.7, 
respectively.

Is this supposed to happen? Which is the correct coefficient b?


Regards,

Wolfgang
Gabor Grothendieck
# 
Write out the objective functions that they are minimizing and it
will be clear they are different so you can't expect the same
results.
On Wed, Mar 5, 2008 at 8:53 AM, Wolfgang Waser <wolfgang.waser at utu.fi> wrote:
Martin Elff
#
On Wednesday 05 March 2008 (14:53:27), Wolfgang Waser wrote:
Models (a) and (b) entail different distributions of the dependent variable y
and different ranges of values that y may take.
(a) implies that y has, conditionally on x, a normal distribution and
has a range of feasible values from -Inf to +Inf.
(b) and (c) imply that log(y) has a normal distribution, that is, 
y has a log-normal distribution and can take values from zero to +Inf.
Given the above considerations, different results with respect to the 
intercept are definitely to be expected.
That depends - is y strictly non-negative or not ...

Just my 20 cents...
Brian Ripley
#
On Wed, 5 Mar 2008, Wolfgang Waser wrote:

            

      
      
      
    

        
Yes.  You are fitting by least-squares on two different scales: 
differences in y and differences in log(y) are not comparable.

Both are correct solutions to different problems.  Since we have no idea 
what 'x' and 'y' are, we cannot even guess which is more appropriate in 
your context.

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Rolf Turner
    

    

      
      #
    

  


  

      
On 6/03/2008, at 2:53 AM, Wolfgang Waser wrote:

        

            

      
      
      
    

        
The two approaches assume two different models.

	Model (1) is y = a*x^b + E (where different errors are independent  
and identically
	--- usually normally --- distributed).

	Model (2) is y = a*(x^b)*E (and you are usually tacitly assuming  
that ln E is normally distributed).

	The point estimates of a and b will consequently be different ---  
although usually not hugely
	different.  Their distributional properties will be substantially  
different.

		cheers,

			Rolf Turner

######################################################################
Attention:\ This e-mail message is privileged and confid...{{dropped:9}}

  
  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
The only thing you are adding to earlier replies is incorrect:

 	fitting by least squares does not imply a normal distribution.

For a regression model, least-squares is in various senses optimal when 
the errors are i.i.d. and normal, but it is a reasonable procedure for 
many other situations (but not for modestly long-tailed distributions, 
the point of robust statistics).

Although values from -Inf to +Inf are theoretically possible for a normal, 
it has very little mass in the tails and is often used as a model for 
non-negative quantities (and e.g. the justification of Box-Cox estimation 
relies on this).

        
On Wed, 5 Mar 2008, Martin Elff wrote:

        

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Wolfgang Waser
    

    

      
      #
    

  


  

      
Thanks for your comments!

            

      
      
      
    

        
I'm fitting metabolic rate data from small fish (oxygen consumption in 
nmol/min vs. body weight in g).
The b coefficient is the interesting part and is generally somewhere around 
0.75.
The one calculated for my data using option (a) is therefore 'better' than 
(b,c), but which one is the correct to use? Log-transformation of metabolic 
rate data is (was) normally performed to be able to determine a and b by 
simple linear regression (or even on paper).

            

      
      
      
    

        
So in view of my context (metabolic rate data) would Model (1) be the 
appropriate model to use?

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Brian Ripley
    

    

      
      #
    

  


  

      
On Thu, 6 Mar 2008, Wolfgang Waser wrote:

        

            

      
      
      
    

        
Unlikely for a rate: those are normally viewed as being on log scale (we 
saya a rate is doubled, for example).  But a residual analysis will show 
if there are departures from assumptions in one or other model.

Usual advice: seek local statistical help, for these are conceptual and 
not R issues.

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Martin Elff
    

    

      
      #
    

  


  

      
On Thursday 06 March 2008 (07:03:34), Prof Brian Ripley wrote:

            

      
      
      
    

        
Thanks for the clarification, 'implies' is to strong. I should have 
written 'suggests' or 'is often motivated by'.

When I wrote my post, there was only one earlier reply, which mentioned
the objective function.  The point I (rather clumsily) tried to make is: (a) 
and (b)+(c) differ in so far that under (a) y may have zero or negative 
values, while under (b) and (c) y may only have values above zero. 
So the models do not just differ in the objective function but also
in their substantive interpretation, which may help deciding which
is the 'correct' coefficient b.