Skip to content

[RsR] error/warning messages with lmrob()

4 messages · Werner Stahel, Martin Maechler, Emily Klein

Original
Werner Stahel
# 
Dear Emily


A short answer also to your questions about non-convergence of lmrob.

You are using it to characterize the trend in many time series.

I assume that the problems are caused by the series being short.

I would be interested in getting some of those that cause problems.


For a simple regression, you may fall back on good old  rlm(...)

In the meantime, I will again try to convince the maintainers of  lmrob

to introduce the possibility of providing  initial  values that allow for

avoiding the initial S step -- since this step causes the problems.


Best regards. Werner


Werner Stahel
M +41 79 784 9330 | P +41 44 364 6424
1 day later
Martin Maechler
# 
> Dear Emily

    >  A short answer also to your questions about
    > non-convergence of lmrob.  You are using it to
    > characterize the trend in many time series.  I assume that
    > the problems are caused by the series being short.  I
    > would be interested in getting some of those that cause problems.

Indeed, me too. (we mean getting the datasets, e.g., sending us
 	the file  'lmrob-problm.rds'  after
   
   saveRDS(<your-data-frame>, file = "lmrob-problm.rds")

)

    > For a simple regression, you may fall back on good old
    > rlm(...)  In the meantime, I will again try to convince
    > the maintainers of lmrob to introduce the possibility of
    > providing initial values that allow for avoiding the
    > initial S step -- since this step causes the problems.

Well, 'init' has been an argument of lmrob() "forever", and has
quite a nice paragraph on the help page of lmrob (search for
"initial estimator") *and* another paragraph in the 'Examples'
of the help page, online versions of which are nowadays easily found by googling 
'lmrob robustbase'), notably these two:
- https://www.rdocumentation.org/packages/robustbase/versions/0.93-5/topics/lmrob
- https://rdrr.io/cran/robustbase/man/lmrob.html

[I really would not go back to use rlm().]

Martin


    > Best regards. Werner

    > Werner Stahel M +41 79 784 9330 | P +41 44 364 6424
    > ________________________________
    >> Von: R-SIG-Robust <r-sig-robust-bounces at r-project.org> im
    >> Auftrag von Emily Klein <emily.klein04 at gmail.com>
    >> Gesendet: Montag, 15. Juli 2019 21:45:00 An:
    >> r-sig-robust at r-project.org Betreff: [RsR] error/warning
    >> messages with lmrob()
    >> 
    >> Dear all,
    >> 
    >> I am running lmrob() under the robustbase package, and I
    >> have a few error/warning messages I would greatly
    >> appreciate help with - I am unfamiliar with robust linear
    >> models, so my apologies in advance if the answers here
    >> are obvious. In addition, any discussion will help me get
    >> a handle on what we're doing, which always helps.
    >> 
    >> (1) I get the repeated error seemingly on random species
    >> when running a loop of lmrob() over all our ecological
    >> time series (different ones almost every time):
    >> 
    >> Error in eigen(ret, symmetric = TRUE) : infinite or
    >> missing values in 'x'
    >> 
    >> I have checked my data many times, and if I just
    >> repeatedly run lmrob() it eventually works. Any idea what
    >> this error is in reference to? Could it be because I am
    >> using a previous version of R (3.4.1)?
    >> 
    >> (2) I have several two error messages that come up with
    >> some time series; I have some idea their meaning, but
    >> again, as a newbie with RLMS, I am wondering if those
    >> time series that garner these warnings should be excluded
    >> from our meta-analysis. Also, any additional insight on
    >> the process or something I might be missing greatly
    >> welcomed!
    >> 
    >> Warning in lmrob.S(x, y, control = control) :
    >> find_scale() did not converge in 'maxit.scale' (= 200)
    >> iterations with tol=1e-10, last rel.diff=0
    >> 
    >> Warning in lmrob.fit(x, y, control, init = init) : M-step
    >> did NOT converge. Returning unconverged SM-estimate
    >> 
    >> Warning in lmrob.S(x, y, control = control) : S-estimated
    >> scale == 0: Probably exact fit; check your data
    >> 
    >> Thank you very much in advance,
    >> 
    >> Emily
    >> 
    >> 
    >> --
    >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    >> Emily S. Klein, Senior Postdoctoral Associate (she / her
    >> / hers) The Frederick S. Pardee Center for the Study of
    >> the Longer-Range Future | Boston University *Co-Chair*,
    >> ICES Working Group on the History of Fish & Fisheries
    >> (WGHIST) esklein04 at gmail.com
    >> 
    >> http://www.bu.edu/pardee/
    >> http://www.ices.dk/community/groups/Pages/WGHIST.aspx
Werner Stahel
# 
Sorry, Martin and Emily, for this stupid mistake.

Just to make things very concrete, you might use


lmrob( y~year, method="M", init=lmrob.lar, psi="lqq")


Please let us know if there are still warnings with this version.


Best regards. Werner


Werner Stahel
M +41 79 784 9330 | P +41 44 364 6424
2 days later
Emily Klein
# 
Hi again both -

My main question is that the lmrob() function quits seemingly randomly
sometimes, but if I just keep calling it, eventually it works fine
(although yes some time series report back errors in either case). I have
run it on a new computer with the most recent version of R and had the same
thing happen.

In any case - attached is the .rds file of the database I am running for
this. I have a function that runs lmrob() on time series of each species x
management type x location (i.e. via the "gind" variable - or "group
identifier"), which is then the subset as "DBx" from the original set "DB".
The call for running lmrob() then is:

lm_test<-tryCatch(lmrob(log(pop_status+1)~observation_year,DBx),
finlly=print(i))

I use "tryCatch" to print out where I have warnings so I can go back and
look at these species, to determine if we should include them or not given
the warnings.

I did also try your suggestion, Werner, but this did not return results for
a species, gind="perdix perdix_in_rec", and once that happened, the next
steps in the function to gather certain aspects of the results returned an
error message (as there weren't any results) and stopped the function via

 Error in dimnames(ans$coefficients) <- list(names(est), cf.nms) :
  length of 'dimnames' [1] not equal to array extent

because there are no coefficients to reference, i.e. the issue is calling
summary(lm_test)$coefficients. In any case, this species, gray partridge in
Indiana under recreational hunt management, has only three observations
over 15 years, and although they are clearly declining over that period -
but it is not the only species with only 3 observations in the time series.

In any case, I subset out this one time series and ran it again - results
are the same as without it, but it does run to the end without the error.
Any ideas why this one species is knocking this out?

Attached is the .rds file, as well as the script I am using, and the
terrestrial species .csv data file.

Thank you again!
Emily


On Mon, Aug 26, 2019 at 4:40 PM Stahel Werner A. <stahel at stat.math.ethz.ch>
wrote: