factor analysis of dynamic structure (FADS) for a huge time-series data

This not being a question about R, but rather about statistics, or possibly about a contributed package, means (per the Posting Guide) that you should be asking in a statistics forum like stats.stackexchange.com or corresponding with the author of the package in question. If you are lucky someone here will have something to offer, but it is not very likely.

Dear R users,

I want to find the latent factors from a kind of time-series data
describing temporal changes of concentration using a factor analysis
technique called 'factor analysis of dynamic structure (FADS).' I
learned
how to form the data for the analysis using a proper package embedding
FADS, such as 'fad' package.

The analysis with 'fad' worked and gave me results, but the problem was
raised when the time-series data is vast.

The time-series data extracted from the 3-dimensional matrix (i.e., 3D
image volume of 50 x 50 x 163) repeatedly acquired at 54-time points is
consisted of 50 x 50 x 163 x 54 = 22,005,000 observations. The desired
number of the latent factor (k) is 4. What I got from fad(MATRIX, k) is
following:

Error in fun(A, k, nu, nv, opts, mattype = "matrix") :
 TridiagEigen: eigen decomposition failed

When I resize the matrix smaller into 5 x 5 x 15, it gives me what I
wanted
properly.

I found that some resampling methods such as random sampling, data
stratification, etc., could resolve this kind of problem, but I have no
ideas which one could be appropriate.

Please teach me with any ideas and comments.

Thanks in advance,

Park

Sent from my phone. Please excuse my brevity.

Dear Newmiller,

Thank  you for your reply. I?ve just posted the same question in the
another forum for stats as you suggested.

Meanwhile, I would like to keep the question submitted to learn from R
users, if it is available .

Park

On Sun, May 9, 2021 at 6:01 AM Jeff Newmiller <jdnewmil at dcn.davis.ca.us>
wrote:

This not being a question about R, but rather about statistics, or
possibly about a contributed package, means (per the Posting Guide) that
you should be asking in a statistics forum like stats.stackexchange.com
or corresponding with the author of the package in question. If you are
lucky someone here will have something to offer, but it is not very likely.

On May 8, 2021 3:05:12 AM PDT, Hyun Soo Park <hyuns at snu.ac.kr> wrote:

Dear R users,

I want to find the latent factors from a kind of time-series data
describing temporal changes of concentration using a factor analysis
technique called 'factor analysis of dynamic structure (FADS).' I
learned
how to form the data for the analysis using a proper package embedding
FADS, such as 'fad' package.

The analysis with 'fad' worked and gave me results, but the problem was
raised when the time-series data is vast.

The time-series data extracted from the 3-dimensional matrix (i.e., 3D
image volume of 50 x 50 x 163) repeatedly acquired at 54-time points is
consisted of 50 x 50 x 163 x 54 = 22,005,000 observations. The desired
number of the latent factor (k) is 4. What I got from fad(MATRIX, k) is
following:

Error in fun(A, k, nu, nv, opts, mattype = "matrix") :
 TridiagEigen: eigen decomposition failed

When I resize the matrix smaller into 5 x 5 x 15, it gives me what I
wanted
properly.

I found that some resampling methods such as random sampling, data
stratification, etc., could resolve this kind of problem, but I have no
ideas which one could be appropriate.

Please teach me with any ideas and comments.

Thanks in advance,

Park

--
Sent from my phone. Please excuse my brevity.

*??????, ?????????*
*????:*
(???) +82-31-787-2936
(????) +82-10-8833-2806
*??:* +82-31-787-4018
*???:* hyuns at snu.ac.kr

*Hyun Soo Park, PhD*
*--*
*Research professor*
Department of Nuclear Medicine
Seoul National University Bundang Hospital, Seongnam, Korea
*Telephone:*
(Office) +82-31-787-2936
(Mobile) +82-10-8833-2806
*Fax:* +82-31-787-4018
*email:* hyuns at snu.ac.kr

	[[alternative HTML version deleted]]

I am an author of the paper behind the fad package. I suspect that the call is not correct. Actually, fad does not quite account for time series or other structured data and you have to enter it, as in all general EFA packages as a n x p matrix, with n the number of observations and p the number of coordinates.
 
So, if you can provide a reproducible example, I can look into it, or you can also file an issue on the github site.
 
One thing to note that EFA requires all variances in the dispersion matrix to be positive, and it is possible that your images have some background where there is no activity and hence the sd for those pixel/voxels are zero. 

Of course, ideally, your EFA should account for the image structure, but that is a different topic and not part of fad or any similar package.

Ranjan

PS: I monitor this e-mail address only through this list.
 

?
?
?

Sent:?Saturday, May 08, 2021 at 5:05 AM
From:?"Hyun Soo Park" <hyuns at snu.ac.kr>
To:?"r-help at r-project.org" <r-help at r-project.org>
Subject:?[R] factor analysis of dynamic structure (FADS) for a huge time-series data
Dear R users,

I want to find the latent factors from a kind of time-series data
describing temporal changes of concentration using a factor analysis
technique called 'factor analysis of dynamic structure (FADS).' I learned
how to form the data for the analysis using a proper package embedding
FADS, such as 'fad' package.

The analysis with 'fad' worked and gave me results, but the problem was
raised when the time-series data is vast.

The time-series data extracted from the 3-dimensional matrix (i.e., 3D
image volume of 50 x 50 x 163) repeatedly acquired at 54-time points is
consisted of 50 x 50 x 163 x 54 = 22,005,000 observations. The desired
number of the latent factor (k) is 4. What I got from fad(MATRIX, k) is
following:

Error in fun(A, k, nu, nv, opts, mattype = "matrix") :
TridiagEigen: eigen decomposition failed

When I resize the matrix smaller into 5 x 5 x 15, it gives me what I wanted
properly.

I found that some resampling methods such as random sampling, data
stratification, etc., could resolve this kind of problem, but I have no
ideas which one could be appropriate.

Please teach me with any ideas and comments.

Thanks in advance,

Park

--
*??????, ?????????*
*????:*
(???) +82-31-787-2936
(????) +82-10-8833-2806
*??:* +82-31-787-4018
*???:* hyuns at snu.ac.kr

*Hyun Soo Park, PhD*
*--*
*Research professor*
Department of Nuclear Medicine
Seoul National University Bundang Hospital, Seongnam, Korea
*Telephone:*
(Office) +82-31-787-2936
(Mobile) +82-10-8833-2806
*Fax:* +82-31-787-4018
*email:* hyuns at snu.ac.kr


______________________________________________
R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html[http://www.R-project.org/posting-guide.html]
and provide commented, minimal, self-contained, reproducible code.