Skip to content

How to avoid the three loops in R?

5 messages · Lingyi Ma, PIKAL Petr, ONKELINX, Thierry +2 more

PIKAL Petr
# 
Hi

I do not see any solution without loops but maybe others find it.

I think that you can do it in one loop. Best structure for loop will be list.

In each cycle you will compute matrix with diagonal NA

mat<-matrix(1,nrow=number of items, ncol=number of items)
diag(mat) <- NA

apply(price chunk * mat, 2, mean, na.rm=T)


So when I named your example as temp I get

fac <- interaction(factor(temp$Product), factor(temp$Year_Month), drop=T)
lll <- split(temp, fac)
for( i in 1:length(lll)) {
 mat <- matrix(1,  nrow=nrow(lll[[i]]), ncol= nrow(lll[[i]]))
 diag(mat) <- NA
 lll[[i]]$others <- apply(mat*lll[[i]][,3], 2, mean, na.rm=T)
 }
$`1.201204`
  Country Product Price Year_Month others
1      AE       1    20     201204     24
2      DE       1    20     201204     24
3      CN       1    28     201204     20

$`2.201204`
  Country Product Price Year_Month others
4      AE       2    28     201204     25
5      DE       2    28     201204     25
6      CN       2    22     201204     28

$`3.201204`
  Country Product Price Year_Month others
7      AE       3    28     201204     28
8      CN       3    28     201204     28

$`1.201205`
   Country Product Price Year_Month others
9       AE       1    20     201205     24
10      DE       1    20     201205     24
11      CN       1    28     201205     20

$`2.201205`
   Country Product Price Year_Month others
12      AE       2    28     201205     28
13      DE       2    28     201205     28

I did not check speed but it shall be OK.

Regards
Petr
________________________________
Tento e-mail a jak?koliv k n?mu p?ipojen? dokumenty jsou d?v?rn? a jsou ur?eny pouze jeho adres?t?m.
Jestli?e jste obdr?el(a) tento e-mail omylem, informujte laskav? neprodlen? jeho odes?latele. Obsah tohoto emailu i s p??lohami a jeho kopie vyma?te ze sv?ho syst?mu.
Nejste-li zam??len?m adres?tem tohoto emailu, nejste opr?vn?ni tento email jakkoliv u??vat, roz?i?ovat, kop?rovat ?i zve?ej?ovat.
Odes?latel e-mailu neodpov?d? za eventu?ln? ?kodu zp?sobenou modifikacemi ?i zpo?d?n?m p?enosu e-mailu.

V p??pad?, ?e je tento e-mail sou??st? obchodn?ho jedn?n?:
- vyhrazuje si odes?latel pr?vo ukon?it kdykoliv jedn?n? o uzav?en? smlouvy, a to z jak?hokoliv d?vodu i bez uveden? d?vodu.
- a obsahuje-li nab?dku, je adres?t opr?vn?n nab?dku bezodkladn? p?ijmout; Odes?latel tohoto e-mailu (nab?dky) vylu?uje p?ijet? nab?dky ze strany p??jemce s dodatkem ?i odchylkou.
- trv? odes?latel na tom, ?e p??slu?n? smlouva je uzav?ena teprve v?slovn?m dosa?en?m shody na v?ech jej?ch n?le?itostech.
- odes?latel tohoto emailu informuje, ?e nen? opr?vn?n uzav?rat za spole?nost ??dn? smlouvy s v?jimkou p??pad?, kdy k tomu byl p?semn? zmocn?n nebo p?semn? pov??en a takov? pov??en? nebo pln? moc byly adres?tovi tohoto emailu p??padn? osob?, kterou adres?t zastupuje, p?edlo?eny nebo jejich existence je adres?tovi ?i osob? j?m zastoupen? zn?m?.

This e-mail and any documents attached to it may be confidential and are intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender. Delete the contents of this e-mail with all attachments and its copies from your system.
If you are not the intended recipient of this e-mail, you are not authorized to use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately accept such offer; The sender of this e-mail (offer) excludes any acceptance of the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into any contracts on behalf of the company except for cases in which he/she is expressly authorized to do so in writing, and such authorization or power of attorney is submitted to the recipient or the person represented by the recipient, or the existence of such authorization is known to the recipient of the person represented by the recipient.
ONKELINX, Thierry
# 
It is possible to do without loops if you start by calculating the totals. Then is just aggregating and merging data.

Best regards,

Thierry


set.seed(21)
n.country <- 5
average.price <- runif(n.country, max = 200)
price <- expand.grid(
  Product = 1:10,
  Country = factor(LETTERS[seq_len(n.country)]),
  Year = 2000:2010
)
price$Price <- rnorm(nrow(price), mean = average.price[price$Country], sd = 30)
#number and sum of all prices of the product over all countries and years
total.product <- aggregate(
  cbind(Price, N = 1) ~ Product,
  data = price,
  FUN = sum
)
#number and sum of all prices of the product per country over all years
total.product.country <- aggregate(
  cbind(Pricec = Price, Nc = 1) ~ Product + Country,
  data = price,
  FUN = sum
)
#merge both tables
combined.price <- merge(total.product, total.product.country)
with(combined.price, Price / N) #average price
with(combined.price, Pricec / Nc) #average price per country
with(combined.price, (Price - Pricec)/ (N - Nc)) #average price in the other countries



ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature and Forest
team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
Kliniekstraat 25
1070 Anderlecht
Belgium
+ 32 2 525 02 51
+ 32 54 43 61 85
Thierry.Onkelinx at inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey


-----Oorspronkelijk bericht-----
Van: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] Namens PIKAL Petr
Verzonden: vrijdag 1 augustus 2014 15:18
Aan: Lingyi Ma
CC: r-help at r-project.org
Onderwerp: Re: [R] How to avoid the three loops in R?

Hi

I do not see any solution without loops but maybe others find it.

I think that you can do it in one loop. Best structure for loop will be list.

In each cycle you will compute matrix with diagonal NA

mat<-matrix(1,nrow=number of items, ncol=number of items)
diag(mat) <- NA

apply(price chunk * mat, 2, mean, na.rm=T)


So when I named your example as temp I get

fac <- interaction(factor(temp$Product), factor(temp$Year_Month), drop=T) lll <- split(temp, fac) for( i in 1:length(lll)) {  mat <- matrix(1,  nrow=nrow(lll[[i]]), ncol= nrow(lll[[i]]))
 diag(mat) <- NA
 lll[[i]]$others <- apply(mat*lll[[i]][,3], 2, mean, na.rm=T)  }
$`1.201204`
  Country Product Price Year_Month others
1      AE       1    20     201204     24
2      DE       1    20     201204     24
3      CN       1    28     201204     20

$`2.201204`
  Country Product Price Year_Month others
4      AE       2    28     201204     25
5      DE       2    28     201204     25
6      CN       2    22     201204     28

$`3.201204`
  Country Product Price Year_Month others
7      AE       3    28     201204     28
8      CN       3    28     201204     28

$`1.201205`
   Country Product Price Year_Month others
9       AE       1    20     201205     24
10      DE       1    20     201205     24
11      CN       1    28     201205     20

$`2.201205`
   Country Product Price Year_Month others
12      AE       2    28     201205     28
13      DE       2    28     201205     28

I did not check speed but it shall be OK.

Regards
Petr
________________________________
Tento e-mail a jak?koliv k n?mu p?ipojen? dokumenty jsou d?v?rn? a jsou ur?eny pouze jeho adres?t?m.
Jestli?e jste obdr?el(a) tento e-mail omylem, informujte laskav? neprodlen? jeho odes?latele. Obsah tohoto emailu i s p??lohami a jeho kopie vyma?te ze sv?ho syst?mu.
Nejste-li zam??len?m adres?tem tohoto emailu, nejste opr?vn?ni tento email jakkoliv u??vat, roz?i?ovat, kop?rovat ?i zve?ej?ovat.
Odes?latel e-mailu neodpov?d? za eventu?ln? ?kodu zp?sobenou modifikacemi ?i zpo?d?n?m p?enosu e-mailu.

V p??pad?, ?e je tento e-mail sou??st? obchodn?ho jedn?n?:
- vyhrazuje si odes?latel pr?vo ukon?it kdykoliv jedn?n? o uzav?en? smlouvy, a to z jak?hokoliv d?vodu i bez uveden? d?vodu.
- a obsahuje-li nab?dku, je adres?t opr?vn?n nab?dku bezodkladn? p?ijmout; Odes?latel tohoto e-mailu (nab?dky) vylu?uje p?ijet? nab?dky ze strany p??jemce s dodatkem ?i odchylkou.
- trv? odes?latel na tom, ?e p??slu?n? smlouva je uzav?ena teprve v?slovn?m dosa?en?m shody na v?ech jej?ch n?le?itostech.
- odes?latel tohoto emailu informuje, ?e nen? opr?vn?n uzav?rat za spole?nost ??dn? smlouvy s v?jimkou p??pad?, kdy k tomu byl p?semn? zmocn?n nebo p?semn? pov??en a takov? pov??en? nebo pln? moc byly adres?tovi tohoto emailu p??padn? osob?, kterou adres?t zastupuje, p?edlo?eny nebo jejich existence je adres?tovi ?i osob? j?m zastoupen? zn?m?.

This e-mail and any documents attached to it may be confidential and are intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender. Delete the contents of this e-mail with all attachments and its copies from your system.
If you are not the intended recipient of this e-mail, you are not authorized to use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately accept such offer; The sender of this e-mail (offer) excludes any acceptance of the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into any contracts on behalf of the company except for cases in which he/she is expressly authorized to do so in writing, and such authorization or power of attorney is submitted to the recipient or the person represented by the recipient, or the existence of such authorization is known to the recipient of the person represented by the recipient.
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
* * * * * * * * * * * * * D I S C L A I M E R * * * * * * * * * * * * *
Dit bericht en eventuele bijlagen geven enkel de visie van de schrijver weer en binden het INBO onder geen enkel beding, zolang dit bericht niet bevestigd is door een geldig ondertekend document.
The views expressed in this message and any annex are purely those of the writer and may not be regarded as stating an official position of INBO, as long as the message is not confirmed by a duly signed document.
John McKown
#
On Fri, Aug 1, 2014 at 6:41 AM, Lingyi Ma <lingyi.ma at gmail.com> wrote:
The output above looks wrong. The Price_average_In_Other_area for AE,
product 1 should be 24?

My possible solution:

# Initialize data.frame & call it "x".
Country <- c("AE","DE","CN","AE","DE","CN","AE","CN","AE","DE","CN","AE","DE");
Product <- c(1,1,1,2,2,2,3,3,1,1,1,2,2);
Price <- c(20,20,28,28,28,22,28,28,20,20,28,28,28);
Year_Month <- c(201204,201204,201204,201204,201204,201204,201204,201204,201205,201205,201205,201205,201205);
x <- data.frame(Country,Product,Price,Year_Month,stringsAsFactors=FALSE);
#
#
library("dplyr");
#
# Get the total Price of all Products and number of Products for each
Product & Year_Month"
y <- summarize(group_by(x, Product,
Year_Month),sumPrice=sum(Price),NoPrice=length(Price));
#
# Merge the above data back into the original data.frame, based on
# Product and Year_Month (similar to SQL inner join).
x <- merge(x=x,y=y);
#
# Now calculate the "other area" average by subtracting the cost in this area
# from the total cost in all areas and divide by the number of areas, minus one.
# Please note that if a Product and Year_Month is unique, i.e. no other areas
# for this Product & Year_Month, this will try to divide by zero.
# This gives "Inf" as an answer.
x$Prive_average_In_Other_area <- (x$sumPrice-x$Price)/(x$NoPrice-1);
# Possible alternate to handle above consideration
x$Avg_other <- ifelse(x$NoPrice>1,(x$sumPrice-x$Price)/(x$NoPrice-1),NA);

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      David L Carlson
    

    

      
      #
    

  


  

      
Here's another approach:

# First put the data in a format that is easier to transmit using dput():
dta <- structure(list(Country = structure(c(1L, 3L, 2L, 1L, 3L, 2L, 
1L, 2L, 1L, 3L, 2L, 1L, 3L), .Label = c("AE", "CN", "DE"), class = "factor"), 
    Product = c(1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 1L, 1L, 1L, 2L, 
    2L), Price = c(20L, 20L, 28L, 28L, 28L, 22L, 28L, 28L, 20L, 
    20L, 28L, 28L, 28L), Year_Month = c(201204L, 201204L, 201204L, 
    201204L, 201204L, 201204L, 201204L, 201204L, 201205L, 201205L, 
    201205L, 201205L, 201205L)), .Names = c("Country", "Product", 
"Price", "Year_Month"), class = "data.frame", row.names = c(NA, 
-13L))
 # Then

            

      
      
      
    

        
+      length(x)))

            

      
      
      
    

        
+      Year_Month, Price_average_in_other_area=(Price.y[,1]-Price.x)/
+      (Price.y[,2]-1)))

            

      
      
      
    

        
Country Product Price Year_Month Price_average_in_other_area
1       AE       1    20     201204                          24
2       DE       1    20     201204                          24
3       CN       1    28     201204                          20
4       AE       1    20     201205                          24
5       DE       1    20     201205                          24
6       CN       1    28     201205                          20
7       AE       2    28     201204                          25
8       DE       2    28     201204                          25
9       CN       2    22     201204                          28
10      AE       2    28     201205                          28
11      DE       2    28     201205                          28
12      AE       3    28     201204                          28
13      CN       3    28     201204                          28

-------------------------------------
David L Carlson
Department of Anthropology
Texas A&M University
College Station, TX 77840-4352

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of John McKown
Sent: Friday, August 1, 2014 9:07 AM
To: Lingyi Ma
Cc: r-help
Subject: Re: [R] How to avoid the three loops in R?

        
On Fri, Aug 1, 2014 at 6:41 AM, Lingyi Ma <lingyi.ma at gmail.com> wrote:

            

      
      
      
    

        
The output above looks wrong. The Price_average_In_Other_area for AE,
product 1 should be 24?

My possible solution:

# Initialize data.frame & call it "x".
Country <- c("AE","DE","CN","AE","DE","CN","AE","CN","AE","DE","CN","AE","DE");
Product <- c(1,1,1,2,2,2,3,3,1,1,1,2,2);
Price <- c(20,20,28,28,28,22,28,28,20,20,28,28,28);
Year_Month <- c(201204,201204,201204,201204,201204,201204,201204,201204,201205,201205,201205,201205,201205);
x <- data.frame(Country,Product,Price,Year_Month,stringsAsFactors=FALSE);
#
#
library("dplyr");
#
# Get the total Price of all Products and number of Products for each
Product & Year_Month"
y <- summarize(group_by(x, Product,
Year_Month),sumPrice=sum(Price),NoPrice=length(Price));
#
# Merge the above data back into the original data.frame, based on
# Product and Year_Month (similar to SQL inner join).
x <- merge(x=x,y=y);
#
# Now calculate the "other area" average by subtracting the cost in this area
# from the total cost in all areas and divide by the number of areas, minus one.
# Please note that if a Product and Year_Month is unique, i.e. no other areas
# for this Product & Year_Month, this will try to divide by zero.
# This gives "Inf" as an answer.
x$Prive_average_In_Other_area <- (x$sumPrice-x$Price)/(x$NoPrice-1);
# Possible alternate to handle above consideration
x$Avg_other <- ifelse(x$NoPrice>1,(x$sumPrice-x$Price)/(x$NoPrice-1),NA);