Input encoding problem when using sweave with xetex

Hello

 

Because I want to use different true type fonts with latex, I'm using the
XeTeX typesetting engine for my sweave-documents. I'm using Lyx with Sweave
on a Windows 7 PC  and have set up LyX to work with XeTeX according to the
following instructions:

http://wiki.lyx.org/LyX/XeTeX

 

Because the input file for XeTeX is assumed to be in UTF-8 encoding, I set
the encoding under LyX - Tools - Language Settings - Language to "Unicode
(XeTeX) (utf8)". Accented letters that I write into the LyX-document are
correctly typeset in the final PDF-document. However, character strings with
accented letters that are read from Excel-files or other sources from within
R during the LyX-Sweave document compilation are not. For instance, the
German umlauts of the following example are not correctly typeset, when
"Unicode (XeTeX) (utf8)" is used as input encoding.

 

<<echo=F>>=

require(gdata)

x <- read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls",
stringsAsFactors = F)[2,2]

x

@

 

I do not have this problem with a Mac computer . I guess, this is because R
under Windows does not use UTF-8 encoding.   I tried to change the encoding
within R by doing the following

 

<<echo=F>>=

Encoding(x) <- 'UTF-8'

x

@

 

Unfortunately, this does not work. Does anybody have solution for this
problem?
  

Duncan Murdoch
 

Regards,

Erich

 


	[[alternative HTML version deleted]]

Hello

 

Because I want to use different true type fonts with latex, I'm using the
XeTeX typesetting engine for my sweave-documents. I'm using Lyx with

on a Windows 7 PC  and have set up LyX to work with XeTeX according to the
following instructions:

http://wiki.lyx.org/LyX/XeTeX

 

Because the input file for XeTeX is assumed to be in UTF-8 encoding, I set
the encoding under LyX - Tools - Language Settings - Language to "Unicode
(XeTeX) (utf8)". Accented letters that I write into the LyX-document are
correctly typeset in the final PDF-document. However, character strings

accented letters that are read from Excel-files or other sources from

R during the LyX-Sweave document compilation are not. For instance, the
German umlauts of the following example are not correctly typeset, when
"Unicode (XeTeX) (utf8)" is used as input encoding.

 

<<echo=F>>=

require(gdata)

x <- read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls",
stringsAsFactors = F)[2,2]

x

@

 

I do not have this problem with a Mac computer . I guess, this is because

under Windows does not use UTF-8 encoding.   I tried to change the

within R by doing the following

 

<<echo=F>>=

Encoding(x) <- 'UTF-8'

x

@

 

Unfortunately, this does not work. Does anybody have solution for this
problem?
  

Duncan Murdoch
 

Regards,

Erich

 


	[[alternative HTML version deleted]]

Thanks. Since the encoding of x is unknown (Encoding(x) gives "unknown"), I
tried

iconv(x, "", "UTF-8") 

Unfortunately, accented letters are still not printed in the final PDF
output.
  

Regards,
Erich



-----Urspr?ngliche Nachricht-----
Von: Duncan Murdoch [mailto:murdoch.duncan at gmail.com] 
Gesendet: Mittwoch, 12. Mai 2010 15:27
An: Erich Studerus
Cc: r-help at r-project.org
Betreff: Re: [R] Input encoding problem when using sweave with xetex

On 12/05/2010 8:37 AM, Erich Studerus wrote:
  

Hello

 

Because I want to use different true type fonts with latex, I'm using the
XeTeX typesetting engine for my sweave-documents. I'm using Lyx with
    

Sweave
  

on a Windows 7 PC  and have set up LyX to work with XeTeX according to the
following instructions:

http://wiki.lyx.org/LyX/XeTeX

 

Because the input file for XeTeX is assumed to be in UTF-8 encoding, I set
the encoding under LyX - Tools - Language Settings - Language to "Unicode
(XeTeX) (utf8)". Accented letters that I write into the LyX-document are
correctly typeset in the final PDF-document. However, character strings
    

with
  

accented letters that are read from Excel-files or other sources from
    

within
  

R during the LyX-Sweave document compilation are not. For instance, the
German umlauts of the following example are not correctly typeset, when
"Unicode (XeTeX) (utf8)" is used as input encoding.

 

<<echo=F>>=

require(gdata)

x <- read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls",
stringsAsFactors = F)[2,2]

x

@

 

I do not have this problem with a Mac computer . I guess, this is because
    

R
  

under Windows does not use UTF-8 encoding.   I tried to change the
    

encoding
  

within R by doing the following

 

<<echo=F>>=

Encoding(x) <- 'UTF-8'

x

@

 

Unfortunately, this does not work. Does anybody have solution for this
problem?
  
    

You need to use iconv() to change an encoding.  What you did just 
changes the declared encoding, but doesn't actually change any bits.  So 
you'd probably get what you want with

x <- iconv(x, "", "UTF-8")
x

(though you may need to declare the input encoding; it is likely CP1252 
on Windows).
  

Duncan Murdoch
 

Regards,

Erich

 


	[[alternative HTML version deleted]]

Thanks. Since the encoding of x is unknown (Encoding(x) gives "unknown"),

tried

iconv(x, "", "UTF-8") 

Unfortunately, accented letters are still not printed in the final PDF
output.
  

Regards,
Erich



-----Urspr?ngliche Nachricht-----
Von: Duncan Murdoch [mailto:murdoch.duncan at gmail.com] 
Gesendet: Mittwoch, 12. Mai 2010 15:27
An: Erich Studerus
Cc: r-help at r-project.org
Betreff: Re: [R] Input encoding problem when using sweave with xetex

On 12/05/2010 8:37 AM, Erich Studerus wrote:
  

Hello

 

Because I want to use different true type fonts with latex, I'm using the
XeTeX typesetting engine for my sweave-documents. I'm using Lyx with
    

Sweave
  

on a Windows 7 PC  and have set up LyX to work with XeTeX according to

following instructions:

http://wiki.lyx.org/LyX/XeTeX

 

Because the input file for XeTeX is assumed to be in UTF-8 encoding, I

the encoding under LyX - Tools - Language Settings - Language to "Unicode
(XeTeX) (utf8)". Accented letters that I write into the LyX-document are
correctly typeset in the final PDF-document. However, character strings
    

with
  

accented letters that are read from Excel-files or other sources from
    

within
  

R during the LyX-Sweave document compilation are not. For instance, the
German umlauts of the following example are not correctly typeset, when
"Unicode (XeTeX) (utf8)" is used as input encoding.

 

<<echo=F>>=

require(gdata)

x <- read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls",
stringsAsFactors = F)[2,2]

x

@

 

I do not have this problem with a Mac computer . I guess, this is because
    

R
  

under Windows does not use UTF-8 encoding.   I tried to change the
    

encoding
  

within R by doing the following

 

<<echo=F>>=

Encoding(x) <- 'UTF-8'

x

@

 

Unfortunately, this does not work. Does anybody have solution for this
problem?
  
    

You need to use iconv() to change an encoding.  What you did just 
changes the declared encoding, but doesn't actually change any bits.  So 
you'd probably get what you want with

x <- iconv(x, "", "UTF-8")
x

(though you may need to declare the input encoding; it is likely CP1252 
on Windows).
  

Duncan Murdoch
 

Regards,

Erich

 


	[[alternative HTML version deleted]]

Putting \usepackage[cp1252]{inputenc} into my preamble is not an option,
because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal is also
to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button from
within Lyx. R code chunks are therefore executed by calling R from the
command line. If anybody knows how to run R with options(encoding="UTF-8")
from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this file:
http://cran.r-project.org/contrib/extra/lyx/preferences
  

On 12/05/2010 11:36 AM, Erich Studerus wrote:

Putting \usepackage[cp1252]{inputenc} into my preamble is not an option,
because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal is also
to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button from
within Lyx. R code chunks are therefore executed by calling R from the
command line. If anybody knows how to run R with options(encoding="UTF-8")
from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this file:
http://cran.r-project.org/contrib/extra/lyx/preferences
  

You can do it with a little work.  If you look at the rhome/bin/Sweave.sh 
file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore --slave

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); Sweave(\"file.Rnw\")" 
| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you always 
want Sweave to use UTF-8, or you could edit your Lyx preference file to put 
in a line like this instead of what it had.

Duncan Murdoch

Thank you! I edited the Sweave.sh file and it works now for reading data 
stored as R data files, but the read.xls function from the gdata-package 
does no longer work.

options('encoding'='UTF-8')
require(gdata)
read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls", 
stringsAsFactors = F)[2,2]

This gives NA as output and a warning that there was an invalid entry for 
the connection. Is there a way to use read.xls with UTF-8 encoding?
  

On Wed, 12 May 2010 12:45:51 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

On 12/05/2010 11:36 AM, Erich Studerus wrote:

Putting \usepackage[cp1252]{inputenc} into my preamble is not an option,
because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal is also
to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button from
within Lyx. R code chunks are therefore executed by calling R from the
command line. If anybody knows how to run R with options(encoding="UTF-8")
from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this file:
http://cran.r-project.org/contrib/extra/lyx/preferences
  

You can do it with a little work.  If you look at the rhome/bin/Sweave.sh 
file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore --slave

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); Sweave(\"file.Rnw\")" 
| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you always 
want Sweave to use UTF-8, or you could edit your Lyx preference file to put 
in a line like this instead of what it had.

Duncan Murdoch

On 12/05/2010 2:29 PM, Erich Studerus wrote:

Thank you! I edited the Sweave.sh file and it works now for reading data 
stored as R data files, but the read.xls function from the gdata-package 
does no longer work.

options('encoding'='UTF-8')
require(gdata)
read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls", 
stringsAsFactors = F)[2,2]

This gives NA as output and a warning that there was an invalid entry for 
the connection. Is there a way to use read.xls with UTF-8 encoding?
  

I don't know, you'd have to ask its authors.  But you can probably set 
options(encoding="") before calling it.  (I'm assuming the read.xls() is in 
a code chunk in your Sweave file.  I would guess that you don't need to set 
the encoding back to UTF-8 afterwards, but you might:  it all depends on 
when Sweave() opens its
output file.)

Duncan Murdoch

On Wed, 12 May 2010 12:45:51 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

On 12/05/2010 11:36 AM, Erich Studerus wrote:

Putting \usepackage[cp1252]{inputenc} into my preamble is not an 

option,

because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal is 

also

to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button from
within Lyx. R code chunks are therefore executed by calling R from the
command line. If anybody knows how to run R with 

options(encoding="UTF-8")

from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this file:
http://cran.r-project.org/contrib/extra/lyx/preferences
  

You can do it with a little work.  If you look at the 

rhome/bin/Sweave.sh 

file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore --slave

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); 

Sweave(\"file.Rnw\")" 

| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you always 
want Sweave to use UTF-8, or you could edit your Lyx preference file to 

put 

in a line like this instead of what it had.

Duncan Murdoch

Thanks again. Putting options(encoding="") into the R code chunk before 
calling the read.xls function indeed did the trick. Now, I have one last 
question: How can I edit the LyX preference file to call R with 
options(encoding='UTF-8'). I prefer to edit the LyX preference file rather 
than the Sweave.sh file, because I want to run R with UTF-8 only when I use 
 XeTeX. By editing the LyX preference file, I assume it would be possible to 
define different R encoding setups for different converters.
  

Erich


On Wed, 12 May 2010 14:43:52 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

On 12/05/2010 2:29 PM, Erich Studerus wrote:

Thank you! I edited the Sweave.sh file and it works now for reading data 
stored as R data files, but the read.xls function from the gdata-package 
does no longer work.

options('encoding'='UTF-8')
require(gdata)
read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls", 
stringsAsFactors = F)[2,2]

This gives NA as output and a warning that there was an invalid entry for 
the connection. Is there a way to use read.xls with UTF-8 encoding?
  

I don't know, you'd have to ask its authors.  But you can probably set 
options(encoding="") before calling it.  (I'm assuming the read.xls() is in 
a code chunk in your Sweave file.  I would guess that you don't need to set 
the encoding back to UTF-8 afterwards, but you might:  it all depends on 
when Sweave() opens its
output file.)

Duncan Murdoch

On Wed, 12 May 2010 12:45:51 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

On 12/05/2010 11:36 AM, Erich Studerus wrote:

Putting \usepackage[cp1252]{inputenc} into my preamble is not an 

option,

because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal is 

also

to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button from
within Lyx. R code chunks are therefore executed by calling R from the
command line. If anybody knows how to run R with 

options(encoding="UTF-8")

from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this file:
http://cran.r-project.org/contrib/extra/lyx/preferences
  

You can do it with a little work.  If you look at the 

rhome/bin/Sweave.sh 

file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore --slave

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); 

Sweave(\"file.Rnw\")" 

| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you always 
want Sweave to use UTF-8, or you could edit your Lyx preference file to 

put 

in a line like this instead of what it had.

Duncan Murdoch

On 12/05/2010 3:55 PM, Erich Studerus wrote:

Thanks again. Putting options(encoding="") into the R code chunk before 
calling the read.xls function indeed did the trick. Now, I have one last 
question: How can I edit the LyX preference file to call R with 
options(encoding='UTF-8'). I prefer to edit the LyX preference file rather 
than the Sweave.sh file, because I want to run R with UTF-8 only when I 
use 
 XeTeX. By editing the LyX preference file, I assume it would be possible 
to 
define different R encoding setups for different converters.
  

One way would be to change

R CMD Sweave $$i

in

\converter "literate" "latex"    "R CMD Sweave $$i"  ""

to

echo "library(\"utils\"); options(encoding=\"UTF-8\"); Sweave(\"$$i\")" | 
R --no-restore --slave

but you may run into trouble because there's an extra set of "" to go 
around there, and I don't know how Lyx handles escaping all those quotes. 
So probably an easier way to go would be to take the Sweave.sh file
that you've already edited, and copy it into UTF8Sweave.sh in the same 
directory.  Then you can change the Lyx preference line to

\converter "literate" "latex"    "R CMD UTF8Sweave $$i"  ""

and things should work.  (Then you can restore the original Sweave.sh file 
for cases where you don't
want UTF-8.)

Duncan Murdoch

Erich


On Wed, 12 May 2010 14:43:52 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

On 12/05/2010 2:29 PM, Erich Studerus wrote:

Thank you! I edited the Sweave.sh file and it works now for reading 

data 

stored as R data files, but the read.xls function from the 

gdata-package 

does no longer work.

options('encoding'='UTF-8')
require(gdata)
read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls", 
stringsAsFactors = F)[2,2]

This gives NA as output and a warning that there was an invalid entry 

for 

the connection. Is there a way to use read.xls with UTF-8 encoding?
  

I don't know, you'd have to ask its authors.  But you can probably set 
options(encoding="") before calling it.  (I'm assuming the read.xls() is 

in 

a code chunk in your Sweave file.  I would guess that you don't need to 

set 

the encoding back to UTF-8 afterwards, but you might:  it all depends on 
when Sweave() opens its
output file.)

Duncan Murdoch

On Wed, 12 May 2010 12:45:51 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:

On 12/05/2010 11:36 AM, Erich Studerus wrote:

Putting \usepackage[cp1252]{inputenc} into my preamble is not an 

option,

because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal 

is 

also

to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button 

from

within Lyx. R code chunks are therefore executed by calling R from 

the

command line. If anybody knows how to run R with 

options(encoding="UTF-8")

from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this 

file:

http://cran.r-project.org/contrib/extra/lyx/preferences
  

You can do it with a little work.  If you look at the 

rhome/bin/Sweave.sh 

file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore 

--slave

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); 

Sweave(\"file.Rnw\")" 

| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you 

always 

want Sweave to use UTF-8, or you could edit your Lyx preference file 

to 

put 

in a line like this instead of what it had.

Duncan Murdoch

You were right, that I would run into trouble with all these "". However, 
the second method doesn't work either. Somehow, the UTF8Sweave.sh file is 
not found during the compilation of my document. It says "can't open perl 
script .../bin/UTF8Sweave.sh: No such file or directory". I double checked 
that a file with this name is present in the same folder as the original 
Sweave.sh file and that the "Sweave" is changed to "UTF8Sweave" in the LyX 
preference file. Any idea, what could be wrong?
  

Erich


On Wed, 12 May 2010 16:20:03 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
  

On 12/05/2010 3:55 PM, Erich Studerus wrote:
    

Thanks again. Putting options(encoding="") into the R code chunk before 
calling the read.xls function indeed did the trick. Now, I have one last 
question: How can I edit the LyX preference file to call R with 
options(encoding='UTF-8'). I prefer to edit the LyX preference file rather 
than the Sweave.sh file, because I want to run R with UTF-8 only when I 
use 
 XeTeX. By editing the LyX preference file, I assume it would be possible 
to 
define different R encoding setups for different converters.
  
      

One way would be to change

R CMD Sweave $$i

in

\converter "literate" "latex"    "R CMD Sweave $$i"  ""

to

echo "library(\"utils\"); options(encoding=\"UTF-8\"); Sweave(\"$$i\")" | 
R --no-restore --slave

but you may run into trouble because there's an extra set of "" to go 
around there, and I don't know how Lyx handles escaping all those quotes. 
So probably an easier way to go would be to take the Sweave.sh file
that you've already edited, and copy it into UTF8Sweave.sh in the same 
directory.  Then you can change the Lyx preference line to

\converter "literate" "latex"    "R CMD UTF8Sweave $$i"  ""

and things should work.  (Then you can restore the original Sweave.sh file 
for cases where you don't
want UTF-8.)

Duncan Murdoch

    

Erich


On Wed, 12 May 2010 14:43:52 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
      

On 12/05/2010 2:29 PM, Erich Studerus wrote:
        

Thank you! I edited the Sweave.sh file and it works now for reading 
          

data 
      

stored as R data files, but the read.xls function from the 
          

gdata-package 
      

does no longer work.

options('encoding'='UTF-8')
require(gdata)
read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls", 
stringsAsFactors = F)[2,2]

This gives NA as output and a warning that there was an invalid entry 
          

for 
      

the connection. Is there a way to use read.xls with UTF-8 encoding?
  
          

I don't know, you'd have to ask its authors.  But you can probably set 
options(encoding="") before calling it.  (I'm assuming the read.xls() is 
        

in 
      

a code chunk in your Sweave file.  I would guess that you don't need to 
        

set 
      

the encoding back to UTF-8 afterwards, but you might:  it all depends on 
when Sweave() opens its
output file.)

Duncan Murdoch
        

On Wed, 12 May 2010 12:45:51 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
          

On 12/05/2010 11:36 AM, Erich Studerus wrote:
            

Putting \usepackage[cp1252]{inputenc} into my preamble is not an 
              

option,
          

because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal 
              

is 
      

also
          

to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button 
              

from
      

within Lyx. R code chunks are therefore executed by calling R from 
              

the
      

command line. If anybody knows how to run R with 
              

options(encoding="UTF-8")
          

from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this 
              

file:
      

http://cran.r-project.org/contrib/extra/lyx/preferences
  
              

You can do it with a little work.  If you look at the 
            

rhome/bin/Sweave.sh 
          

file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore 
            

--slave
      

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); 
            

Sweave(\"file.Rnw\")" 
          

| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you 
            

always 
      

want Sweave to use UTF-8, or you could edit your Lyx preference file 
            

to 
      

put 
          

in a line like this instead of what it had.

Duncan Murdoch
            

On 12/05/2010 5:12 PM, Erich Studerus wrote:

You were right, that I would run into trouble with all these "". However, 
the second method doesn't work either. Somehow, the UTF8Sweave.sh file is 
not found during the compilation of my document. It says "can't open perl 
script .../bin/UTF8Sweave.sh: No such file or directory". I double checked 
that a file with this name is present in the same folder as the original 
Sweave.sh file and that the "Sweave" is changed to "UTF8Sweave" in the LyX 
preference file. Any idea, what could be wrong?
  

Sorry, I forgot that if you make up your own command to be run by R CMD 
then you need to give the full path to it, and you can't suppress the .sh 
suffix.  So your command should be

R CMD /path/to/UTF8Sweave.sh $$i


(where /path/to/ is the actual path to where you decide to keep this file, 
which needn't be in rhome/bin).

Duncan Murdoch

Erich


On Wed, 12 May 2010 16:20:03 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
  

On 12/05/2010 3:55 PM, Erich Studerus wrote:
    

Thanks again. Putting options(encoding="") into the R code chunk before 
calling the read.xls function indeed did the trick. Now, I have one last 
question: How can I edit the LyX preference file to call R with 
options(encoding='UTF-8'). I prefer to edit the LyX preference file rather 
than the Sweave.sh file, because I want to run R with UTF-8 only when I 
use 
 XeTeX. By editing the LyX preference file, I assume it would be possible 
to 
define different R encoding setups for different converters.
  
      

One way would be to change

R CMD Sweave $$i

in

\converter "literate" "latex"    "R CMD Sweave $$i"  ""

to

echo "library(\"utils\"); options(encoding=\"UTF-8\"); Sweave(\"$$i\")" | 
R --no-restore --slave

but you may run into trouble because there's an extra set of "" to go 
around there, and I don't know how Lyx handles escaping all those quotes. 
So probably an easier way to go would be to take the Sweave.sh file
that you've already edited, and copy it into UTF8Sweave.sh in the same 
directory.  Then you can change the Lyx preference line to

\converter "literate" "latex"    "R CMD UTF8Sweave $$i"  ""

and things should work.  (Then you can restore the original Sweave.sh file 
for cases where you don't
want UTF-8.)

Duncan Murdoch

    

Erich


On Wed, 12 May 2010 14:43:52 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
      

On 12/05/2010 2:29 PM, Erich Studerus wrote:
        

Thank you! I edited the Sweave.sh file and it works now for reading 
          

data 
      

stored as R data files, but the read.xls function from the 
          

gdata-package 
      

does no longer work.

options('encoding'='UTF-8')
require(gdata)
read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls", 
stringsAsFactors = F)[2,2]

This gives NA as output and a warning that there was an invalid entry 
          

for 
      

the connection. Is there a way to use read.xls with UTF-8 encoding?
  
          

I don't know, you'd have to ask its authors.  But you can probably set 
options(encoding="") before calling it.  (I'm assuming the read.xls() is 
        

in 
      

a code chunk in your Sweave file.  I would guess that you don't need to 
        

set 
      

the encoding back to UTF-8 afterwards, but you might:  it all depends on 
when Sweave() opens its
output file.)

Duncan Murdoch
        

On Wed, 12 May 2010 12:45:51 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
          

On 12/05/2010 11:36 AM, Erich Studerus wrote:
            

Putting \usepackage[cp1252]{inputenc} into my preamble is not an 
              

option,
          

because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal 
              

is 
      

also
          

to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button 
              

from
      

within Lyx. R code chunks are therefore executed by calling R from 
              

the
      

command line. If anybody knows how to run R with 
              

options(encoding="UTF-8")
          

from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this 
              

file:
      

http://cran.r-project.org/contrib/extra/lyx/preferences
  
              

You can do it with a little work.  If you look at the 
            

rhome/bin/Sweave.sh 
          

file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore 
            

--slave
      

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); 
            

Sweave(\"file.Rnw\")" 
          

| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you 
            

always 
      

want Sweave to use UTF-8, or you could edit your Lyx preference file 
            

to 
      

put 
          

in a line like this instead of what it had.

Duncan Murdoch
            

Thanks again. It finally worked after copying the Sweave.sh file to a path 
that contains no spaces. Is there a special command for windows specific 
path names with spaces?
  

Erich


On Wed, 12 May 2010 18:04:44 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
  

On 12/05/2010 5:12 PM, Erich Studerus wrote:
    

You were right, that I would run into trouble with all these "". However, 
the second method doesn't work either. Somehow, the UTF8Sweave.sh file is 
not found during the compilation of my document. It says "can't open perl 
script .../bin/UTF8Sweave.sh: No such file or directory". I double checked 
that a file with this name is present in the same folder as the original 
Sweave.sh file and that the "Sweave" is changed to "UTF8Sweave" in the LyX 
preference file. Any idea, what could be wrong?
  
      

Sorry, I forgot that if you make up your own command to be run by R CMD 
then you need to give the full path to it, and you can't suppress the .sh 
suffix.  So your command should be

R CMD /path/to/UTF8Sweave.sh $$i


(where /path/to/ is the actual path to where you decide to keep this file, 
which needn't be in rhome/bin).

Duncan Murdoch


    

Erich


On Wed, 12 May 2010 16:20:03 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
  
      

On 12/05/2010 3:55 PM, Erich Studerus wrote:
    
        

Thanks again. Putting options(encoding="") into the R code chunk before 
calling the read.xls function indeed did the trick. Now, I have one last 
question: How can I edit the LyX preference file to call R with 
options(encoding='UTF-8'). I prefer to edit the LyX preference file rather 
than the Sweave.sh file, because I want to run R with UTF-8 only when I 
use 
 XeTeX. By editing the LyX preference file, I assume it would be possible 
to 
define different R encoding setups for different converters.
  
      
          

One way would be to change

R CMD Sweave $$i

in

\converter "literate" "latex"    "R CMD Sweave $$i"  ""

to

echo "library(\"utils\"); options(encoding=\"UTF-8\"); Sweave(\"$$i\")" | 
R --no-restore --slave

but you may run into trouble because there's an extra set of "" to go 
around there, and I don't know how Lyx handles escaping all those quotes. 
So probably an easier way to go would be to take the Sweave.sh file
that you've already edited, and copy it into UTF8Sweave.sh in the same 
directory.  Then you can change the Lyx preference line to

\converter "literate" "latex"    "R CMD UTF8Sweave $$i"  ""

and things should work.  (Then you can restore the original Sweave.sh file 
for cases where you don't
want UTF-8.)

Duncan Murdoch

    
        

Erich


On Wed, 12 May 2010 14:43:52 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
      
          

On 12/05/2010 2:29 PM, Erich Studerus wrote:
        
            

Thank you! I edited the Sweave.sh file and it works now for reading 
          
              

data 
      
          

stored as R data files, but the read.xls function from the 
          
              

gdata-package 
      
          

does no longer work.

options('encoding'='UTF-8')
require(gdata)
read.xls("http://www.schwerhoerigkeit.pop.ch/hoergeraete_test.xls", 
stringsAsFactors = F)[2,2]

This gives NA as output and a warning that there was an invalid entry 
          
              

for 
      
          

the connection. Is there a way to use read.xls with UTF-8 encoding?
  
          
              

I don't know, you'd have to ask its authors.  But you can probably set 
options(encoding="") before calling it.  (I'm assuming the read.xls() is 
        
            

in 
      
          

a code chunk in your Sweave file.  I would guess that you don't need to 
        
            

set 
      
          

the encoding back to UTF-8 afterwards, but you might:  it all depends on 
when Sweave() opens its
output file.)

Duncan Murdoch
        
            

On Wed, 12 May 2010 12:45:51 -0400
  Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
          
              

On 12/05/2010 11:36 AM, Erich Studerus wrote:
            
                

Putting \usepackage[cp1252]{inputenc} into my preamble is not an 
              
                  

option,
          
              

because XeTeX unlike LaTeX needs UTF-8 has input encoding. My goal 
              
                  

is 
      
          

also
          
              

to have a LyX document that can be compiled both on Mac and Windows.

I usually compile my Lyx-Sweave documents by one click of a button 
              
                  

from
      
          

within Lyx. R code chunks are therefore executed by calling R from 
              
                  

the
      
          

command line. If anybody knows how to run R with 
              
                  

options(encoding="UTF-8")
          
              

from the command line under windows, that would be helpful.

The command that calls R during compilation is contained in this 
              
                  

file:
      
          

http://cran.r-project.org/contrib/extra/lyx/preferences
  
              
                  

You can do it with a little work.  If you look at the 
            
                

rhome/bin/Sweave.sh 
          
              

file, you'll see that

R CMD Sweave file.Rnw

just executes something like

echo "library(\"utils\"); Sweave(\"file.Rnw\")" | R --no-restore 
            
                

--slave
      
          

What you want is to execute

echo "library(\"utils\"); options(encoding="UTF-8"); 
            
                

Sweave(\"file.Rnw\")" 
          
              

| R --no-restore --slave

You could edit the rhome/bin/Sweave.sh file appropriately if you 
            
                

always 
      
          

want Sweave to use UTF-8, or you could edit your Lyx preference file 
            
                

to 
      
          

put 
          
              

in a line like this instead of what it had.

Duncan Murdoch