Skip to content

problems with iconv

12 messages · RogerP, Brian Ripley, Uwe Ligges +3 more

RogerP
# 
I'm at wit's ends here and need some help.

I've downloaded and compiled iconv versions 1.13 and 1.14  -
libiconv.so.2.5.0 and libiconv.so.2.5.1 and copied the iconv all over,
replacing the native iconv on my Solaris machine.

Still when I try to run the configure I get:

checking iconv.h usability... yes
checking iconv.h presence... yes
checking for iconv.h... yes
checking for iconv... yes
checking whether iconv accepts "UTF-8", "latin1", "ASCII" and "UCS-*"...
ac_fn_c_try_run  29941
no
configure: error: a suitable iconv is essential

If I cheat and rig the configure file so that it continues even with the
error, I get a nice version of R that runs several of the demos, but should
I try to update the packages I get:

* installing *source* package 'foreign' ...
** package 'foreign' successfully unpacked and MD5 sums checked
Error in iconv(x, "latin1", "ASCII") :
  unsupported conversion from 'latin1' to 'ASCII'

So, what is wrong with iconv 1.14?  It should be latest and greatest.  What
other iconv should I have used?  Where should I have put iconv?

Thanks very much in advance and I promise to share the binaries with who
ever wants them,
roger

--
View this message in context: http://r.789695.n4.nabble.com/problems-with-iconv-tp4191177p4191177.html
Sent from the R devel mailing list archive at Nabble.com.
Brian Ripley
#
On 13/12/2011 17:12, RogerP wrote:
Like reading the manual?

You need to ensure that GNU libiconv is actually used: you are obviously 
not finding it, and I suspect your error is in not setting the path to 
its header file.

There are explicit instructions for libiconv on Solaris in the manual 
which the INSTALL file asked you to read before trying to install R and 
to resolve any problems.

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      RogerP
    

    

      
      #
    

  


  

      
As I said in my email: help I appreciate - sarcasm not so much.

I re-read the manual - just in case I'd missed something and still do not
have any idea.  The manual, BTW, needs some serious help for it to be useful
to its intended audiance - people who don't already know how to install R. 
For example, too many times example are given as to what changes to make,
but oh, there is no mention of where to make the changes.  It is assumed
that the reader knows where to make the changes.

Anyway, configure states that if found the iconv header - iconv.h, which i
put in the includes directory.  It says it finds iconv - though it does not
say where.  

Anyone with some useful help?

--
View this message in context: http://r.789695.n4.nabble.com/problems-with-iconv-tp4191177p4191892.html
Sent from the R devel mailing list archive at Nabble.com.

  
  


  


      

    

    

      
      

        


  

        

      Uwe Ligges
    

    

      
      #
    

  


  

      
Please cite the original messages!

        
On 13.12.2011 21:32, RogerP wrote:

            

      
      
      
    

        
Where are the patches to improve that?

            

      
      
      
    

        
You altready got useful help but you are not accepting it.

Please read the manual and follow the advice given there (for R-release 
in Section C.5), as Brian asked you already. If something is not clear 
enough, tell us what you do not understand.
Saying that it does not work if you do not follow the advice is not 
helpful at all for people who try to help!

Uwe Ligges

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Kasper Daniel Hansen
    

    

      
      #
    

  


  

      
Roger,

Since Ripley is usually right, if I was you, I would focus on

"You need to ensure that GNU libiconv is actually used: you are
obviously not finding it, and I suspect your error is in not setting
the path to its header file."

Based on your description

"I've downloaded and compiled iconv versions 1.13 and 1.14  -
libiconv.so.2.5.0 and libiconv.so.2.5.1 and copied the iconv all over,
replacing the native iconv on my Solaris machine."

I would guess you have not copied over the header files.  Although
that is really hard to guess since you don't provide any details.  It
certainly sounds like you have done some hacking.

I agree that it can sometimes be bewildering to figure out which
library gets picked up from the output ... but the way configure works
on R is the same as for almost all other unix tools.  And you can see
the compiler command on the command line.

Kasper

2011/12/13 Uwe Ligges <ligges at statistik.tu-dortmund.de>:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      RogerP
    

    

      
      #
    

  


  

      
Sorry, but IMHO saying "read the manual" does not constitute actual help.

But here it is from the manual:

A suitably comprehensive iconv function is essential. The R usage requires
iconv to be able to translate between "latin1" and "UTF-8", to recognize ""
(as the current encoding) and "ASCII", and to translate to and from the
Unicode wide-character formats "UCS-[24][BL]E" ? this is true for glibc but
not of most commercial Unixes. However, you can make use of GNU libiconv
(possibly as a plug-in replacement: see
http://www.gnu.org/software/libiconv/). 

Well, that's just what I did.  I downloaded libiconv and compiled and linked
it.  Oh, so where to put it.  Hmmmm, don't see it anywhere here.  Do you?  

Also from the manual:

/You will need GNU libiconv and readline: the Solaris version of iconv is
not sufficiently powerful. 

For the Solaris Studio compilers a little juggling of paths was needed to
ensure GNU libiconv (in /usr/local) was used rather than the Solaris iconv:

     CC="cc -xc99"
     CFLAGS="-O -xlibmieee"
     F77=f95
     FFLAGS=-O4
     CXX="CC -library=stlport4"
     CXXFLAGS=-O
     FC=f95
     FCFLAGS=$FFLAGS
     FCLIBS="-lfai -lfsu"
     R_LD_LIBRARY_PATH="/usr/local/lib:/opt/csw/gcc4/lib:/opt/csw/lib"

For a 64-bit target add -m64 to the compiler macros and use something like
LDFLAGS=-L/usr/local/lib/sparcv9 or LDFLAGS=-L/usr/local/lib/amd64 as
appropriate. /

Well, I did that. Here are some of my options from the config.site:

CC="cc"

## Debugging and optimization options for the C compiler.
## Use this to specify CFLAGS for the version of the C compiler
## specified above.
## If unspecified, defaults to '-g -O2' for gcc, 
## and '-g' in all other cases except icc (for which see R-admin.html).
#CFLAGS="-xlibmieee"
CFLAGS="-xO5 -xc99 -xlibmieee -xlibmil -nofstore -xtarget=native -m64"

FFLAGS="-xO5 -libmil -m64"

## Options for safe compilation under the FORTRAN 77 compiler.
## Use this to specify FFLAGS for the version of the compiler specified
## above, using as accurate a result as possible, e.g. no optimization
## or using -ffloat-store.
SAFE_FFLAGS="-xO5 -libmil"

CXX="CC -library=stlport4"

## Options for the C++ compiler.
CXXFLAGS="-xO5 -xlibmil -nofstore -features=tmplrefstatic"

## Any special flags which must be used when compiling C++ code to be
## turned into a shared object.  If this variable is left unspecified
## an attempt is made to automatically detect the correct value.
CXXPICFLAGS=-Kpic

LDFLAGS=-L/opt/sunstudio12.1/lib/amd64
SHLIB_LDFLAGS=-shared
SHLIB_CXXLDFLAGS=-G
SHLIB_FCLDFLAGS=-G

## The command to be used to load shared objects which contain object
## files from the C++ compiler.  This is usually the C++ compiler/linker,
## but the automatic choice can be overridden by means of this
## variable.
## SHLIB_CXXLD=

## Any special flags which are required when creating shared objects
## containing object files from a C++ compiler.  This is usually
## automatically detected by configure, and anything set here will be in
## addition unless SHLIB_CXXLD is given.
## SHLIB_CXXLDFLAGS=


## FORTRAN 95 compiler: optional for use in packages.
## Analogous to the F77 and CXX settings.
FC=f95
FCFLAGS=$FFLAGS
FCPICFLAGS=-Kpic
SHLIB_FCLD=${FC}
## Unlike SHLIB_CXXLDFLAGS, SHLIB_FCLDFLAGS is never additional
SHLIB_FCLDFLAGS=${SHLIB_LDFLAGS}
## Additional libraries needed to link a shared object, e.g. on x86 Solaris
FCLIBS="-lfai -lfai2 -lfsu"

## Tcl/Tk settings.
## Use TCLTK_LIBS for all '-L' and '-l' options needed for linking
## against the Tcl and Tk library.
## TCLTK_LIBS=
## Use TCLTK_CPPFLAGS for all '-I' options needed for finding the tcl.h
## and tk.h headers.
## TCLTK_CPPFLAGS=

## Browser default
## Default setting for the R_BROWSER env variable
## If unset configure searches in turn for (currently)
## firefox mozilla galeon opera xdg-open kfmclient gnome-moz-remote open
## and uses the full path.
## R_BROWSER=

## BLAS and LAPACK settings
## Use BLAS_LIBS for all '-L' and '-l' options needed for linking
## against an external BLAS implementation.
## BLAS_LIBS=
## Use LAPACK_LIBS for all '-L' and '-l' options needed for linking
## against an external LAPACK implementation.  
## Note that (see R-admin) that our main intention is to allow a
## LAPACK-containing BLAS to be used, so this is rarely needed, and
## it is not used if the BLAS already contains LAPACK.
LAPACK_LIBS=/opt/solstudio12.2/lib

## Make name.
## Set this if you want to use a make by another name.
## For example, if your GNU make is called 'gmake', use 'MAKE=gmake'.
## MAKE=

## Tar name
## Set this to prefer a tar which has the capability to automagically
## read compressed archives.  The default is to choose 'gtar' or
## 'gnutar'(normally GNU tar) then 'tar'.
## Note that 'bsdtar' (from the libarchive project) is known to create
## archives in a non-POSIX format that untar() cannot read,
## so should be avoided.
## TAR=

## Library path
## This is be default created from libraries added to LIBS.
## Allow user to override.
R_LD_LIBRARY_PATH="/usr/local/lib:/opt/csw/gcc4/lib:/opt/csw/lib"
READLINE_LIBS = /usr/local/lib
USE_NLS = no
AR_FLAGS=-crS
lt_AR_FLAGS=-crS
LIBICONV=/usr/local/lib

So, here it is - all the revelent documentation on iconv.  If I've missed
anything please let me know.  If you see where in the documentation I missed
some cryptic clue on how to get iconv to work or where to put it or it's
headers, also please let me know.  

One things for sure - you can't say I've not read the documentation!

Roger

--
View this message in context: http://r.789695.n4.nabble.com/problems-with-iconv-tp4191177p4192313.html
Sent from the R devel mailing list archive at Nabble.com.

  
  


  


      

    

    

      
      

        


  

        

      Joshua Ulrich
    

    

      
      #
    

  


  

      
As Uwe asked, please cite the original messages.

        
On Tue, Dec 13, 2011 at 4:10 PM, RogerP <rpickeri at mail.nih.gov> wrote:

            

      
      
      
    

        
The location isn't mentioned in Appendix A, which you cite a portion
of above, but it is mentioned in Solaris section of Appendix C, which
you cite below.

            

      
      
      
    

        
This seems to indicate GNU libiconv should be installed in /usr/local,
which is the location specified in the libiconv installation
instructions:

As usual for GNU packages:
$ ./configure --prefix=/usr/local
$ make
$ make install

Did you try the suggestion two paragraphs above this in Appendix C?
"Some people have reported that the Solaris libintl needs to be
avoided, for example by using --disable-nls or --with-included-gettext
or using libintl from OpenCSW."

            

      
      
      
    

        
<snip>

            

      
      
      
    

        
Did you take the steps required to resolve the circular dependency
between libiconv and gettext?

            

      
      
      
    

        
Best,
--
Joshua Ulrich  |  FOSS Trading: www.fosstrading.com

  
  


  


      

    

    

      
      

        


  

        

      RogerP
    

    

      
      #
    

  


  

      
For some reason when I click on reply and sign-in  - the message I'm replying
to is not carried forward.  As you requested that include your message I
copied and pasted it from my email.

            

      
      
      
    

        
The location isn't mentioned in Appendix A, which you cite a portion 
of above, but it is mentioned in Solaris section of Appendix C, which 
you cite below.

            

      
      
      
    

        
This seems to indicate GNU libiconv should be installed in /usr/local, 
which is the location specified in the libiconv installation 
instructions: 

As usual for GNU packages: 
$ ./configure --prefix=/usr/local 
$ make 
$ make install 

Did you try the suggestion two paragraphs above this in Appendix C? 
"Some people have reported that the Solaris libintl needs to be 
avoided, for example by using --disable-nls or --with-included-gettext 
or using libintl from OpenCSW." 

As it turns out I put iconv and libiconv.so and libiconv.so.2.5.1 in
/usr/local/bin and /usr/local/lib.  Just for grins I copied these files to
/usr/local and tried again with the same results.  

This is my configure statement:

./configure --with-blas=-library=sunperf --with-lapack --with-readline=no
--x-includes=/usr/X11/include --x-libraries=/usr/X11/lib --prefix=/usr/local
--disable-nls  --with-included-gettext 

So, yes, I did try the suggestions mentioned in Appendix C.

            

      
      
      
    

        
<snip>

            

      
      
      
    

        
Did you take the steps required to resolve the circular dependency 
between libiconv and gettext? 

I used the --disable-nls  and --with-included-gettext options.  I also
downloaded the more up-to-date cairo package.  

BTW, is there a program I can run to test my iconv binary?  I think that
would rule out any problem with the compilation.

Thanks for you suggestions and for any future help.  This is frustrating
because I have compiled R, just not with a workable iconv, which prevents me
from updating and adding packages.

Roger




--
View this message in context: http://r.789695.n4.nabble.com/problems-with-iconv-tp4191177p4195139.html
Sent from the R devel mailing list archive at Nabble.com.

  
  


  


      

    

    

      
      

        


  

        

      Joshua Ulrich
    

    

      
      #
    

  


  

      
On Wed, Dec 14, 2011 at 8:15 AM, RogerP <rpickeri at mail.nih.gov> wrote:

            

      
      
      
    

        
I guess it's some Nabble nonsense then...

            

      
      
      
    

        
I was referring to the instructions on the libiconv page:
http://www.gnu.org/software/libiconv/

"On systems other than GNU/Linux, the iconv program will be
internationalized only if GNU gettext has been built and installed
before GNU libiconv."

That seems to suggest that libiconv *may* require you to install GNU
gettext before installing GNU libiconv.  Have you done that?

            

      
      
      
    

        
Best,
--
Joshua Ulrich  |  FOSS Trading: www.fosstrading.com

  
  


  


      

    

    

      
      

        


  

        

      RogerP
    

    

      
      #
    

  


  

          

      
      
      
    

        
downloaded the more >up-to-date cairo package.

            

      
      
      
    

        
would rule out any >problem with the compilation.

            

      
      
      
    

        
because I have >compiled R, just not with a workable iconv, which prevents
me from updating and adding >packages. 


I did some looking at iconv and tried 'iconv --list', but got an error
saying the libgcc was the wrong elf class - elf class 32.  So I changed the
/opt/csw/lib to /opt/csw/lib/amd64 on my env var LD_LIBRARY.  It ran!  So,
thought I, this is it - I've solved the problem.

But no, even though 'iconv --list' ran I still got the same error message.  

I did notice though that the list returns values for LATIN1 and not latin1
that configure and the update.packages() both seem to want.  

Here is the results of 'iconv --list':

646 ANSI_X3.4-1968 ANSI_X3.4-1986 ASCII CP367 IBM367 ISO-IR-6 ISO646-US
ISO_646.IRV:1991 US US-ASCII CSASCII
UTF-8
ISO-10646-UCS-2 UCS-2 CSUNICODE
UCS-2BE UNICODE-1-1 UNICODEBIG CSUNICODE11
UCS-2LE UNICODELITTLE
ISO-10646-UCS-4 UCS-4 CSUCS4
UCS-4BE
UCS-4LE
UTF-16
UTF-16BE
UTF-16LE
UTF-32
UTF-32BE
UTF-32LE
UNICODE-1-1-UTF-7 UTF-7 CSUNICODE11UTF7
UCS-2-INTERNAL
UCS-2-SWAPPED
UCS-4-INTERNAL
UCS-4-SWAPPED
C99
JAVA
CP819 IBM819 ISO-8859-1 ISO-IR-100 ISO8859-1 ISO_8859-1 ISO_8859-1:1987 L1
LATIN1 CSISOLATIN1
ISO-8859-2 ISO-IR-101 ISO8859-2 ISO_8859-2 ISO_8859-2:1987 L2 LATIN2
CSISOLATIN2
ISO-8859-3 ISO-IR-109 ISO8859-3 ISO_8859-3 ISO_8859-3:1988 L3 LATIN3
CSISOLATIN3
ISO-8859-4 ISO-IR-110 ISO8859-4 ISO_8859-4 ISO_8859-4:1988 L4 LATIN4
CSISOLATIN4
CYRILLIC ISO-8859-5 ISO-IR-144 ISO8859-5 ISO_8859-5 ISO_8859-5:1988
CSISOLATINCYRILLIC
ARABIC ASMO-708 ECMA-114 ISO-8859-6 ISO-IR-127 ISO8859-6 ISO_8859-6
ISO_8859-6:1987 CSISOLATINARABIC
ECMA-118 ELOT_928 GREEK GREEK8 ISO-8859-7 ISO-IR-126 ISO8859-7 ISO_8859-7
ISO_8859-7:1987 ISO_8859-7:2003 CSISOLATINGREEK
HEBREW ISO-8859-8 ISO-IR-138 ISO8859-8 ISO_8859-8 ISO_8859-8:1988
CSISOLATINHEBREW
ISO-8859-9 ISO-IR-148 ISO8859-9 ISO_8859-9 ISO_8859-9:1989 L5 LATIN5
CSISOLATIN5
ISO-8859-10 ISO-IR-157 ISO8859-10 ISO_8859-10 ISO_8859-10:1992 L6 LATIN6
CSISOLATIN6
ISO-8859-11 ISO8859-11 ISO_8859-11
ISO-8859-13 ISO-IR-179 ISO8859-13 ISO_8859-13 L7 LATIN7
ISO-8859-14 ISO-CELTIC ISO-IR-199 ISO8859-14 ISO_8859-14 ISO_8859-14:1998 L8
LATIN8
ISO-8859-15 ISO-IR-203 ISO8859-15 ISO_8859-15 ISO_8859-15:1998 LATIN-9
ISO-8859-16 ISO-IR-226 ISO8859-16 ISO_8859-16 ISO_8859-16:2001 L10 LATIN10
KOI8-R CSKOI8R
KOI8-U
KOI8-RU
CP1250 MS-EE WINDOWS-1250
ANSI-1251 CP1251 MS-CYRL WINDOWS-1251
CP1252 MS-ANSI WINDOWS-1252
CP1253 MS-GREEK WINDOWS-1253
CP1254 MS-TURK WINDOWS-1254
CP1255 MS-HEBR WINDOWS-1255
CP1256 MS-ARAB WINDOWS-1256
CP1257 WINBALTRIM WINDOWS-1257
CP1258 WINDOWS-1258
850 CP850 IBM850 CSPC850MULTILINGUAL
862 CP862 IBM862 CSPC862LATINHEBREW
866 CP866 IBM866 CSIBM866
CP1131
MAC MACINTOSH MACROMAN CSMACINTOSH
MACCENTRALEUROPE
MACICELAND
MACCROATIAN
MACROMANIA
MACCYRILLIC
MACUKRAINE
MACGREEK
MACTURKISH
MACHEBREW
MACARABIC
MACTHAI
HP-ROMAN8 R8 ROMAN8 CSHPROMAN8
NEXTSTEP
ARMSCII-8
GEORGIAN-ACADEMY
GEORGIAN-PS
KOI8-T
CP154 CYRILLIC-ASIAN PT154 PTCP154 CSPTCP154
KZ-1048 RK1048 STRK1048-2002 CSKZ1048
MULELAO-1
CP1133 IBM-CP1133
ISO-IR-166 TIS-620 TIS620 TIS620-0 TIS620.2529-1 TIS620.2533 TIS620.2533-0
TIS620.2533-1
CP874 WINDOWS-874
VISCII VISCII1.1-1 CSVISCII
TCVN TCVN-5712 TCVN5712-1 TCVN5712-1:1993
ISO-IR-14 ISO646-JP JIS_C6220-1969-RO JP CSISO14JISC6220RO
JISX0201-1976 JIS_X0201 X0201 CSHALFWIDTHKATAKANA
ISO-IR-87 JIS0208 JIS_C6226-1983 JIS_X0208 JIS_X0208-1983 JIS_X0208-1990
X0208 CSISO87JISX0208
ISO-IR-159 JIS_X0212 JIS_X0212-1990 JIS_X0212.1990-0 X0212
CSISO159JISX02121990
CN GB_1988-80 ISO-IR-57 ISO646-CN CSISO57GB1988
CHINESE GB_2312-80 ISO-IR-58 CSISO58GB231280
CN-GB-ISOIR165 ISO-IR-165
ISO-IR-149 KOREAN KSC_5601 KS_C_5601-1987 KS_C_5601-1989 CSKSC56011987
EUC-JP EUCJP EXTENDED_UNIX_CODE_PACKED_FORMAT_FOR_JAPANESE
CSEUCPKDFMTJAPANESE
MS_KANJI PCK SHIFT-JIS SHIFT_JIS SJIS CSSHIFTJIS
CP932
ISO-2022-JP CSISO2022JP
ISO-2022-JP-1
ISO-2022-JP-2 CSISO2022JP2
CN-GB EUC-CN EUCCN GB2312 CSGB2312
GBK
CP936 MS936 WINDOWS-936
GB18030
ISO-2022-CN CSISO2022CN
ISO-2022-CN-EXT
HZ HZ-GB-2312
CNS11643 EUC-TW EUCTW CSEUCTW
BIG-5 BIG-FIVE BIG5 BIGFIVE CN-BIG5 CSBIG5
CP950
BIG5-HKSCS:1999
BIG5-HKSCS:2001
BIG5-HKSCS:2004
BIG5-HKSCS BIG5-HKSCS:2008 BIG5HKSCS
5601 EUC-KR EUCKR CSEUCKR
CP949 UHC
CP1361 JOHAB KO_KR.JOHAP92
ISO-2022-KR CSISO2022KR

Is there still something wrong with iconv?

I put the header in the <R home>/include.  Was that right?

Thanks again for your attention,
Roger

--
View this message in context: http://r.789695.n4.nabble.com/problems-with-iconv-tp4191177p4196455.html
Sent from the R devel mailing list archive at Nabble.com.

  
  


  


      

    

    

      
      

        


  

        

      Peter Dalgaard
    

    

      
      #
    

  


  

      
On Dec 14, 2011, at 19:43 , RogerP wrote:

        

            

      
      
      
    

        
Looks quite normal. But is that the iconv that R is connecting to? You had an error that it couldn't convert latin1 to ASCII and both are on that list....

            

      
      
      
    

        
Probably not. I would expect something like /usr/local/include, and maybe a -I option to make sure it was found before the Solaris-supplied one. Does make install (for iconv) not do that for you?

You may need to look into config.log and see exactly what is going wrong when configure fails the test about supported conversions.

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      RogerP
    

    

      
      #
    

  


  

          

      
      
      
    

        
-I option to make sure it was >found before the Solaris-supplied one. Does
make install (for iconv) not do that for you?

Wow!  Thanks!  That was it!  I checked out /usr/local/include and the header
was there, but I'd uncommented a line in the config.site that caused it NOT
to look at /usr/local/include.  Re-commenting the line fixed that problem.

But now, Rsched will not compile from the rhome, though it does from the
rhome/src/unix directory.  So, it's like an onion.  You take a layer off and
cry - then repeat.  But, I'll make it.

Thanks again for your help!

Roger

--
View this message in context: http://r.789695.n4.nabble.com/problems-with-iconv-tp4191177p4200071.html
Sent from the R devel mailing list archive at Nabble.com.