Message-ID: <1115843233.32309.20.camel@blue.chem.psu.edu>
Date: 2005-05-11T20:27:13Z
From: Rajarshi Guha
Subject: entropy and conditional entropy for continous variables
Hi,
this is not a R question per se, but since I'm on the lookout for an R
solution I thought this was the best place:
I would like to calculate the entropy for a variable and the conditional
entropy between two variables, H(X|Y) for variables X & Y
I have coded the case for the categorical case but I'm having problems
understanding how to do it for the continous case.
>From what I understand, for continous variables the entropy would be
\integral_{A} f(x) log( f(x) ) dx
where f(x) is the density of X and A is the support.
I have tried obtaining the density of X by using the density estimation
methods of the KernSmooth package and then doing a numerical integration
- however I am confused about how to interpret the support in this
context.
So the question is, are there any R functions to determing the entropy
and conditional entropy for continous variables? (Searching CRAN did not
turn up anything significant)
Thanks,
-------------------------------------------------------------------
Rajarshi Guha <rxg218 at psu.edu> <http://jijo.cjb.net>
GPG Fingerprint: 0CCA 8EE2 2EEB 25E2 AB04 06F7 1BB9 E634 9B87 56EE
-------------------------------------------------------------------
All theoretical chemistry is really physics; and all theoretical
chemists
know it.
-- Richard P. Feynman