Hal writes:
The first is whether this mysterious black box, the entropy estimator, is really possible. In practice the only way to know how much entropy you've gotten is to have a model for how the data is being generated, and to deduce from that an estimate of the entropy rate. So the entropy estimator can't be a general-purpose calcluation, but it must be one which is specifically chosen, developed and tuned for the specific source of entropy you are dealing with.
I couldn't possibly say that better. Its the central point.
So I think the lesson is that there is only one way to estimate entropy, and that is to study your source. I have to agree with Perry that this filtering concept is not the way to go. It is a red herring that lures you in the direction of automatic entropy estimation, and that is really not safe.
Thank you; you are making the point far better than I did. .pm