This is the approach taken by PGPfone also. If the value of the conversations was high (>$100,000?) passable voice imitation wouldn't be that hard I suspect.
I have long considered how easy it would be to use a sound card to modify the human voice to match within certain tolerances the voice of another.
There are currently on the market, phones specifically designed to modify the voice of the user so that kids can answer as adults, women can answer as thier own protective boyfriends, bosses can answer anonymous calls as the secretary, etc...
There are currently on the market keyboards that allow you to sample some real world sound and use it as a voice in your music, (the model I saw, a toy produced by Radio Shack, simply sped up or slowed down the sound to achieve this.)
I have thought, if a machine were to take the incoming voice, analize (apologies for spelling) it to get a spectrum signature, a pattern that can be added or subtracted from another, and could then add the difference between that and the victims signature to the users voice, then real-time, on-the-fly con jobs would be easy.
The only thing that the user would be responsible for would be the accent, and the day-to-day vocabulary of the victim.
I told a friend about this and he confirmed that such was available if you knew where to look.
A friend of mine, an expert on signal processing, vocei systhesis and recognition, showed me a journal article (think it was an IEEE) in 1990 of some university researchers who had prototyped just such a device. Never followed up, but it seems entirely reasonable a practicle. In fact I'm surprised that Hollywood hasn't latched onto this in order to dub film stars to different languages w/o loosing their recognizable voice characteristics. --Steve