The problem (among others) is that this allows a virus to steal the client cert. If it is protected by a password, the malware must hang around long enough for the user to unlock the cert (perhaps because the malware sent a spoofed email calling for the user to visit the site, even the real site!). It can then read the user's keystrokes and acquire the password. Now it has the cert and password and can impersonate the user at will.
The solution to this is Palladium (NGSCB).
BAH! *shudders* All we need for this is an external cryptographic token - a smartcard with a keypad, an USB device, a Bluetooth-enabled thingy. You plug it into the machine, the server you connect to sends its certificate name and challenge to the browser, which passes it unchanged to your token. The token asks you for a PIN, and calculates a response. The browser then transparently relays the response back. There is nothing in the unit that's accessible from the computer, and because of a physically different keypad nothing can be sniffed from the computer. The cost of the unit can get as low as few dollars, can easily interface with just about any OS including PDAs, and doesn't require The Megacorp Whose Name Shouldn't Be Spoken to take over your machine.