(Just to summarise my answer to this duplicate question on SO and its comments...)
It's certainly bad practice, and it can also be a major legal problem.
Because both the user and your service will technically have had access to the private key, you can no longer guarantee who the actual signer was to a third party.
Do you know any javascript library that signs xml, pdf and office documents? This way, the private key would only be used on client-side.
Don't do JavaScript crypto. You won't be able to prove to the user what your code does (it could still send the key). As far as I'm aware, some of the WebCrypto project was trying to address some of these issues (e.g. by letting some JS code use the private key stored in the browser without letting it out), but it's not quite ready yet, and it doesn't solve all the problems (e.g. proving the the users what they're signing with that private key).
And of course, the technical show-stopper...
Most smartcard mechanisms don't let you get access to the private key itself (often by design), so you'd somehow have to interface with the signature API for the card anyway.
That API will not actually give you the private key, it will only let you use it to sign something, the processing being done on the card itself. So, any plan to send the private key somewhere or have some code (that doesn't use this API) use it wouldn't work.
While certificate on smartcards can be used by the browser for client-certificate authentication, because browsers are implemented to support this as part of their own user interface (nothing to do with any webpage you can provide), I don't think there's anything obvious to have a script on your webpage access any of this. (This would also still leave the problem of trusting what your JS code actually signs.)