If you do disable support for TLS1.0, you'll have plenty of company. The payment card industry (PCI) has mandated sites that accept credit card payments to phase out support for TLSv1.0. Soon, all sites that accept card information will be required to support TLSv1.1 or higher.
Without question, there are still a fair number of users that are using older browsers that do not support TLS1.0. But these users are sure to find that they are unable to connect to a quickly growing number of sites (among yours, if you decide to disable TLS1.0), and will be compelled to upgrade to a newer browser sooner rather than later.
See this Wikipedia page for a good reference on SSL/TLS protocols supported by various browsers.