Where I am, the perceived problems are:
support costs - that the company will bear the burden of issues which aren't realy it's problem (Hello helpdesk, how do I configure my Whizzbang 3728-T for accessing the VPN?)
that unmanaged user devices become a conduit for malware into a controlled environment
that data on this devices is not adequately secured and the devices are more susceptible to physical loss
and to being compromised for data loss
data leakage - where the user deliberately stores company data outside of the company's control - e.g. using a cloud based app to maintain an address book
Personally, I disagree with the idea that there should be a very different inside and outside to a network in terms of security (yes, firewall policy should be different - but only to cut down on the noise). But I would have to concede that not everyone who works here has the skills / time to maintian their own devices to a reasonable standard.
an administrator cannot force restrictions as easily as with a company-owned device
This is a tricky one.
Firstly, your assertion is not necessarily true. You should have adequate controls in place that merely bringing a device into your workplace does not provide access to services you might consider restricted.
Once this is the case, then as an administrator you can enforce certain constraints on that access. There are lots of tools available now which provide remote wipe and remote access - but then you need to address the problem of how you seperate the company's data from the users data. Should you be asking your users to surrender their privacy? Should you be spending company money to buy security software to install on non-company devices?