It is actually possible to set your DNS servers in the phone's UI, by choosing a static IP in the advanced network connections settings. According to a question on StackOverflow, these settings can be programmatically altered as long as the WRITE_SETTINGS permission is set, using the following code:
android.provider.Settings.System.putString(getContentResolver(), android.provider.Settings.System.WIFI_USE_STATIC_IP, "0");
android.provider.Settings.System.putString(getContentResolver(), android.provider.Settings.System.WIFI_STATIC_DNS1, "192.168.0.2");
android.provider.Settings.System.putString(getContentResolver(), android.provider.Settings.System.WIFI_STATIC_DNS2, "192.168.0.3");
android.provider.Settings.System.putString(getContentResolver(), android.provider.Settings.System.WIFI_STATIC_GATEWAY, "192.168.0.1");
android.provider.Settings.System.putString(getContentResolver(), android.provider.Settings.System.WIFI_STATIC_NETMASK, "255.255.255.0");
android.provider.Settings.System.putString(getContentResolver(), android.provider.Settings.System.WIFI_STATIC_IP, "1");
If you have USB Debugging enabled, the settings can also be changed by software on your PC, without any permission requirements or user interaction.
Onto the second part of your question: yes, this is most definitely a security concern. If an app sets your DNS server to a rogue server, it can redirect traffic to any IP address it wishes, facilitating a man-in-the-middle attack. Combined with tools like BeEF, sslstrip and sslsniff, it's almost trivial to completely spoof your browsing experience.