With few exceptions, your ISP most certainly can - and in all likelihood probably does - know your physical location at any given time. This stems from two general facts:
- They are providing a service that you signed up for. During the subscription process, you most likely already gave them your location for installation and/or billing purposes.
- They most likely own the infrastructure devices that are directly connected to your device, or the gateway device you're connecting to.
Essentially, the only relatively convenient, usually legal, and reasonably inexpensive way to avoid location tracking by your ISP is to not use your ISP. This means hooking up to the Internet at a friend's house, or some other location that provides free or guest network access. Unfortunately, that also means never using the Internet from home or on your cell phone.
Breaking this down into a bit more detail, you'll see how it's quite clear that your ISP not only can but most often must be capable of knowing your physical location. There are a few ways to get around this, but they're generally inconvenient, costly, and/or illegal.
1. Billing
Apart from courtesy "guest networks", Internet services are rarely just given away for free. (Even most guest networks aren't really free, or at least not intended to be - you're just not paying for the Internet service explicitly.) So, the ISP must have some way of charging you for the service.
Unless your ISP permits you to physically visit their offices and pay them in cash every month, the billing process will usually require some collection of personal information. Normally, this includes the mailing address that is associated with your payment mechanism. For most people, this also happens to be their business or residential address. In the case of non-wireless services, this is also probably the location from which you'll be consuming the service (i.e.: connecting to the Internet).
In some cases, you may be able to avoid giving away your physical location through billing processes. For example, you could use a P.O. Box as your mailing address. However, some ISPs and/or payment providers (e.g.: credit card companies or banks) may not permit this.
2. Installation
Non-wireless broadband Internet services generally require some amount of equipment to be installed, or special cables run, at the location from which the services will be used. This necessitates telling the ISP where you're going to be connecting from, so that they can send technicians out to set up the devices and cables necessary for you to connect.
You can get around this by using wireless services, such as your cellular phone provider, but these connections are usually slower and less reliable and they often can still be traced. Another work-around could be using dial-up connections, but these will likely be even slower and again are still not immune to other tracking mechanisms.
3. Emergency Services
If your ISP is also providing cellular, VoIP, or other phone services, they may be required by law to know your location in order to properly route emergency calls and direct police and rescue personnel to you when needed.
One way that providers may do this is through a GPS receiver in the device that is used to initiate the emergency call. In the case of VoIP services, the receiver may be built into the VoIP gateway. This is why some providers will recommend that you place your gateway device near a window during setup (if not permanently) - that enables the built-in GPS receiver to have better visibility to the satellites required to determine its location.
You could try to negate the GPS lock by use of a Faraday cage or other GPS-opaque structure over the gateway device. However, if your service is coming through a wireless connection, this will most likely degrade or flat-out kill your Internet link as well. Besides, if a GPS lock is not available (or the system isn't using GPS to begin with) the provider will usually fall back to mechanisms listed in other sections of this answer.
4. Subscriber Authentication
When you connect to your ISP, the network will need some way of authenticating you before providing service. This allows them to verify that you are indeed one of their customers, and to determine what quality of service (e.g.: bandwidth allowances and caps) they should provide to the connection per the terms of your subscription.
This is practically unavoidable, without illegally hijacking someone else's authenticators. While it may not necessarily provide your ISP with your location directly, (except perhaps by way of items 1 and 2) it does enable the ISP to associate location data gathered from other sources (e.g.: items 3 and 5) with your identity.
5. ISP-Owned Infrastructure
This is the piece that is probably the hardest and most expensive to avoid, and is effectively the whole reason you have to pay someone else for Internet service. Chances are, the device that your system is directly connected to (if not, then probably the first hop outside of your LAN) is owned and maintained by your ISP.
Your ISP probably also controls the next few devices after that, and these devices are not likely to be easily accessible - lat alone movable - by you. By the very nature of their business, the ISP must know the locations of these devices in order to maintain and repair them in a timely fashion when needed. If these devices aren't actually in the same room or building you are in, they are still probably very close to you - within a couple miles or less.
This applies to most wired and wireless broadband services. For wired services, the "gateway" device that is on-premises is usually provided and maintained by the ISP. If not, then it will be the next hop after that and a few more from there.
For wireless services, the ISP will usually own the transceivers (e.g.: cell towers) that your device is directly communicating with. If your device is only within range of one or two provider-owned transceivers, they may not be able to pinpoint you very closely. Three or more, and they can usually nail you down pretty well. (Of course, your device is only ever actually using one of the ISP's transceivers at a given time, but they will often have several within "listening" range of your device in order to have good service coverage.)
The workarounds for this are usually fairly expensive, very inconvenient, or flat-out illegal. In some cases, it may be just plain impossible. Essentially, they all entail getting your device off of ISP-owned infrastructure for at least the first hop past your LAN.
For wired connections, this tracking will often not be avoidable because the gateway links that your device needs to connect to are unreachable outside of the ISP's networks. A likely exception to this would be dial-up networks. However, these links tend to be very slow and they are usually still traceable to some extent via Caller ID. (Caller ID can be spoofed, but that is often illegal.)
For wireless connections, avoiding ISP-owned infrastructure usually necessitates substantial physical relocation (or subscribing to a provider who does not have transceivers near your usual location) and will often incur additional fees (typically called "data roaming charges" or similar) as well as service degradation and/or limitation. If the wireless service is through a device also used to make phone calls (as most wireless Internet services tend to be), trying to avoid ISP collection of location data like this could ultimately be foiled by item 3 above anyway.
