Yes it does work as you say. The chip is "tamper resistant" and will erase the "seed" (secret key) if any attempt is made to attack it. This is often accomplished by having a non-user-replaceable battery and a "trap" that breaks power to the device once the device is opened, or the chip surface is removed. The key is then stored in a SRAM, requiring power to keep the key.
The key is a seed, that combined with the current time in 60 second step (effectively, the current UNIX timestamp / 60), refreshes the code.
No, the device does NOT need to be precise. Instead, the server will store the time of the last accepted code. Then the server will accept a code one minute earlier, one minute ahead, and at the current time, so if the current time at server is 23:20, then it will accept a code from 23:19, 23:20 and 23:21.
After this, it will store the time of the last accepted code, eg if a 23:21 code was accepted, it will store 23:21 in a database, and refuse to accept any code that was generated at 23:21 or earlier.
Now to the interesting part: To prevent an imprecise token from desynchronizing from the server, the server will store in its database, if it was required to accept a 23:19 code or a 23:21 code at 23:20 time. This will ensure that at next logon, the server will correct the code with the number of steps.
Lets say you at Clock 23:20 login with a 23:19 code. Server stores "-1" in its database (and if it would be a 23:21 code, it would store "+1" in database). Next time you login, Clock is 23:40. Then the server will accept a 23:38, 23:39 or 23:40 code.
If a 23:38 code is accepted, it will store "-2" in database, at 23:39 it will keep "-1" in database, and at 23:40 it will store "0" in database.
This effectively makes sure to keep the server synchronized to your token. On top of this, the system, if a token "ran too far away" from the server (due to it being unused for a long time), allows resyncronization. This is accomplished either by a system administrator, or a self service resynchronization service is presented where the token user is asked to provide 2 subsequent codes from the token, like 23:20 and 23:21, or 19:10 and 19:11. Note that the server will NEVER accept a token code generated at or prior to the time that "last used token code" was (as this would allow reuse of OTP codes). When a resyncronization is done, the token will store the difference from the provided 2 token codes, and the current server time and in a resync, the search window could be like plus/minus 50 steps (which would allow about 0,75 hours of desync in both directions).
The server can detect a desynchronized token by generating the 50 prior codes and 50 future codes, and if the specified code matches that, it will launch the resync process automatically. Many times, to prevent an attacker from using the resync process to find valid codes, once an account is in resync mode, login will not be accepted without resyncing, which would require the attacker to find the exact code subsequent or prior to the code just found.