Does the CA need to mandatorily publish the new CRL file even though there are no new certificates that have been compromised since the last CRL published?
yes. CRL has fixed validity period determined by
NextUpdate fields. After
NextUpdate timestamp, CRL is considered expired and client application will return
RevocationOffline chain validation status.
If CA needs to publish the CRL's periodically, how do I identity whether there are new records added to the revocation list or not?
by periodically checking for new CRLs and examining them.
Is timestamping the CRL file a mandate or an optional feature?
in practice -- mandatory requirement.
Do All CA certificates need to use EKU(Extended Key Usage) set to CRLSign for the purposes of issuing the CRL's in the future?
Also, If the CRL validity is a week and published every Sunday, what would be implications on security if a certificate is compromised on Tuesday
then this will be detected by clients no later than upcoming Sunday.
As an end device, the next CRL will be only looked at after Sunday and so I would have accepted the connection from the compromised party for the last 5 days.
it highly depends on client implementation. Most clients will cache once downloaded CRL for the CRL validity duration and won't download it again until it expires. Some clients, such as Microsoft CryptAPI client implements E-tag and Max-age HTTP headers that allow HTTP CDP endpoint polling without having to download CRL if it wasn't changed on HTTP server. Once new CRL is published to HTTP endpoint, E-tag will change and CryptoAPI client will forcibly re-download CRL, so detection of newly revoked certificates occurs faster. It is
Another option is to use Delta CRLs that include only revoked certificates since last Base CRL was published. Delta CRLs are meant to be smaller in size and can be published frequently, say every day.
Depending on use case, clients may utilize OCSP stapling (when we talk about online TLS connections), when TLS server obtains OCSP response for its own certificate and staple in response during handshake. But this vary between certificate usage scenarios (hence certificates are not only TLS, there are miriads of other usages).