I have added the X-Frame-Options header for my HTML files from web server itself. Do I still need to add it for other static files like CSS and JS( which I serve from cloudfront/s3) ?
No. The reason we need XFO (or the corresponding CSP frame-ancestors directive) is clickjacking. Clickjacking is not an issue for JS or CSS files, so you do not need to set the XFO header for them. But doing so doesn't really hurt, so to save yourself from configuration mistakes you might as well set it for all resources. It's simpler that way.
It sounds like you are setting the header from the HTML file? If so, here's a note from OWASP:
Meta-tags that attempt to apply the X-Frame-Options directive DO NOT WORK. For example, ) will not work. You must apply the X-FRAME-OPTIONS directive as HTTP Response Header as described above.