If I have lost my phone, reported it, but I had the SIM, would the SIM be tracked back to me if I got a new phone and used this SIM?
Yes, when used the IMSI is constantly tracked by cell-phone providers, primarily for billing purposes.
If you only lost the phone and not the SIM card you need to mention that to the phone company as well. They will likely cancel your SIM card and re-issue you a new one with the same phone number anyway. Assuming you are getting a police report for insurance purposes and your own legal protection you will definitely want to correctly mention what was and was not stolen and make sure the police report states that clearly.
Note: It's always helpful to report the phone stolen to police even if it's not valuable to you. If there is a spike in cell phone theft it helps them justify spending money to track and arrest more of the people staling the phones. Make sure you call their non-emergency line to do this or better yet see if your police department has a web-page for reporting this type thing. More and more cities have web pages for reporting things like this and it can be done by simply filling out a one-page form that in some cases can even be done anonymously.