While John Deters and Schroeder have covered most of the points, it is basic data hygeine that any information you capture should have a planned lifecycle. That's not just about deciding how long you are going to keep it, but having the mechanisms in place to capture its age/planned retiral date and a way of removing the data.
That you don't know how long ...
In addition to schroeder's good answer, it helps to work with the Project Managers to identify why they need the data. What do they use old data for? Does it save them time? Does it provide a shortcut for return customers? Does it produce valuable business intelligence? Does it allow you to re-run a job in case of a mistake?
Once you know how they plan ...
Never hold anything for longer than you absolutely need it. Your factors in how long that is can include:
financial reporting requirements
legitimate business reasons
I think that the US as a 10-year requirement for financial reporting (it's been a while so I might be wrong).
At a high level, the answer is, that we know something about the files or directory structure. If we know something, that means that we can look for it on a disk, even if it is not in the current directory tree.
This may sound very abstract, and it probably is. If you have a Linux system available, you can use a small (stay below the 1G if you can) USB ...