Personally I like it if my uses clauses are as small as possible, but in many applications the really big units (in terms of bloating the executable) like Forms or VirtualTrees are needed in at least another unit anyway.
So: Does it make a difference if I clean my uses clauses even if in the end no unit is removed from the project? If so: In what way? And: Is cleaning the uses clause something which should be done as soon as possible or can it wait until I find an unused unit by chance?
If it's used elsewhere in the project, it won't make much of a difference, except to produce cleaner code that's easier to read. There are a few minor things it might affect, though.
Compilation order: The compiler decides what order to compile units in based on which units use which units. If you remove a unit from the uses clause of an early unit, that might cause the used unit to be compiled later in the compile cycle. This may not sound like much, but bear in mind that initialization sections run in the same order as the units were compiled. This really shouldn't make much of a difference to your project, though.
CodeInsight: When you pull up the code completion drop-down, it will offer choices based on all the units currently available. You can reduce the number of choices it has to filter through--and thus the amount of time it takes to pull the bloody thing up!--by reducing the number of units you're using. (No, I'm not bitter. Why do you ask?)