Would it be more efficient to work up rather than down? That is, for each point N, if N is not already marked as redundant, compute the set of values >N which are equivalent, and mark them as redundant.

Probably not a significant gain, but worth throwing out there.