I encounter many tasks in which I need to filter python (2.7) list to keep only ordered unique values. My usual approach is by using odereddict from collections:
from collections import OrderedDict
ls = [1,2,3,4,1,23,4,12,3,41]
ls = OrderedDict(zip(ls,['']*len(ls))).keys()
print ls
the output is:
[1, 2, 3, 4, 23, 12, 41]
is there any other state of the art method to do it in Python?
listedit - a comparison of the methods can be found here: https://www.peterbe.com/plog/uniqifiers-benchmark
the best solution meanwhile is:
def get_unique(seq):
seen = set()
seen_add = seen.add
return [x for x in seq if not (x in seen or seen_add(x))]
If you need to preserve the order and get rid of the duplicates, you can do it like:
ls = [1, 2, 3, 4, 1, 23, 4, 12, 3, 41]
lookup = set() # a temporary lookup set
ls = [x for x in ls if x not in lookup and lookup.add(x) is None]
# [1, 2, 3, 4, 23, 12, 41]
This should be considerably faster than your approach.