I have a list of dictionaries like this:

[ {'id': 1, 'name': 'one'}, {'id': 2, 'name': 'two'}, {'id': 1, 'name': 'one'}, {'id': 3, 'name': 'three'} ]

I want a list with unique id (no matter what is in the other fields of the dictionary). That is, from the example above should get this:

 [ {'id': 1, 'name': 'one'}, {'id': 2, 'name': 'two'}, {'id': 3, 'name': 'three'} ]

4 answers 4

To leave in the list only dictionaries with a unique id (remove repetitions):

 result = list({d['id']: d for d in list_of_dicts}.values())

This is a linear time and memory algorithm.

Result:

 [{'id': 1, 'name': 'one'}, {'id': 2, 'name': 'two'}, {'id': 3, 'name': 'three'}] 
     from itertools import groupby lst = [ {'id': 1, 'name': 'one'}, {'id': 2, 'name': 'two'}, {'id': 1, 'name': 'one'}, {'id': 3, 'name': 'three'} ] key = lambda dct: dct['id'] lst.sort(key=key) lst = [list(tpl[1])[0] for tpl in groupby(lst, key=key)] print(lst)
    • one
      can be simplified slightly: [next(dups) for _, dups in groupby(lst, key=key)] or simply: list(more_itertools.unique_justseen(lst, key=key)) or not at all: result = list(more_utils.unique_everseen([{'id':1,...}, {'id":2, ...}], key=lambda d: d['id'])) - jfs
    • Indeed, it came out more clearly. Plus, a small optimization, where next instead of creating a list. - Xander

    I would probably use a dictionary.

     lst = [ {'id': 1, 'name': 'one'}, {'id': 2, 'name': 'two'}, {'id': 1, 'name': 'one'}, {'id': 3, 'name': 'three'} ] d = {} for a in lst: d[a['id']] = a d.items()
    • at the end: lst = list(d.values()) is missing. - jfs
     lst = [{'id': 1, 'name': 'one'}, {'id': 2, 'name': 'two'}, {'id': 1, 'name': 'one'}, {'id': 3, 'name': 'three'}] items = lambda d: tuple(d.items()) list(map(dict, set(map(items, lst))))

    Comparison of performance methods:

     from timeit import Timer from itertools import groupby def test_1(items=lambda d: tuple(d.items())): list(map(dict, set(map(items, lst)))) def test_2(): d = {} for a in lst: d[a['id']] = a list(d.values()) def test_3(key=lambda dct: dct['id']): lst.sort(key=key) [next(dups) for _, dups in groupby(lst, key=key)] def test_4(): list({d['id']: d for d in lst}.values()) def test_5(): result = [] _id = [] rapp = result.append iapp = _id.append for d in lst: i = d['id'] if i not in _id: iapp(i) rapp(d) if __name__ == '__main__': gbls = globals() for fn in sorted(k for k in gbls if k.startswith('test_')): print('{:<6} = {:.2} ัะตะบ'.format(fn, Timer(gbls[fn]).timeit()))

    out:

     test_1 = 9.4 ัะตะบ test_2 = 1.8 ัะตะบ test_3 = 5.8 ัะตะบ test_4 = 1.9 ัะตะบ test_5 = 1.9 ัะตะบ