Remove duplicate items from a list in Python

Output

['a', 'b', 'c']

Removing duplicates from a list can be helpful in many applications where we want to reduce data and want to get distinct records from the list. Here we are explaining some of the methods that can be useful to get the distinct records from the list.

Remove duplicates from a list using list comprehension

We can use list comprehension to remove duplicates from a list. We can define a new list that will contain distinct items and in each iteration, it will check that item does not exist already in the new list.

The new list with distinct records will also maintain the sequence of the item from the input list.

Code Example

alphabets = [4, 4, 5, 9, 4, 3, 6, 9]

result = []

[result.append(x) for x in alphabets if x not in result]

print(result)

Output

[4, 5, 9, 3, 6]

Remove duplicate items using list() and set() function

Another example of removing duplicate items from the list using list() and set() functions. You just need to pass your list to set() function and then convert it to the list..

Not that this method will not maintain the order of items in the list

Syntax

list(set(List))

Code Example

my_list = ['a', 'b', 'a', 'c', 'd', 'a', 'a']

result = list(set(my_list))

print(result)

Output

['c', 'd', 'a', 'b']

Remove duplicate items using For Loop

If you do not want to use any function or extra module to remove duplicate elements from the list then you can use Python For Loop only to do that.

Code Example

lst = [2, 4, 2, 5, 9, 2, 5, 4, 9]

result = []
for item in lst:
  if item not in result:
    result.append(item)

print(result)

Output

[2, 4, 5, 9]

The above method will also maintain the order of items in the final list.