Remove duplicate items from a list in Python
If you are working on a list in Python and have a list of items that contains duplicate elements in it. To remove the duplicate elements we can use the methods explained in this post.
# Define a list
numbers = ['a', 'b', 'a', 'c', 'a', 'b']
# Define result list
result = []
# Use list comprehension to remove duplicates
[result.append(x) for x in alphabets if x not in result]
# print the result
print(result)
Output
['a', 'b', 'c']
Removing duplicates from a list can be helpful in many applications where we want to reduce data and want to get distinct records from the list. Here we are explaining some of the methods that can be useful to get the distinct records from the list.
Remove duplicates from a list using list comprehension
We can use list comprehension to remove duplicates from a list. We can define a new list that will contain distinct items and in each iteration, it will check that item does not exist already in the new list.
The new list with distinct records will also maintain the sequence of the item from the input list.
Code Example
alphabets = [4, 4, 5, 9, 4, 3, 6, 9]
result = []
[result.append(x) for x in alphabets if x not in result]
print(result)
Output
[4, 5, 9, 3, 6]
Remove duplicate items using list() and set() function
Another example of removing duplicate items from the list using list() and set() functions. You just need to pass your list to set() function and then convert it to the list..
Not that this method will not maintain the order of items in the list
Syntax
list(set(List))
Code Example
my_list = ['a', 'b', 'a', 'c', 'd', 'a', 'a']
result = list(set(my_list))
print(result)
Output
['c', 'd', 'a', 'b']
Remove duplicate items using For Loop
If you do not want to use any function or extra module to remove duplicate elements from the list then you can use Python For Loop only to do that.
Code Example
lst = [2, 4, 2, 5, 9, 2, 5, 4, 9]
result = []
for item in lst:
if item not in result:
result.append(item)
print(result)
Output
[2, 4, 5, 9]
The above method will also maintain the order of items in the final list.
from collections import OrderedDict
source = [1, 1, 3, 2, 3, 1, 2, 4]
result = list(OrderedDict.fromkeys(source))
print(result)
# -> [1, 3, 2, 4]
- Pandas - Remove duplicate items from list
- Remove all items from a list in Python
- Remove items less than a specific value from Python list
- Remove duplicate rows from DataFrame in Pandas
- String characters to list items using python
- Add two list items with their respective index in python
- Add items to list in Python