基于所有或选定的列，在数据框架中查找重复的行

在这篇文章中，我们将讨论如何基于所有或一列列的数据框架中找到重复的行。为此，我们将使用Pandas的Dataframe.duplicated()方法。

语法: DataFrame.duplicated(subset = None, keep = ‘first’)
参数:
subset ：这需要一个列或列标签的列表。它的默认值是无。传递列后，它将只考虑它们的重复。
keep ：这控制如何考虑重复的值。它只有三个不同的值，默认是 “first”。

• 如果是’first’，则认为第一个值是唯一的，其余的相同值是重复的。
• 如果是’last’，则认为最后一个值是唯一的，其余相同的值是重复的。
• 如果’False’，这就认为所有相同的值都是重复的。

返回：布尔系列，表示重复的行。

让我们创建一个简单的数据框架，其中有一个列表的字典，比如列名是。’姓名’、’年龄’和’城市’。

# Import pandas library
import pandas as pd
 
# List of Tuples
employees = [('Stuti', 28, 'Varanasi'),
            ('Saumya', 32, 'Delhi'),
            ('Aaditya', 25, 'Mumbai'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Mumbai'),
            ('Aaditya', 40, 'Dehradun'),
            ('Seema', 32, 'Delhi')
            ]
 
# Creating a DataFrame object
df = pd.DataFrame(employees,
                  columns = ['Name', 'Age', 'City'])
 
# Print the Dataframe
df

输出 :

例子1：根据所有列选择重复的行。
这里，我们没有传递任何参数，因此，它对两个参数都采用默认值，即subset = None和keep = ‘first’。

# Import pandas library
import pandas as pd
 
# List of Tuples
employees = [('Stuti', 28, 'Varanasi'),
            ('Saumya', 32, 'Delhi'),
            ('Aaditya', 25, 'Mumbai'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Mumbai'),
            ('Aaditya', 40, 'Dehradun'),
            ('Seema', 32, 'Delhi')
            ]
 
# Creating a DataFrame object
df = pd.DataFrame(employees,
                  columns = ['Name', 'Age', 'City'])
 
# Selecting duplicate rows except first
# occurrence based on all columns
duplicate = df[df.duplicated()]
 
print("Duplicate Rows :")
 
# Print the resultant Dataframe
duplicate

输出 :

例子2：根据所有列选择重复的行。
如果你想考虑除最后一个以外的所有重复，那么就把keep = ‘last’作为一个参数传给他。

# Import pandas library
import pandas as pd
 
# List of Tuples
employees = [('Stuti', 28, 'Varanasi'),
            ('Saumya', 32, 'Delhi'),
            ('Aaditya', 25, 'Mumbai'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Mumbai'),
            ('Aaditya', 40, 'Dehradun'),
            ('Seema', 32, 'Delhi')
            ]
 
# Creating a DataFrame object
df = pd.DataFrame(employees,
                  columns = ['Name', 'Age', 'City'])
 
# Selecting duplicate rows except last
# occurrence based on all columns.
duplicate = df[df.duplicated(keep = 'last')]
 
print("Duplicate Rows :")
 
# Print the resultant Dataframe
duplicate

输出 :

示例3：如果你想只根据某些选定的列来选择重复的行，那么就把subset中的列名列表作为一个参数来传递。

# import pandas library
import pandas as pd
 
# List of Tuples
employees = [('Stuti', 28, 'Varanasi'),
            ('Saumya', 32, 'Delhi'),
            ('Aaditya', 25, 'Mumbai'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Mumbai'),
            ('Aaditya', 40, 'Dehradun'),
            ('Seema', 32, 'Delhi')
            ]
 
# Creating a DataFrame object
df = pd.DataFrame(employees,
                  columns = ['Name', 'Age', 'City'])
 
# Selecting duplicate rows based
# on 'City' column
duplicate = df[df.duplicated('City')]
 
print("Duplicate Rows based on City :")
 
# Print the resultant Dataframe
duplicate

输出 :

示例4：根据一个以上的列名选择重复的行。

# import pandas library
import pandas as pd
 
# List of Tuples
employees = [('Stuti', 28, 'Varanasi'),
            ('Saumya', 32, 'Delhi'),
            ('Aaditya', 25, 'Mumbai'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Delhi'),
            ('Saumya', 32, 'Mumbai'),
            ('Aaditya', 40, 'Dehradun'),
            ('Seema', 32, 'Delhi')
            ]
 
# Creating a DataFrame object 
df = pd.DataFrame(employees,
                   columns = ['Name', 'Age', 'City'])
 
# Selecting duplicate rows based
# on list of column names
duplicate = df[df.duplicated(['Name', 'Age'])]
 
print("Duplicate Rows based on Name and Age :")
 
# Print the resultant Dataframe
duplicate

输出 :