A set data structure may be unfamiliar to a Python beginner. A set is used for sequence data structures. Therefore, you can have an image against a set like a list, tuple, and dictionary.
First, let’s look at a list as an example. The sample list “sample_list” has six elements, however, whose unique elements are three kinds. The elements are four “apple”, one “orange”, and one “grape”.
sample_list = [
"apple",
"orange",
"grape",
"apple",
"apple",
"apple"
]
print(sample_list)
>> ['apple', 'orange', 'grape', 'apple', 'apple', 'apple']
You might have encountered the situation that you would like to know the unique elements of a list. Such a situation is the time to use the set() function in Python. Note that the set() is included in Python as a standard module, so you don’t need to import any external module.
It is easy. You just pass the list to the set() function as follows.
sample_list_unique = set(sample_list)
print(sample_list_unique)
>> {'apple', 'orange', 'grape'}
You have found out the unique elements of the sample list.
Like the output of the above cell, a set is created by the curly braces {}.
set_sample_without_overlaps = {"apple", "orange", "grape"}
print(set_sample_without_overlaps)
>> {'apple', 'orange', 'grape'}
Of course, if there are overlaps when we define the elements, these are ignored. Let’s put several “apple” elements in the set when defining. You will see that additional “apple” elements are ignored.
set_sample_with_overlaps = {"apple", "orange", "grape", "apple", "apple", "apple"}
print(set_sample_with_overlaps)
>> {'apple', 'orange', 'grape'}
Application example
A set is so useful in data science. This is because we have many situations to confirm the overlaps between datasets.
Here is an example. We will check the overlaps of the id column between training and validation datasets. We can do this easily by the set() function and the “intersection” method. The intersection method returns the overlaps elements between two set-type data.
id_train = ["01", "02", "03", "04", "05"]
id_validation = ["04", "05", "06", "07", "08"]
# Into a set data structure
id_train_set = set(id_train)
id_validation_set = set(id_validation)
# Check an overlaps
id_overlap = id_train_set.intersection(id_validation_set)
print(id_overlap)
>> {'05', '04'}
We have known that the elements of “05” and “04” coexist in the training and validation datasets. From this fact, we should perform a preprocessing, for example dropping overlap data.
Mixing the same information as the training data with the validation data is called data leakage, which leads to an overestimation of accuracy.
Summary
As a sequence data structure, a list, tuple, dictionary are famous. However, a set is practical when we treat unique elements.
You will surely come across a situation where you want to know the unique element of sequence-type data. Recall that there is a set data structure in the Python standard module s at that time.
