Linux Uniq Command

A Linux text application called Uniq searches the duplicated lines in a file. In simplest terms, uniq is a tool that aids in both identifying and eliminating all duplicate lines from a file. The uniq command gives us access to several distinct flags including “-c”, “-u”, “-d”, “-D”, and “-i” among others, to perform various operations. By employing these uniq command options, we can display the number of words in any line, print only the lines that are repeated, or show the unique content of the file. We’ll talk about a few of the uniq command flags in this article.

Example: Utilizing the Linux Uniq Command

In this example, we use the “uniq” command from the Linux operating system. This command is used to condense or summarize the information. The “Uniq” command is used to locate the repetitive lines that follow one another. Let’s just begin the execution of the uniq command. However, before doing so on the terminal, we must first create a text file. Since the text file already exists on our desktop, we will use that. The “Text file.txt” is the name of the file that is already on our desktop. The text file’s extension must be “.txt”.

The “cat” command, which is often used in Linux, is used to open this text file on the terminal that contains some data. The file’s data is extracted, and its contents are output. It makes merging, reading, and creating the files easier. Therefore, we type “cat” first, followed by the file’s name which is “textfile.txt”.

Now, as you can see, when we run this command, the data from the currently existing file in the text file called “textfile.txt” is displayed on the terminal window. This file has nine lines. The first two of which are identical and carry the “Alex likes coding with various technologies” information. The next line is empty, and the fourth and fifth lines include the same information including “Working with scripting languages is my interest”. The last two lines have two separate lines which are “Some of the programming languages are “java”, “oop”, “php”, “c++,” “python,” etc. and “The language used to create computer programs is called a programming language.” Now, it is apparent that the first two lines are identical and repeated, as the following two. While the last two lines have different content, we use the “uniq” command to remove the duplicate lines from the file.

Now, let’s just use the uniq command to delete the duplicate line from the file. To do this, execute the following command on the terminal. Then, enter the “uniq” keyword followed by the “textfile.txt” file name.

As seen in the following, when we run this command, it does not display the duplicated lines on the output in the terminal window. There are nine lines in the original file. But after applying the uniq command, there are only six. This is because the first line, “Alex likes coding with various technologies”, appears twice in the original file but only once after applying the uniq command. The second line appears twice in the original file but only once after applying the uniq command. The last two lines are presented exactly as they are because they do not repeat in the file.

Using the Uniq Command’s Options

The uniq command gives us a few different options like “-c”, “-d”, “-u”, etc. to carry out the various tasks. We can count the number of repeated lines which are contained in the files using the uniq command’s options. We can only display the repeated lines of the files in the output, etc. Let’s utilize some of the uniq command’s following flags:

Counting the Number of Lines with the “-C” Flag

The number of lines in the file is counted using the “-c” flag of the uniq function. It counts the repeated lines as well as the single lines and displays the occurrence of each line by displaying a number as a prefix with the line. To accomplish this, we use the command in which we first enter the “uniq” word, then the “-c” option, and then the file’s name, “textfile.txt”.

The output of this command shows the repetition of lines by showing the number as their prefix. For example, the “Alex likes coding with various technologies” line appears twice in the file. So, the number “2” is shown in front of it. Similarly, the “Working with scripting languages is my interest” line appears twice in the file as well. The remaining lines are only used once, and “1” is shown as the prefix.

Using the “-D” Flag to Print the Repeated Lines

In this part, the repeating lines in the file are displayed using the “-d” option of the uniq command. For this, we use the command in which we first enter the “uniq” word, then the “-d” option, and then the “textfile.txt” filename.

Only the lines that repeat themselves are presented in the output that follows the execution of this command. The other lines, which are only used once, are not shown.

Using the Uniq Command’s -U Flag

The “-u” option of the uniq command is now used to print the file’s unique content straightforwardly using this option. It displays the lines or content of the file in the output that is only used once. The repeated text in the file is not displayed in the output; it is deleted. Only the “-u” flag is used in place of the following option. Otherwise, the command is the same as the one that is used in the earlier example.

When we perform this command, the output is as follows. The repeated lines of the files are removed, and the file’s unique content is revealed. This output includes the two lines that are shown in the following demonstration; both of these lines are used just once in the file, which is why they appear.

Conclusion

The Linux Uniq command which is used to eliminate the duplicate content from files and only displays it once on the output using the “uniq” keyword with the given text file is covered in this article. In the given examples, we used the uniq command flags which are “-c”, “-u”, and “-d”. These flags allowed us to carry out the various operations with the uniq command. In this article, we repeatedly demonstrated that a line is repeated by prefixing the line with a number. Additionally, we examined how to utilize these markers to print only the repeated lines and unique content.