working with files in Linux

Sometimes while doing hacking , we come across one problem where we have two tools to get something for example getting subdomains from sublist3r and subfinder . we get some different subdomains in both the files. Now , we want to merge it and get unique subdomain.

In this tutorial we will learn 

• how to merge to files in linux

• How to sort the content of file

• How to remove duplicate lines from a file.

How to merge two files in linux ?

There are two ways to to merge to files in linux . 

Suppose we have files aayush1.txt and aayush2.txt .

In aayush1.txt the content is 

In aayush2.txt the content is 

To merge these two files we use the command : 

cat file1.txt file2.txt > file3.txt

Now applying the above command to our aayush1.txt and aayush2.txt 

Now after merging files and writing content into aayush3.txt. 

Lets see aayush3.txt 

Same work can be done with this command : 

cat  file1.txt file2.txt | > file3.txt

In Linux, the operator ` |`  is called a pipe.

How to sort the content of merged file alphabetically ?

Lets say we have file sorting.txt. 

The content of sort is : 

We want to sort the content of this file. 

Command used to sort the content of this file is : 

cat file.txt | sort | > file.txt

The above command says that we read a file and sort the content then overwrite the content into the same file.

Now to merge the file with content in sorted form , we use the command : 

cat file1.txt file2.txt | sort | > file3.txt

How to remove duplicate content from file ? 

Suppose we have a file duplicate.txt. 

The content of duplicate is : 

To remove the duplicate content is same as getting the unique content. This is the logic used in linux to remove duplicate content.

The command is : 

cat  file1.txt | uniq | > file1.txt

Applying  this command ,

Now we get , 

If you think that why `varun` is duplicated here, actually it is not duplicate. One is ‘varun<space>’ and another is `varun` only, which is not visible here.

Finally now to get the merged file with sorted and unique content we will use the command : 

cat file1.txt file2.txt | sort | uniq > merged.txt

Applying this command , you will get sorted and unique content from two files.