Ubuntu: Putting all lines that match specific pattern at the end of the file



Question:

I have number of .txt files that look like this:

XXXXXXXXXXXXXX  NNNNNNNNNNNNNN  NNNNNNNNNNNNNN  NNNNNNNNNNNNN  NNNNNNNNNNNN  NNNNNNNNNNN  XXXXXXXXXXXXXX  XXXXXXXXXXXXX  XXXXXXXXXXXXX  XXXXXXXXXXXX  NNNNNNNNNNNNN  NNNNNNNNNNNN  NNNNNNNNNNNNNN  NNNNNNNNNNNNNN  

Is it possible to give a command in bash that all the lines that start with N should end up at the end of the file. So I am expecting something like this:

XXXXXXXXXXXXXXXXXX  XXXXXXXXXXXXXXXXX  XXXXXXXXXXXXXXXX  XXXXXXXXXXXXXXXX  XXXXXXXXXXXXXXXXX  XXXXXXXXXXXXXXXXXX  XXXXXXXXXXXXXXXXXXX  NNNNNNNNNNNNNNNN  NNNNNNNNNNNNNNNNNN  NNNNNNNNNNNNNNNNNNNN  NNNNNNNNNNNNNNNNNNN  NNNNNNNNNNNNNNNNNN  

..and to do it on all files from one folder.


Solution:1

You can use this:

{      grep -v '^N' file.txt      grep '^N' file.txt  } > file.txt.temp  mv file.txt.temp file.txt  

This works by matching all lines that do not start with 'N' (grep -v '^N') then matching all lines that do start with 'N' (grep '^N'), outputting into a temporary file, then replacing the original with the temp file.

Then if you wanted to use this on all *.txt files in a directory:

for f in *.txt; do      {          grep -v '^N' "$f"          grep '^N' "$f"      } > "$f".temp      mv "$f".temp "$f"  done  


Solution:2

Here are a couple of options using awk

First, the KISS version - using a couple of temporary files a and b:

for f in *.txt; do     awk '/^N/ {print > "b"; next} {print > "a"}' "$f"    cat a b > "$f"  done  

Just because. . .

What you really want to do is an in-place sort with a custom sort order (lines starting with N after everything else, then original order), for example in GNU awk:

function mycmp(ia, a, ib, b) {    x = substr(a,1,1) == "N"    y = substr(b,1,1) == "N"    if (x && !y) return 1    else if (!x && y) return -1    else return ia - ib  }    {a[FNR] = $0}    ENDFILE {    for (i in a) print a[i]  }  

Newer versions of gawk provide an in-place extension - borrowing the reference implementation from 16.7.4 Enabling In-Place File Editing we can put it all together as a gawk script, Nsort.awk say:

#!/usr/bin/gawk -f    @load "inplace"    function mycmp(ia, a, ib, b) {    x = substr(a,1,1) == "N"    y = substr(b,1,1) == "N"    if (x && !y) return 1    else if (!x && y) return -1    else return ia - ib  }    BEGIN {    inplace = 1    INPLACE_SUFFIX = ".bak"    PROCINFO["sorted_in"] = "mycmp"  }    BEGINFILE {      if (_inplace_filename != "")          inplace_end(_inplace_filename, INPLACE_SUFFIX)      if (inplace)          inplace_begin(_inplace_filename = FILENAME, INPLACE_SUFFIX)      else          _inplace_filename = ""  }    {a[FNR] = $0}    ENDFILE {    for (i in a) print a[i]  }    END {      if (_inplace_filename != "")          inplace_end(_inplace_filename, INPLACE_SUFFIX)  }  

Make it executable with chmod +x Nsort.awk and run it as

./Nsort.awk *.txt  


Solution:3

GNU awk (version 4.1.0 and later)

awk -i inplace '/^N/{ printlast = printlast $0 RS ; next } { print $0 } ENDFILE { printf "%s",printlast ; printlast="" }' file1 file2 file3  
  1. Use -i inplace to replace the original file with the reordered file. CAUTION: This will overwrite files that are set read-only.
  2. If a line starts with N, append the line to the buffer printlast.
  3. Otherwise, print the line.
  4. At the end of the file, print the buffer printlast, then reset it for the next file.
  5. Specify as many input files as you want (eg, a.txt b.txt c.txt or *.txt).

Replace -i inplace with -i inplace -v INPLACE_SUFFIX=".bak" to keep a copy of each original file. In this example, the backup copy filenames will all end in .bak.

Other versions of awk

If your version of awk can't edit files in place, and if you don't want to create temporary files, then you can pipe the output of awk through sponge (from the moreutils package):

ls -1 *.txt | \  while read file  do      chmod +w "${file}"      awk '/^N/{ printlast = printlast $0 RS ; next } { print $0 } ENDFILE { printf "%s",printlast }' "${file}" | \      sponge "${file}"  done  

You need to make the input files writable -- chmod +w "${file}" -- for sponge to modify them.


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »