Mass file processing in Linux.
November 14th, 2007Whether its mass deleting, moving, or renaming moving files within Linux can quickly become a huge headache. There are many clever, fast, elegant, and even ugly examples of how to do this what seems to be a basic task. Here I want regurgitate a few things I’ve found after spending hours doing these sorts of tasks.
- Deleting files:
- Moving files
- Renaming files
This is an easy one, even in the case that the filenames have spaces in them:
This simply utilizes find to locate any file matching my criteria (see ‘man find’ for more info) and pipes its output to xargs. The print0 as well as -0 options in find and xargs, respectively, is there to handle the case in which spaces are in the filename and/or directory. The flag –replace={} simply places the input into xargs in any location where {} appears in the command being called by xargs.
Similar to above
One problem with this particular usage is that you cannot do any further processing on a file, like renaming it AND moving it, unless you have an executable on your system which you can call using xargs; As a corollary you can also not use parameter expansion in BASH to do anything to the input.
This is a very difficult problem to solve. Because there are so many different ways in which you might wish to rename a file. First, perhaps you just want to replace the suffix.
Maybe some other text anywhere in the filename.
Now the degenerate case of all these moving procedures is something like this, what if you want to recursively move files within folders which may contain spaces and the files contain spaces, and you wish to do some processing on the name AND you want to leave the file in place. This one took me a while. In my situation I actually had a tool which compressed a ton of Powerpoint files in Windows for me, however the program refused to replace the files in place instead it required to add a prefix or suffix. Using Cygwin I thought I could easily go back and just remove the prefix. Wrong. Given all the constraints of xargs, a for loop and everything else I was about ready to give up. Then I found it.
Sweet Dobbs it works!!! This solution is so elegant plus it actually gives me back all the power of parameter expansions. Something which any xargs method does not.
July 13th, 2008 at 5:14 am
Thanks for the great info. I hope you’ll follow this with some more great content.
July 20th, 2008 at 9:35 am
I came across this blog the other day and you got some great info here - thanks.