• Home
  • Log
  • Projects
  • Gallery
  • About
  • Contact

Mass file processing in Linux.

November 14th, 2007

Whether its mass deleting, moving, or renaming moving files within Linux can quickly become a huge headache. There are many clever, fast, elegant, and even ugly examples of how to do this what seems to be a basic task. Here I want regurgitate a few things I’ve found after spending hours doing these sorts of tasks.

  • Deleting files:
  • This is an easy one, even in the case that the filenames have spaces in them:

    find . -name ‘prefix.suffix’ -print0 | xargs -0 –replace={} rm -rf {}

    This simply utilizes find to locate any file matching my criteria (see ‘man find’ for more info) and pipes its output to xargs. The print0 as well as -0 options in find and xargs, respectively, is there to handle the case in which spaces are in the filename and/or directory. The flag –replace={} simply places the input into xargs in any location where {} appears in the command being called by xargs.

  • Moving files
  • Similar to above

    find . -name ‘prefix.suffix’ -print0 | xargs -0 –replace={} mv {} /new/location/

    One problem with this particular usage is that you cannot do any further processing on a file, like renaming it AND moving it, unless you have an executable on your system which you can call using xargs; As a corollary you can also not use parameter expansion in BASH to do anything to the input.

  • Renaming files
  • This is a very difficult problem to solve. Because there are so many different ways in which you might wish to rename a file. First, perhaps you just want to replace the suffix.

    for file in *.jpeg; do mv $file ${file%.jpeg}.jpg ; done

    Maybe some other text anywhere in the filename.

    for file in *; do mv $file ${file/foo/bar} ; done

    Now the degenerate case of all these moving procedures is something like this, what if you want to recursively move files within folders which may contain spaces and the files contain spaces, and you wish to do some processing on the name AND you want to leave the file in place. This one took me a while. In my situation I actually had a tool which compressed a ton of Powerpoint files in Windows for me, however the program refused to replace the files in place instead it required to add a prefix or suffix. Using Cygwin I thought I could easily go back and just remove the prefix. Wrong. Given all the constraints of xargs, a for loop and everything else I was about ready to give up. Then I found it.

    find . -name ‘prefix.suffix’ -print0 | while read file; do mv “$file” “${file/foo/bar/}; done”

    Sweet Dobbs it works!!! This solution is so elegant plus it actually gives me back all the power of parameter expansions. Something which any xargs method does not.

Posted in Linux | 2 Comments »


Acer Ferrari 4006 and Edgy Eft Wireless

November 14th, 2006

Recently I decided to upgrade my laptop from Dapper Drake to Edgy Eft (Ubuntu) out of boredom. As usual there were several things which ended up broken. My wireless was the most frustrating of these.

After scouring forums and following every help guide I could find I still was stuck. Compiling ndiswrapper from source wasn’t helping, it didn’t matter what version of the bcmwl5 driver I used, it just didn’t work. Now to clarify the specific wireless card I have in the laptop is thus:

‘lspci|grep Broadcom’

06:02.0 Network controller: Broadcom Corporation BCM4318 [AirForce One 54g] 802.11g Wireless LAN Controller (rev 02)

Which I’ve heard several complaints about. In the end I did manage to get it working and for the sake of brevity I’ll simply post the solution.

If you are running the 2.6.17.10 kernel (as I was) you might be stuck, you may even have a problem if you are running any of the sub-versions of the 2.6.17 kernel. So I went out and compiled the 2.6.18 kernel following this guide How to compile 2.6.18

After compiling the kernel you then need to go out and download the source code for ndiswrapper, I used 1.28. After compiling and installing use the windows driver as usual and you should be up and running again.

Posted in Linux | No Comments »


Compile GD with JPEG Support

November 3rd, 2006

In the wee hours of the morning as I work on compiling random bits and pieces here and there I stumbled across a problem when attempting to compile gd and have it recognize my jpeg-6b libraries. I thought it worth noting that gd is specifically looking for ‘libjpeg.so’, which on my particular system doesn’t exist, instead I have a copy of ‘libjpeg.so.62′ which I simply created a softlink to the file and ran ‘configure’ again.

ln -s /usr/lib/libjpeg.so.62 /usr/lib/libjpeg.so

Don’t forget to set your CPPFLAGS environment variable to include your jpeg-6b header files!!

Once again all is right with the world.

Posted in Programming | No Comments »


Firefox 2.0 on Ubuntu AMD64

October 24th, 2006

Well now that I have all the tag-lines out of the way. I figured I would just share my simple help for installing the new Firefox 2.0 (released today) for Ubuntu … well you already know the rest.

To begin at the beginning. You need to create your .mozconfig file. Your best bet is just put this in your home directory. You can scrutinize the following, or just accept it as is, but this is my .mozconfig file.

. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt-static
ac_cv_visibility_pragma=no
ac_add_options –enable-optimize
ac_add_options –disable-debug
ac_add_options –enable-static
ac_add_options –disable-shared
ac_add_options –disable-tests
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options –enable-application=browser
ac_add_options –enable-optimize=”-march=k8 -O2 -funroll-loops -ffast-math -fschedule-insns2 -fexpensive-optimizations”

The next item of business is of course to go download a copy of the new Firefox source (yes it’s early enough that we are going to have to compile it, if you haven’t already guessed that).

I don’t want to direct anyone to any particular FTP as apparently there has been some concern about this lately about bandwidth. Simply find a copy for yourself and extract it into your favorite build directory.

You are only going to need to write two more commands after you’ve finished extracting the files (from inside the mozilla directory in your extraction folder).

make -f client.mk build
and
sudo make -f client.mk install

Enjoy!

Posted in Programming, Misc | No Comments »


« Previous Entries
  • Pages

    • About
  • Archives

    • November 2007
    • November 2006
    • October 2006
    • August 2006
    • July 2006
    • June 2006
    • April 2006
    • March 2006
    • February 2006
    • January 2006
    • December 2005
    • October 2005
    • May 2005
  • Categories

    • Uncategorized (1)
    • Music (1)
    • Graphics (2)
    • Programming (7)
    • Misc (15)
    • Travel (3)
    • Linux (2)
  • Blogroll

    • Bill Stevenson
    • Solano Systems
  • Links

    • Deviant Art
    • Game Programming Wiki
    • Hyperphysics
    • Project Gutenberg
  • Meta

    • Register
    • Login
    • Valid XHTML
    • XFN
    • WordPress
 Use OpenOffice.org

Powered by WordPress Entries (RSS)
and Comments (RSS). 18 queries. 0.318 seconds.