05 Jan 2012 @ 8:00 PM 

Hello all,

The last post before a long week-end (I will not be able to post tomorrow, and on Monday and Tuesday. I’ll see you on wednesday.

Today, the topic is “The difference between find -exec and find | xargs”

The situation

You have to look into a specific directory and subdirectories for all the files named “*log” because you are quite sure one of them contains the word “error”.

You have two ways to do this :

1 – using -exec

This is a parameter to the find commands that allows you to do an extra command on the found files. The syntax is as follow :

-exec <command> {} \;

<command> replaces your command

{} will be replaced by the name of the found file.

\; terminates the command

i.e. you type :

find . -name \*log -exec grep -i error {} \;

and it will return all the lines containing error, regardless of the case.

2 – using xargs.

xargs is a command that allows the piped data to be passed as parameter to another command. The syntax is as follow :

| xargs <command>

xargs simply puts at the end of the command the piped data. i.e. you type :

find . -name \*log | xargs grep -i

3 – Using a while loop.

Yes, you can do like that, but it is not the topic of this discussion.


The difference

What is the main difference between the two?

-exec is going to take each file, and execute the command with it. Using this, you will get a list of all the lines, but not the name of the file, as grep assumes you know which file he talks about, as you have passed the name as parameter !

A sub process will be spawn for each file to be checked.

xargs, on the other end, is going to pass the complete list of files to grep. The name of the files will be shown in the result list.

The separator for the parameters will be the space, and this is OK as long as there is no space in the names of the files (but who puts spaces in the names of the files ? Ah, ok, users…).

In fact, the fail proof  way to deal with this specific request is to make the while loop.


The conclusion

Both ways can be useful, depending of the request and the situation. It is important that you understand the different ways those tools work, so that you can choose which one is the best for your usage.

Thank you for reading, and see you next wednesday ! (told you, long week-end for me 🙂 )


Posted By: Dimi
Last Edit: 02 Jan 2012 @ 05:17 PM

EmailPermalinkComments (0)
Tags: , , , ,
Categories: Bash, basics, Snippets
 22 Dec 2011 @ 8:00 PM 

As you know, Unix has different ways of chaining commands.

The most well-known way to chain commands in Unix is the pipe (|).

Using a pipe, the output (stdout) of the first application is sent as input (stdin) of the second command, and it is done like this :

echo “hello world !” | sed -e ‘s/h/H/’

(which is, by the way, a difficult way to do something simple)

If this is something new for you, you should write it down immediately : it is bound to come in any interview you will do if you have “unix”,”linux” or something equivalent in your resume…

File Descriptors Reminder (Skip it if you understand 2>&1)

A File descriptors are the way Unix is communicating with you. The main ones (the default ones) are 1 for stdout and 2 for stderr.

If you do the following :

find / -type f

You will get a lot of error messages (if you are not root. If you are root, you should change to your own user immediately, and respectfully tell your System Administrator that you have done something bad. [And, by the way, you should always talk respectfully to your System Administrator])

The normal messages will be sent to 1>, the errors will be sent to 2>, which means that you see both on screen, because both are redirected to your screen by default. If you do this :

find / -type f >/tmp/find.out 2>/tmp/find.err

then the errors will be sent to find.err, and the stdout will be sent to find.out.

Now, how would you send both outputs to the same file ? One way is this :

find / -type f >>/tmp/find.all 2>>/tmp/find.all

But it’s long, and we are lazy. The other way to do is this :

find / -type f >/tmp/find.all 2>&1

which means “Send stdout to find.all and stderr to stdout”. &1 represent stdout. The following would work as well :

find / -type f 2>/tmp/find.all >&2

OK, let’s go back to pipe now.

Pipes and file descriptors

The pipe only passes stdout (1>) which means that whatever you have in the stderr (2>) is not redirected.

Let’s imagine you do a find, then you pipe it to grep “toto”, case insensitive because your find is not able to do it :

find . -type f | grep -i toto

If you do not have the rights on certain files or directories, some error messages will pour, and you think you can get rid of them by doing :

find . -type f | grep -i toto 2>/dev/null

Well, no luck. Error messages are still there, as the stderr is not passed through the pipe… The command that gets rid of the error messages generated by find is :

find . -type f 2>/dev/null | grep -i toto

Pipes and return code

The return code of a pipe is always the last command to be on the line of pipe, none of the other will return something to the next line (it is returned to the next piped command, if you want to check it, but is is pretty useless).

This was another one of the basics, I hope you enjoyed it !


Posted By: Dimi
Last Edit: 21 Dec 2011 @ 09:33 AM

EmailPermalinkComments (0)

 Last 50 Posts
Change Theme...
  • Users » 67
  • Posts/Pages » 25
  • Comments » 4
Change Theme...
  • VoidVoid « Default
  • LifeLife
  • EarthEarth
  • WindWind
  • WaterWater
  • FireFire
  • LightLight