29 Dec 2011 @ 8:42 PM 

Today, something a bit more complicated. This is a fun way to use a trap function we have seen yesterday .

The problem

You receive a new requirement from your customer (I did receive it a couple of times). You need to develop a program that will :

– Not consume CPU or disk until it is necessary.

– Read the content of a directory, then process each file in a directory A (move it to another directory B ) one by one. Not two files at the same time.

– Once the file has been “taken” by another process (no file in directory B), the next file can be published.

– You can’t have the system polling every minute or so, because the files need to be copied as fast as possible.

– The process that provides the files can start another process, but can’t wait more that a millisecond before giving the hand back.

Good luck…

The Solution

Well, at least the one I found, was to have a main process going to sleep, then being awoken by another one (triggered by the “file-provider”), and starts polling the directory A until it is empty. Then it goes back to sleep.

How ? well, like this :

First, we need to trap the “KILLUSR” signal :

trap “wakeup” 30

Then, we need to create a script, on the fly, that will be called to signal the main script that there is something to process (the “trigger” process)

echo “#!/bin/ksh” >$trigger
echo “#Created by $0 on “`date` >>$trigger
echo ‘if [ “$1” == STOP ]’>>$trigger
echo “then” >>$trigger
echo “kill $$” >>$trigger
echo “else” >>$trigger
echo “kill -30 $$” >>$trigger
echo “fi” >>$trigger
echo “#End of the script” >>$trigger
chmod 755 $trigger

Next step, we create the function that will be executed in case of a trap.


wakeup () {
trap ‘echo “\c”‘ 30
# This is basically doing “noop”, to avoid being killed.
echo “Awoken at “`date` >>$0.log
while [ -f ${_From}/* ]
while [ -f ${_To}/* ]
sleep 1
File2Move=`ls -tr ${_From}| head -1`
mv ${_From}/${File2Move} ${_To}
echo “${_From} is empty” >>$0.log

And, last step, we need the code that will be executed while waiting :

while true
sleep 1800&

When the script is waiting, the main part is just looping on sleep. Them, once it receives a trap, the function that does the copy is executed.

Afterwards, we take the first file sent, move it over, then wait until the destination directory is empty, and look for the next file.

If you have another idea on how this process could be done, feel free to comment !

Thank you for reading, and I hope to see you on tuesday !

Posted By: Dimi
Last Edit: 30 Dec 2011 @ 03:15 PM

EmailPermalinkComments (0)
Tags: , ,
Categories: Bash, Signals, Snippets
 28 Dec 2011 @ 8:00 PM 


A very short one today, but it is usually not very well known.

As you know, Unix uses signals to communicate with processes. A process receives a signal, then takes an action. If no action is defined, the default action for most signals is to kill the process… which is maybe something you want to avoid.

The problem.

You have written the best script in the world, that does… let’s say… make ice-cream (we can dream).

The problem is that if someone kills your script (i.e. kill -15, SIGTEM, which is the default one), you process will simply stop, and throw ice-cream everywhere on the walls. (Ice-cream is here an image for temporary files).

How can you catch this message and close the ice-cream tap and clean-up a bit ?

The Solution.

Use trap.

At the beginning of your script, you can add the following line (after the shebang) :

trap 15 cleanup

This will trap the default SIGTERM, then start the procedure “cleanup” ( you need to define the “cleanup” procedure somewhere).

You can define a function that will do the cleanup for you, maybe add a message in the log, kill sub-processes, send mail, whatever is needed.

The only thing, is that you need to fail proof your function, in order to be sure that your process is not going in an infinite loop, because the way to kill it afterwards is to use another kill.

Note also that not all the signals might be trapped. Some of them, like the signal 9 (SIGKILL) is not trappable in a script.

Thank you for reading, and if I find some time, I’ll talk tomorrow about an inventive way to use the trap function.

Posted By: Dimi
Last Edit: 27 Dec 2011 @ 08:44 PM

EmailPermalinkComments (0)
Tags: , , ,
Categories: Bash, Signals, Snippets
 19 Dec 2011 @ 8:00 PM 

Background info.

As you know (and if you don’t, just say you do), when you quit a terminal, all the processes that have been started receive a signal type 1, SIGHUP, for SIGnal HangUP.

Usually, the processes that receive this signal, if they are not designed to trap it, will simply exit. This is why, at school, your teacher told you to start the background scripts with nohup, to protect the process.This is a well-known procedure and everybody uses it.

Your code should look like this :

nohup ./myGreatScript.sh >>./logs/mygreatlog 2>&1 &

The issue.

The issue we had was that some of the scripts and processes (Java and C)  we started on one of the new servers were crashing after we quit the session. The first reflex of the support guy was to start the scripts with nohup, which is correct but not very convenient.

The problem, you see, is that I have been using theses scripts for, like 10 years, without experiencing this behavior once. I assumed the main script contained this nohup command, and that, for some reason, someone did remove it… It was not the case, the nohup has never been into this script.

How did it work ?

This was a legitimate question… The script worked for ten years on different machines, but on this one, without using nohup.

I went then through the whole script, line by line, to try to understand what was happening. My first idea was that the processes were started using tee, which can be done to protect it.

Tee can be used to protect a service from a SIGINT process (using -i option), but not a SIGHUP

The solution ?

In fact, the issue was nowhere near the script, but lied in the default shell used by the user on each machine.

On all the machine where it worked, it was either ksh or bash. On the one where it crashed (all the machine were Solaris), it was sh.

Both ksh and bash seem to react differently on the exit of a session, ksh and bash do not send a signup signal to all the child processes unless the huponexit shell option is set. To check the option status, at least in bash, you can use the following command :


and look for the line that says huponexit. To change the value, you can use the -u or -s options.

The solution here was to change the default shell from /bin/sh to /usr/bin/bash.

Please note that the shell options are not passed on to the child processes for some reason. I tried to get an explanation on this, but could’t. If anyone has an idea, please make yourself heard !


Thank you.

Posted By: Dimi
Last Edit: 19 Dec 2011 @ 10:53 PM

EmailPermalinkComments (0)
Tags: , , , , , ,
Categories: Bash, Signals

 Last 50 Posts
Change Theme...
  • Users » 67
  • Posts/Pages » 25
  • Comments » 4
Change Theme...
  • VoidVoid « Default
  • LifeLife
  • EarthEarth
  • WindWind
  • WaterWater
  • FireFire
  • LightLight