03 Jan 2012 @ 8:00 PM 

Today, we’ll see something interesting about the scope of variable in bash, and maybe more a bug than something else.

Before going further, I want to say that I have never had problem with the scope of variables in Bash, but this specific case is very strange.

The situation

Let’s say you have a file “tempo1”, containing three lines :

COMP 2 + 3 + 4
COMP 4 + 5 + 6

You want to compute all the lines that contain “COMP” and provide the total result.

You then write the following script (usually in one line) :

typeset -i i j
cat tempo1 | while read line; do if echo $line | grep COMP >/dev/null 2>&1;  then   i=`expr \`echo $line | cut -f2- -d’ ‘\“;   echo $i; let j=j+i;fi; done

(if the script is unclear, I’ll detail later on)

You assume that, at the end, “j” will have the sum of all, and “i” is going to be the last processed number. Well, you’re wrong.

The problem

At least if you use bash. There seems to be an issue with the context copy for a piped while. i.e. both your variable will be defined, but empty.

$ echo $0 i is $i j is $j
-bash i is j is

Now, let’s re-run the whole lot in ksh :

echo $0 i is $i j is $i
ksh i is 15 j is 15

The reason

Well, after looking on the web, it seems that I am not the only one puzzled. This has been reported as a bug, and we’ll see if it changes in the future.

The solution

Change your code, to avoid the piped while :

while read line; do if echo $line | grep COMP >/dev/null 2>&1;  then   i=`expr \`echo $line | cut -f2- -d’ ‘\“;   echo $i; let j=j+i;fi; done <tempo1

and the problem disappears. Or use ksh for this specific purpose.

Thanks for reading this, and see you tomorrow ! (Should you require more explanation about the script, feel free to read on.)


Explanation of the script

Let’s first explode the script on multiple lines :

typeset -i ij
cat tempo1 | while read line
if echo $line | grep COMP >/dev/null 2>&1;  then
i=`expr \`echo $line | cut -f2- -d’ ‘\“
echo $i
let j=j+i

So, the first line :

typeset -i i j

is there to define the variables i and j as integer (-i) (equivalent to int i,j; in c)

The next line is parsing the file, and looping each line between the “do” and “done”. The line is put in the variable “line”

If you had written

cat tempo1 | while read JamesTKirk

then the variable containing the line would have been JamesTKirk, which is a bit long to type and also a bit of an overkill to have one of the best Starfleet Captain as your variable.

Next line speaks by itself :

if echo $line | grep COMP >/dev/null 2>&1;  then

means “if the result of the command “echo $line | grep COMP” is 0, then execute the following” and of course, this will only be the case if the line in question contains the word “COMP” somewhere. We could have done maybe with “grep -e ‘^COMP'”, or with the funny test we’ll see tomorrow, but let’s not split the hair.

i=`expr \`echo $line | cut -f2- -d’ ‘\“

expr allows you to do basic arithmetical calculation. This line means “Take everything after the first field on the line, space being the separator, then compute it and store it in the variable “i”“.

let j=j+i

let is another way to do arithmetical operations, using integer  variables.

and voila, done.

Should you require more assistance, feel free to post in the comments or to man the command you want to know about.

Thank you for reading, and see you tomorrow for a special test operator.

Posted By: Dimi
Last Edit: 02 Jan 2012 @ 04:37 PM

EmailPermalinkComments (0)
Tags: , , ,
Categories: Bash, Snippets
 19 Dec 2011 @ 8:00 PM 

Background info.

As you know (and if you don’t, just say you do), when you quit a terminal, all the processes that have been started receive a signal type 1, SIGHUP, for SIGnal HangUP.

Usually, the processes that receive this signal, if they are not designed to trap it, will simply exit. This is why, at school, your teacher told you to start the background scripts with nohup, to protect the process.This is a well-known procedure and everybody uses it.

Your code should look like this :

nohup ./myGreatScript.sh >>./logs/mygreatlog 2>&1 &

The issue.

The issue we had was that some of the scripts and processes (Java and C)  we started on one of the new servers were crashing after we quit the session. The first reflex of the support guy was to start the scripts with nohup, which is correct but not very convenient.

The problem, you see, is that I have been using theses scripts for, like 10 years, without experiencing this behavior once. I assumed the main script contained this nohup command, and that, for some reason, someone did remove it… It was not the case, the nohup has never been into this script.

How did it work ?

This was a legitimate question… The script worked for ten years on different machines, but on this one, without using nohup.

I went then through the whole script, line by line, to try to understand what was happening. My first idea was that the processes were started using tee, which can be done to protect it.

Tee can be used to protect a service from a SIGINT process (using -i option), but not a SIGHUP

The solution ?

In fact, the issue was nowhere near the script, but lied in the default shell used by the user on each machine.

On all the machine where it worked, it was either ksh or bash. On the one where it crashed (all the machine were Solaris), it was sh.

Both ksh and bash seem to react differently on the exit of a session, ksh and bash do not send a signup signal to all the child processes unless the huponexit shell option is set. To check the option status, at least in bash, you can use the following command :


and look for the line that says huponexit. To change the value, you can use the -u or -s options.

The solution here was to change the default shell from /bin/sh to /usr/bin/bash.

Please note that the shell options are not passed on to the child processes for some reason. I tried to get an explanation on this, but could’t. If anyone has an idea, please make yourself heard !


Thank you.

Posted By: Dimi
Last Edit: 19 Dec 2011 @ 10:53 PM

EmailPermalinkComments (0)
Tags: , , , , , ,
Categories: Bash, Signals

 Last 50 Posts
Change Theme...
  • Users » 66
  • Posts/Pages » 25
  • Comments » 4
Change Theme...
  • VoidVoid « Default
  • LifeLife
  • EarthEarth
  • WindWind
  • WaterWater
  • FireFire
  • LightLight