Month: June 2007

Spaces within

“Unix doesn’t works properly with filenames with spaces”. This assertion from a coworker of mine prompted my harsh reply: “Unix works perfectly with spaces, it’s just programmer sloppiness that prevents it from properly handling blanks”. I was true (as always, I would humbly add) but my pal wasn’t completely wrong, at least when talking about the specific subpart of Unix named shell. Although far superior than MS-DOS batch shell (that’s about the same command line you find in Windows) I bet it originated in more or less the same way – a core with functionalities clustered over during time in response of new needs or new opportunities.
Unix shell (be it bash, ksh, zsh or fish) is nowadays a powerful programming tool allowing the programmer to craft rather complex artifacts. This scope is for sure much broader than the one envisioned by first developers, and this turns out in multiple ways to do the same thing, different ways to do similar things and cryptic ways to do simple things.
The Unix command line conception dates nearly 40 years back in time! Things were pretty different, but I won’t annoy you with details, just leave your imagination wild… likely it was even worse. Fish is a recent attempt to overcome most of the shell problems, but it is not widespread as bash could be. As put by a professional some time ago: emacs may have tons of neat features, but you are SURE you’ll always find vi on any Unix, while you are not certain you’ll have emacs. So better invest your learning time on vi.
Well, back to shell. What’s wrong with blanks? The main problem is that a space is a valid character in a file name and, at the same time, it is a separator for command line arguments. Back to when every single byte could make the difference, it seemed the right thing to do to have optional quotes if the filename doesn’t contain any space. So you can write:

$ ls foo/

To get the listing of directory foo, but you have to write:

$ ls "bar baz/"

if you want the listing of directory “bar baz” (or you could escape the space with a backslash). This could be boring on interactive shells, but is usually overcome by the auto-completion feature (type ‘ba’ then tab and the line gets completed with the available options, in this case: bar baz).
From boring it turns in the range from annoying to irritating in shell scripts, where variables are not real variables like those you are used in high level languages, but just convoluted macros. For example:

a="bar baz"
ls $a

is processed and interpreted as:

ls bar baz

As you see the quotes disappears because they are processed by the assignment to put bar+space+baz in the ‘a’ variable. Once ‘$a’ is expanded, quotes are just forgotten memories. In order to write proper shell scripts you have to do something like:

a="bar baz"
ls "$a"

Of course this is error prone, because not only the syntax is valid, not only the script is likely to work perfectly in the simple test case the programmer uses to test the script, but also it is likely to work fine most of the times. After all the space character is used only by those Windows naïve users that aren’t aware of the blanks-hating-device they keep hidden under their desks.
Well, I proud myself of writing space-safe shell scripts, at least until I tried to write a script to find duplicated files on a filesystem.
The goal is simple, after many virtual relocations, multiple pet-projects and home-works, I have many files scattered around with the same content. It is not a matter of space saving, rather it is a question of order. Avoid redundant information or make sure that it is really the same stuff.
My design was to have a command similar to ‘find’, something that accepts any number of directories or files on the command line, such as:

$ find_dupes dir1/ dir2/ file …

Shell has two ways for operating this pattern – use the shift command or use one of the special variables $@ and $*.
The first way is useful if you want the shell to process one argument at time, while the latter is handy when you want to relay the command line to a command. In my case I wanted to pass the entire command line to the ‘find’ command, something like:

$ find $@ -type f -exec md5sum {} ;

This line works fine until a filename with space is encountered. In this case, since variables are indeed macros, a single argument with space is expanded into two (or more) distinct arguments. And there is no way to work around the limitation, unless you read the manual J. In this area, the discoverability of bash is quite lacking. The man page states that $* expands in the sequence of arguments separated by the first character of the IFS environment variable. E.g. if IFS is set to dash (‘-‘) and the command line has the following arguments foo bar baz, then $* expands to foo-bar-baz.
Conversely $@ expands to a space separated sequence of arguments, but if you enclose it in quotes, then single arguments are expanded in quotes. E.g. $@ expands to foo bar baz, and “$@” expands to “foo” “bar” “baz”. Eventually this is the solution.
So, basically it is true that Unix has no problem whatsoever with spaces inside filenames, it is also true that shell programming can handle them as well and ultimately is up to programmer sloppiness if the batch script fails, but it has to be recognized that a great effort and investment is required to the programmer to climb out his sloppiness.

One more rant

Oh, I forgot yesterday, maybe it wasn’t yet clear in my mind and I needed some more dig and headscraping on laws and funds to have it more vivid. Our government is trying to protect Italians from harming themselves by not subscribing an integrative pension fund. This is for sure a laudable intention – I warn you, if you want to survive the 3rd age, you have to put aside some money.
The problem is that this strongly contrast with the current fund schemes that offer several lines of investment with different aggressiveness and features. In other words the money that the government forces you to put aside to grant a wealthy old age are allowed to go into the slot machine of the financial trades and you could end with empty hands even emptier than if you had put money in the mattress. The law requires only the most conservative line to grant the sum you put into (that is 0% interest, you are nonetheless wasting money in taxes and losing power of purchasing because of the inflation). The law set the TFR interest rate as a goal, not as a requirement for the most conservative line. Nothing happens if the goal is not reached.
Even worse the law allows to chose more aggressive investment that could bring higher positive rates or dragging you into losing money, maybe helping fund managers interests but going against the noble goal to avoid social problems when the current working class will reach the retirement age.
There is an underlying problem of trust, lacking of trust. If the government wants Italians to trust this new pension form, then the new form must grant everything the old one granted. The law must grant at least the same rate and at least the chance to get all the money at retirement.
After all if everyone is so staunch about the TFR performing worse than the financial fund why don’t they grant the fund to perform at least as bad as the TFR?

A Bleak Future?

What awaits for us in the midst of the future? How current choices are going to impact on our future existance? Public pensions has been for a long time now a solid foundation for Italian families. Pensions allowed parents to keep their sons and daughter with them, helping them to have a roof and warm meals while occasionally earning 1000€ per month. Basically elderly pensions balanced precarious and occasional work, unemployment and dramatically high house prices.
There’s no way for a young couple to start a life together in any of the Northern Italy cities with such poor wages, with the ghost of sudden unemployment and such high apartment rents, not to talk about buying one.
(Well not that a young couple could start a life together if they live with their parents, neither… that could be part of the reasons for Italy has such a low natality rate).
Unfortunately the Italian pension system is lame.
Rather than implementing a system where everyone put aside a bit of money that are wisely invested and then given back as a life annuity after worker retirement, politicians and unions decided a long ago an inter-generation agreement. Under this agreement the current generation of workers pays the pensions of the current generation of retired. Brilliant? Maybe until the money set aside for pensions of the current work class is enough to pay for the current pension expense.
What is going wrong is that Italians make less and less babies and tend to survive (despite of the public health care) longer. So the agreement between generations is broken, the promise that we workers are going to receive the same treatment we are providing to our elders is void.
According to current law, when the time will come to retire I’ll have an annuity of about 40% of the average salary of the past 5 years. Not enough to survive unless I’m starting a extra bright career right now.
Although this trend is clear from about 15 years ago, only now drastic decisions have been taken. Workers money (the so called TFR a form a delayed salary awarded when leaving the job) have been confiscated leaving little or no choice to the former owners. Well, it isn’t completely true, you can leave everything as is, but you are on the verge of a cliff. The idea is to force most of the workers to move that money into complementary contributive pension funds. And when in a fund this money is no longer completely yours. You cannot move in another kind of investment or get them back to the former delayed salary form. Moreover you get no real warranty. If you chose for a conservative fund line you are granted that you will have back your money, no interest. Given the current inflation levels, that means that you could lose 2.5% of purchasing value of your money per year. The delayed salary is bound by law to produce an interest of 1,5% + 0,75 * inflation.
All this funds are privately held and what you get is an eased fiscal drag on the earnings, a bit more convenient than the TFR (19% against 23%).
The worst nightmare is not only having a ludicrous public pension, but losing every bit of the TFR money thanks to financial market volatility and irresponsible fund choices.
We have time up to the end of the month to decide what to do with our money, what is sure is that there will be lot of problems (the social kind) when the time will come for us to retire – precarious workers won’t have any pension at all, excessively conservative people won’t do any complementary form and will have just the TFR to add to the 40% of the last salaries, a number of funds will have problems (Cirio, Parmalat, Enron and 09/11 don’t ring any bell?) returning little or no money.
But there is a loop, the only way to provide a better future is to increase current wages so that everyone could save for retirement to achieve at least a decent life. Traditional manufacturing is in a difficult position with emerging country economies that allow for extremely low work costs. We have to do like the rest of Europe and follow the advanced services and technologies direction, we have to focus on industries where the cost of work is negligible respect to the cost of delivered good or service. And to do this we need both public investments in Research, in Universities, we need easier way to start business activities and to attract foreign investments in the advanced industries.
Are we hopeless?