Tuesday, November 27, 2012

Windows Command Line: Display & Redirect Output

Never thought MS-DOS could do this much, LOL


Display & Redirect Output

On this page I'll try to explain how redirection works.
To illustrate my story there are some examples you can try for yourself.
For an overview of redirection and piping, view my original redirection page.

Display text

To display a text on screen we have the ECHO command:
ECHO Hello world
This will show the following text on screen:
Hello world
When I say "on screen", I'm actually referring to the "DOS Prompt", "console" or "command window", or whatever other "alias" is used.


The output we see in this window may all look alike, but it can actually be the result of 3 different "streams" of text, 3 "processes" that each send their text to thee same window.
Those of you familiar with one of the Unix/Linux shells probably know what these streams are:
  • Standard Output
  • Standard Error
  • Console
Standard Output is the stream where all, well, standard output of commands is being sent to.
The ECHO command sends all its output to Standard Output.
Standard Error is the stream where many (but not all) commands send their error messages.
And some, not many, commands send their output to the screen bypassing Standard Output and Standard Error, they use the Console. By definition Console isn't a stream.
There is another stream, Standard Input: many commands accept input at their Standard Input instead of directly from the keyboard.
Probably the most familiar example is MORE:
where the MORE command accepts DIR's Standard Output at its own Standar Input, chops the stream in blocks of 25 lines (or whatever screen size you may use) and sends it to its own Standard Output.
(Since MORE's Standard Input is used by DIRMORE must catch its keyboard presses (the "Any Key") directly from the keyboard buffer instead of from Standard Input.)


You may be familiar with "redirection to NUL" to hide command output:
ECHO Hello world>NUL
will show nothing on screen.
That's because >NUL redirects all Standard Output to the NUL device, which does nothing but discard it.
Now try this (note the typo):
EHCO Hello world>NUL
The result may differ for different operating system versions, but in Windows XP I get the following error message:
'EHCO' is not recognized as an internal or external command, operable program or batch file.
This is a fine demonstration of only Standard Output being redirected to the NUL device, but Standard Error still being displayed.
Redirecting Standard Error in "true" MS-DOS (COMMAND.COM) isn't possible (actually it is, by using the CTTY command, but that would redirect all output including Console, and input, including keyboard).
In Windows NT 4 and later (CMD.EXE) and in OS/2 (also CMD.EXE) Standard Error can be redirected by using 2> instead of >
A short demonstration. Try this command:
ECHO Hello world 2>NUL
What you should get is:
Hello world
You see? The same result you got with ECHO Hello world without the redirection.
That's because we redirected the Standard Error stream to the NUL device, but the ECHO command sent its output to the Standard Output stream, which was not redirected.
Now make a typo again:
EHCO Hello world 2>NUL
What did you get? Nothing
That's because the error message was sent to the Standard Error stream, which was in turn redirected to the NUL device by 2>NUL
When we use > to redirect Standard Output, CMD.EXE interprets this as 1>, as can be seen by writing and running this one-line batch file "test.bat":
Now run test.bat in CMD.EXE and watch the result:

C:\>DIR  1>NUL

It looks like CMD.EXE uses 1 for Standard Output and 2 for Standard Error. We'll see how we can use this later.
Ok, now that we get the idea of this concept of "streams", let's play with it.
Copy the following code into Notepad and save it as "test.bat":
ECHO This text goes to Standard Output
ECHO This text goes to Standard Error 1>&2
ECHO This text goes to the Console>CON
Run test.bat in CMD.EXE, and this is what you'll get:
This text goes to Standard Output
This text goes to Standard Error
This text goes to the Console

Now let's see if we can separate the streams again.
test.bat > NUL
and you should see:
This text goes to Standard Error
This text goes to the Console

We redirected Standard Output to the NUL device, and what was left were Standard Error and Console.
Next, run:
test.bat 2> NUL
and you should see:
This text goes to Standard Output
This text goes to the Console

We redirected Standard Error to the NUL device, and what was left were Standard Output and Console.
Nothing new so far. But the next one is new:
test.bat > NUL 2>&1
and you should see:
This text goes to the Console

This time we redirected both Standard Output and Standard Error to the NUL device, and what was left was only Console.
It is said Console cannot be redirected, and I believe that's true. I can assure you I did try!
To get rid of screen output sent directly to the Console, either run the program in a separate window (using the START command), or clear the screen immediately afterwards (CLS).
In this case, we could also have used test.bat >NUL 2>NUL
This redirects Standard Output to the NUL device and Standard Error to the same NUL device.
With the NUL device that's no problem, but when redirecting to a file one of the redirections will lock the file for the other redirection.
What 2>&1 does, is merge Standard Error into the Standard Output stream, so Standard output and Standard Error will continue as a single stream.

Redirect "all" output to a single file:

test.bat > test.txt 2>&1
and you'll get this text on screen (we'll never get rid of this line on screen, as it is sent to the Console and cannot be redirected):
This text goes to the Console
You should also get a file named test.txt with the following content:
This text goes to Standard Output
This text goes to Standard Error
Note:The commands
test.bat  > test.txt 2>&1
test.bat 1> test.txt 2>&1
test.bat 2> test.txt 1>&2
all give identical results.

Redirect errors to a separate error log file:

test.bat > testlog.txt 2> testerrors.txt
and you'll get this text on screen (we'll never get rid of this line on screen, as it is sent to the Console and cannot be redirected):
This text goes to the Console
You should also get a file named testlog.txt with the following content:
This text goes to Standard Output
and another file named testerrors.txt with the following content:
This text goes to Standard Error
Nothing is impossible, not even redirecting the Console output.
Unfortunately, it can be done only in the old MS-DOS versions that came with a CTTY command.
The general idea was this:
ECHO Echo whatever you want, it won't be displayed on screen no matter what.
ECHO By the way, did I warn you that the keyboard doesn't work either?
ECHO I suppose that's why CTTY is no longer available on Windows systems.
ECHO The only way to get control over the computer again is a cold reboot,
ECHO or the following command:
A pause or prompt for input before the CTTY CON command meant one had to press the reset button!
Besides being used for redirection to the NUL device, with CTTY COM1 the control could be passed on to a terminal on serial port COM1.

Escaping Redirection (not to be interpreted as "Avoiding Redirection")

Redirection always uses the main or first command's streams:
START command > logfile
will redirect START's Standard Output to logfilenot command's!
The result will be an empty logfile.
A workaround that may look a bit intimidating is grouping the command line and escaping the redirection:
START CMD.EXE /C ^(command ^> logfile^)
What this does is turn the part between parentheses into a "literal" (uninterpreted) string that is passed to the command interpreter of the newly started process, which then in turn does interpret it.
So the interpretation of the parenthesis and redirection is delayed, or deferred.
Note:Be careful when using workarounds like these, they may be broken in future (or even past) Windows versions.
A safer way to redirect STARTed commands' output would be to create and run a "wrapper" batch file that handles the redirection.
The batch file would look like this:
command > logfile
and the command line would be:
START batchfile

Some "best practices" when using redirection in batch files:

  • Use >filename.txt 2>&1 to merge Standard Output and Standard Error and redirect them together to a single file.
    Make sure you place the redirection "commands" in this order.
  • Use >logfile.txt 2>errorlog.txt to redirect success and error messages to separate log files.
  • Use >CON to send text to the screen, no matter what, even if the batch file's output is redirected.
    This could be useful when prompting for input even if the batch file's output is being redirected to a file.
  • Use 1>&2 to send text to Standard Error.
    This can be useful for error messages.
  • It's ok to use spaces in redirection commands. Note however, that a space between an ECHO command and a > will be redirected too.
    DIR>filename.txt and DIR > filename.txt are identical, ECHO Hello world>filename.txt and ECHO Hello world > filename.txt are not, even though they are both valid.
    It is not ok to use spaces in >> or 2> or 2>&1 or 1>&2 (before or after is ok).
  • In Windows NT 4, early Windows 2000 versions, and OS/2 there used to be some ambiguity with ECHOed lines ending with a 1 or 2, immediately followed by a >:
    ECHO Hello world2>file.txt would result in an empty file file.txt and the text Hello world (without the trailing "2") on screen (CMD.EXE would interpret it as ECHO Hello world 2>file.txt).
    In Windows XP the result is no text on screen and file.txt containing the line Hello world2, including the trailing "2" (CMD.EXE interprets it asECHO Hello world2 >file.txt).
    To prevent this ambiguity, either use parentheses or insert an extra space yourself:
    ECHO Hello World2 >file.txt
    (ECHO Hello World2)>file.txt
  • "Merging" Standard Output and Standard Error with 2>&1 can also be used to pipe a command's output to another command's Standard Input:
    somecommand 2>&1 | someothercommand

Saturday, November 24, 2012

Israeli-Palestinian Conflict: some info I'd gathered


For the exact detail, please go to the link above as the capture down here was not successful, especially the pictures and maps.

I think you have to start with geography.

The piece of land in question has been called a lot of things; we'll use "the Southern Levant," since that's probably the least contentious nomenclature.

Since the Bronze age (3500-5000 years ago), the Southern Levant has been a crossroads between major world powers. First it was Mesopotamia and Egypt on either side, then the Greeks, Hittites, and Elamites joined the party. By the beginning of the Iron Age (2500-3500 years ago), the Levant had great strategic and economic value to the empires around it. Unfortunately, while it was nice enough to be worth conquering, it was not fertile or productive enough to give rise to its own empire. The region was basically destined to be fought over like a juicy bone among jackals, and more's the pity for the poor saps that lived there.

Om nom nom, delicious Canaan.

One somewhat successful attempt at a native kingdom in antiquity was Israel/Judah. Yahweh-worshipping Canaanites pulled together a couple of nice little states for themselves and bullied their next-door neighbors for a few centuries. Eventually, of course, the big guys down the block rolled in and took their lunch money. Assyria smashed and depopulated Israel, turning Judah into a vassal state; when Assyria collapsed, Egypt and Babylon set to fighting over the Judean leftovers. The end result was "population transfer" -- Babylon captured prominent citizens and took them home, and other refugees ended up in Egypt. This marks the birth of the Jewish diaspora.

And you really do have to go that far back to understand Jews' relationship to the Southern Levant. Their religion said that their deity led them to that piece of land and promised that they'd always rule it. It was also really important that they observe religious rituals at the temple in Jerusalem. Then, oopsie, they aren't ruling it anymore and their temple is a smoking heap (not for the last time). Some went back to the Southern Levant after Persia conquered Babylon and said the Jews could go home, but many stayed in their new cities -- all the while retaining their identity as a levantine people, a people whose native home would be ruled by foreigners for thousands of years.

The diaspora only increased in the following centuries. Where Alexander and Hellenism went, Judaism went. Same thing in the Roman Empire. There were Jews in Rome and Jews in India 2000 years ago.

Oh yeah, something else happened about 2000 years ago. Christianity had an unpleasant impact on the Levant: It brought new world powers into the jackal-fight. Now, this little strip of land was not only economically and strategically valuable -- it was spiritually valuable. To more people with armies. You see where this is going. And then, things got EVEN BETTER when Islam emerged, and decided they ALSO thought Jerusalem was kind of a big deal. By the time Europe re-civilized itself, it decided it would be a great idea to go head-to-head with the Muslims over the Holy Land. This was not a super brilliant plan since the Fatimids and Seljuks had a home field advantage and cross-continental supply lines were somewhat lacking.

Guess who's coming to dinner?

But hey, the Europeans had so much fun slaughtering Jews in the Levant that they decided to bring this new pastime home. European antisemitism really kicked into high gear in the Middle Ages, kept going strong during the Renaissance, and had a certain retro allure by the 19th century. European Jews were hanging in there, strengthening ties with each other while trying to avoid attracting the ire of nearby Christians, but life could really suck. Jews were actually doing better in Muslim lands for most of this period.

Of course, Muslim lands were shrinking as the Ottoman Empire "slowly collapsed like a flan in a cupboard" (hat-tip to Eddie Izzard). Geopolitics had gotten quite complicated by this point. If you thought the growing secularity of the 18th and 19th centuries would lessen interest in the Levant, boy, were you wrong! England, France, Russia, Prussia, and Austria were terribly interested in strategic waterway access -- um, I mean, the fate of the poor Christians living under Muslim rule. Sure. That sounds good. Every time the European powers got into a tiff, somebody would suggest a nice friendly plan to divvy up the Ottoman Empire amongst themselves. Then they'd turn around and pledge to protect the Ottoman Empire's territorial integrity.

The Ottoman Empire was pretty badass, actually.

 Aside from schizophrenic obsession with Ottoman lands, the other growing trend in the region was nationalism. Nations had changed, and so had ideas about nations. People were no longer primarily loyal to their regions and cities; nor were they content to be ruled by a far-off empire. They liked the idea of organizing into ethnically and linguistically homogenous countries. The Greeks wanted independence from the Ottomans, the Czechs wanted independence from Austria, Bismarck unified the Germans, Italy came together, and Arabs began to discuss "throwing off the yoke of the Turks."

Jews also got nationalism fever, but there was a problem: They weren't concentrated in one region that could announce its independence and organize itself into a Jewish state. They were scattered inside other ethnic groups' nations -- and those nations made it pretty clear that the Jews were interfering with their shiny happy patriotic ethnic unity. Antisemitic violence kicked into high gear in Europe. So when Jewish thinkers started talking about moving back en masse to the Southern Levant (which had been called Palestine for centuries), Europeans were ready to throw a festive going-away pogrom.

We look favorably upon you getting the hell out.

This is the beginning of modern Zionism ("Zion" being an old Hebrew name for Jerusalem). The return of the Jews to Palesine was enthusiastically supported by British Prime Minister Benjamin Disraeli, who -- in a surprise twist that will shock you silly -- imagined that Britain would provide "guidance" to the new nation. Which obviously had nothing to do with Britain's strategic interest in strengthening its influence in the Middle East and checking Russian designs on Afghanistan. Totally altruistic. With help from donors, a steady trickle of European Jews began to immigrate to the Southern Levant.

Now, so far I've been talking about Jews but not Palestinians. That's because, according to most scholars, Palestinian national identity didn't really exist until the 19th century. People had been living in the Southern Levant all those centuries -- some Jews, but more Arab Muslims and Christians -- but they didn't see themselves as Palestinians per se until they, too, got swept up in nationalism and rebelled against the Ottomans. Unsurprisingly, their Palestinian identity solidified even further as a bunch of European Jews suddenly started moving into town. Locals vs. Interlopers is the oldest one in the book. Folks got along well enough at the turn of the century, but tensions were simmering. By the eve of World War I, Arab Palestinians were complaining about the Ottomans' unwillingness to check the foreign immigration and expressing concern about the social changes that would come from land sales to foreigners.

But hey, don't worry about the Ottomans, guys! World War I finally dealt the killing blow to the "sick man of Europe," and the Great Powers finally got the partition they'd been salivating over for a hundred years. This is where the map of the Middle East starts to look like the one we know today.

Totally reasonable borders that the locals just loved.

See, the newly created League of Nations said all the world's peoples had the right to self-government ... and then carved out a bunch of colonies euphemistically called "mandates," wrapped them up in bows, and exchanged them as Christmas presents. You got me Syria? How did you know! The Southern Levant ended up as the British Mandate of Palestine. Zionists were happy about this turn of events. After all, Britain had issued the Balfour Declaration in 1917, which clearly and unequivocally said that the Zionists had the green light and Britain had their back.

Here's another shocking twist: It said nothing of the sort. One of the most impressively vague documents in history, the Balfour Declaration does not promise to support the creation of a Jewish nation in the Southern Levant. It is simply "in favour of the establishment in Palestine of a national home for the Jewish people." What, you don't know what a "national home" is? Good! That's the idea! Because at the same time, Britain was making other vague promises to the Sharif of Mecca, things that kinda sounded like "You guys can have Palestine" but included a good deal of wiggle room. In other words, they were saying anything and everything to secure the support of whoever seemed important at the time -- good, solid European diplomacy. When Sharif Hussein learned of the Balfour Declaration at the end of the war, he thought little of it. So some Jews want to move back into Palestine. Sure. No prob. There's not too many of them, right? The locals still get to run things? Great.

It became clear pretty quickly that that would not be the case. By 1920, Jews comprised about 10 percent of the population of Palestine and the numbers were growing. Their influence over the British administration was considerable. Local Arabs believed that the British were favoring the Jewish newcomers over the existing Arab population and the end result would be Jewish political and economic domination of the area. Which was, in fact, the goal of the Zionist leadership: "There can only be one National Home in Palestine, and that a Jewish one, and no equality in the partnership between Jews and Arabs, but a Jewish preponderance as soon as the numbers of the race are sufficiently increased." Many Palestinians decided they were being conquered yet again. Groups of Arabs rioted in 1920 and 1921, prompting the British to arm the Jewish settlers. They began to worry about what exactly they'd gotten themselves into.

During the 1930s, shit got real. An Arab nationalist militant group called the Black Hand began attacking Jewish settlers and destroying property. Meanwhile, believing that the best defense is a good offense, hard-liners broke off from the Jewish defensive militia, the Haganah, to form the more aggressive Irgun. Their logo was a fist holding a rifle with the slogan "Only Thus." Friendly guys. Worsening economic conditions for the Palestinian lower class made them more ready for general rebellion. The revolt started in 1936 and continued right up to the beginning of World War II. It's in this period that the familiar patterns of modern asymmetrical levantine warfare emerge: bus and train attacks, pipeline sabotage, civilian murders, collective punishment, torture, curfews, checkpoints, and the wholesale destruction of villages suspected of harboring combatants. All participants -- the British, the Arab rebels, and the Haganah and Irgun forces -- behaved barbarically. In the end, the revolt hurt the Arabs more than the Jews, and pushed the two sections of the population further apart. The prospects for coexistence did not look good. 

During the revolt, Britain had for the first time proposed a "two-state solution" to the conflict. Nobody liked that. Britain backed off from that in 1939, instead suggesting one nation that could be home to Jews and Arabs. This did not satisfy the Zionists, but Britain had decided it didn't care about pissing off the Zionists because Arab support would be more important during World War II. Once again, Europe was playing strategic games with the Levant to protect its own military interests. 

Cut along the dotted line: Instant partition.

Britain also limited Jewish immigration into Palestine in 1939. Bad timing. Only 15,000 Jews were allowed into Palestine per year, but hundreds of thousands of Jews were trying to flee the Nazis -- and it may surprise you to learn that the other nations of the world were not exactly eager to accept these refugees! If any Zionists were still unsure about the necessity of establishing their own nation that they controlled, the Holocaust fixed that. Jews clearly could not count on other nations. Toward the end of the war, the Irgun and other Jewish militants announced an open revolt against the British mandate. Britain just wanted out. After the war, they handed the problem over to the newly formed United Nations. We made a mess, guys. Clean it up. You have until May of 1948, when we're packing up and leaving.

Once again, partition was proposed. The Palestinian Arabs and the member nations of the Arab League hated the idea. Most of the Jews in Palestine were fine with it, though Irgun was not. The UN approved the recommendation, and the Arab League began preparing for war. In the months leading up to the British withdrawal, a civil war broke out between Jews and Arabs, and the writing was on the wall: The partition plan would not be peacefully adopted. Truman tried to stall for time, suggesting that the UN establish a "trusteeship" over Palestine, essentially taking over the mandate and delaying the creation of any independent states in the region. But in May 1948, David Ben-Gurion declared the establishment of the state of Israel. The Arab League announced that this was illegal and invalid because it did not have the support of the local Arab majority, and thousands of troops from the surrounding Arab nations invaded "to restore law and order."

As wars go, the 1948 Arab-Israeli war didn't last long -- about 9 months. Israel won a resounding military victory. It signed treaties with its neighbors to establish its borders, which were better than the original UN partition plan. There were around 20,000 deaths. 

The real impact, however, was demographic: Over 700,000 Palestinian Arabs fled or were expelled from their homes. Whenever anyone is trying to be neutral about this, they will say "fled or were expelled." Because that disagreement -- whether the Palestinians left voluntarily or were driven out -- is right at the heart of the problem today. Palestinians today call this event "al Nakba," the catastrophe. This created a horrifying refugee crisis. Most of Israel's Arab neighbors didn't welcome the refugees with open arms and offer them citizenship, ostensibly to help them retain their Palestinian identity. They also didn't help them establish a Palestinian state. That was never their priority. And Israel wouldn't let the refugees come back, ostensibly because it feared that the returning population would include would-be insurgents who would plot a civil war. Meanwhile, over the next few decades, about 700,000 Jews immigrated to Israel from the surrounding Arab nations -- again, "fled or were expelled." Overall, I suppose you could call this "population exchange," which was all the rage in the 20th century. Greece and Turkey, Italy and Austria, all over Central Europe -- jump on the bandwagon, Middle East! Force people to go live with their own kind and reap the rewards of ethnic unity! Now we call this sort of thing "ethnic cleansing" and frown vigorously in its general direction. 

During the 1950s, the Palestinian fedayeen emerged -- freedom fighters to some, terrorists to others -- killing and wounding hundreds of Israelis between 1951 and 1956. The Israeli Defense Forces retaliated with extreme prejudice, massacring civilians in some cases and hoping to "prove that the price for Jewish blood is high." A Cold War pissing contest turned into the Suez Crisis, with Israeli, French, and British forces invading the Sinai peninsula. It almost blew up into a full-scale confrontation between NATO and the Soviet Union, since the world powers were once again playing the region like a chessboard. Anti-Western and anti-Israel sentiment in the Arab world only increased.

In the 1960s, the Palestinian Liberation Organization (PLO) formed, its stated goal the dissolution of the state of Israel and the right of return for Palestinian refugees. The Six-Day War was another decisive military victory for Israel in 1967, expanding its borders to include all of mandatory Palestine, the Sinai peninsula, and the Golan Heights, which had been Syria's. Anticipating international pressure to give these occupied territories back, Israel promptly began settling Jewish families there. The international community considers these settlements to be in violation of the Geneva Convention, but does nothing to back that up aside from vigorous frowning. 

They got a lot done in six days.

The 1970s marked the beginnings of what we now call "the peace process." Mighty slow process, ain't it. Various proposals have come and gone, with different suggested borders for Israel and disagreements about whether or not an independent Palestinian state should exist. Should Israel give back the land it annexed in 1967? Maybe just some of it? If one wants to negotiate with the Palestinians, with whom does one negotiate? For years, the Israeli right wing said it would never negotiate with the PLO because of its guerrila/terrorist tactics, and the PLO stalwartly refused to acknowledge Israel's right to exist. Israel and the PLO finally sat down together in the early 90s and set a timetable for forging a solution, but in 2000, it became clear that the differences between them were too great. Both sides said they wanted peace, but each had a very different idea of an acceptable peace. 

Peace for our time! Or not.

Over the last decade, the prevailing vision for the peace process has been the "Road Map to Peace." Phase I, which called for an end to Palestinian violence, Palestinian political reform, and Israeli withdrawal and settlement freeze, was initially projected for 2003 or 2004. It's 2012 and it hasn't happened. Israel keeps building settlements, the Palestinian political parties Fatah and Hamas fought a civil war, and there's been plenty of violence back and forth. Interest in a one-state solution has reemerged, but very few Israelis support that. They know full well that if the refugees' descendants get full citizenship and full voting rights, Jews will no longer control Israel. Israeli Prime Minister Ehud Olmert predicted that the outcome would resemble South Africa. Possibly not the most flattering comparison for you, Ehud, but okay. And that's basically where we are today: Completely stalled.