The Limber Lambda

… is not a word that you’ve likely encountered too often.  It happens to be an alias for the ubiquitous backslash.

\

Now as a programmer, I’ve come to react to certain ASCII characters instinctively:

That last one is special … it transcends ordinary meaning.  That’s because its scope extends beyond just itself … to what follows it.  And that, with little argument, is generally accepted the world-over.  The backslash character is the universal escape character.  Why then, is it so difficult for Microsoft to get in line?

Larry Osterman sums up the origins of the gross misuse of the backslash character:

Many of the DOS utilities (except for command.com) were written by IBM, and they used the "/" character as the "switch" character for their utilities (the "switch" character is the character that’s used to distinguish command line switches – on *nix, it’s the "-" character, on most DEC operating systems (including VMS, the DECSystem-20 and DECSystem-10), it’s the "/" character" (note: I’m grey on whether the "/" character came from IBM or from Microsoft – several of the original MS-DOS developers were old-hand DEC-20 developers, so it’s possible that they carried it forward from their DEC background).

The fact that the "/" character conflicted with the path character of another relatively popular operating system wasn’t particularly relevant to the original developers – after all, DOS didn’t support directories, just files in a single root directory.

Then along came DOS 2.0.  DOS 2.0 was tied to the PC/XT, whose major feature was a 10M hard disk.  IBM asked the Microsoft to add support for hard disks, and the MS-DOS developers took this as an opportunity to add support for modern file APIs – they added a whole series of handle based APIs to the system (DOS 1.0 relied on an application controlled structure called an FCB).  They also had to add support for hierarchical paths.

Now historically there have been a number of different mechanisms for providing hierarchical paths.  The DecSystem-20, for example represented directories as: "<volume>:"<"<Directory>[.<Subdirectory>">"FileName.Extension[,Version]" ("PS:<SYSTEM>MONITR.EXE,4").   VMS used a similar naming scheme, but instead of < and > characters it used [ and ] (and VMS used ";" to differentiate between versions of files).  *nix defines hierarchical paths with a simple hierarchy rooted at "/" – in *nix’s naming hierarchy, there’s no way of differentiating between files and directories, etc (this isn’t bad, btw, it just is).

For MS-DOS 2.0, the designers of DOS chose a hybrid version – they already had support for drive letters from DOS 1.0, so they needed to continue using that.  And they chose to use the *nix style method of specifying a hierarchy – instead of calling the directory out in the filename (like VMS and the DEC-20), they simply made the directory and filename indistinguishable parts of the path.

But there was a problem.  They couldn’t use the *nix form of path separator of "/", because the "/" was being used for the switch character.

So what were they to do?  They could have used the "." character like the DEC machines, but the "." character was being used to differentiate between file and extension.  So they chose the next best thing – the "\" character, which was visually similar to the "/" character.

And that’s how the "\" character was chosen.

Everywhere else we instinctively know that a backslash escapes things … everywhere … except for Windows/DOS path names.

In defence of Windows, the OS is perfectly happy to interpret slash (“/”) as it would slosh … and for the most part application software observes this.  There are occasional heretical instances where this is not true though and the application software writer has (<<shudder>>), deliberately chosen the backslash as a categorisation delimiter.

So, today I battled through an instance of this, trying to call sqlcmd.exe from within a Bash script.  Now I realise that this is largely a function of the implementation (which was re-expanding parameters multiple times before they were eventually passed to sqlcmd.exe), but oh how this would so have been a non-issue if someone, somewhere had simply respected the good ‘ol escape character and left it to only ever escape and nothing more.

What was needed in the invocation of my Bash script:

… to eventually get into the form that sqlcmd.exe desires:

What a wonderful place the world would be if we could all agree on one thing: slosh is for escaping … nothing more, nothing less.

I’ve been working now for some fifteen years, and I think that for someone who’s had a bit of experience, they can honestly look back and draw some valuable conclusions on “what’s important”.  By implication, there are also a whole bunch of things that are not important.

Today, sitting with a prospective hire, I was asked what I thought of courses.  I interpreted it is a question about the value of courses, in general.  That’s a tough one.  Any academic endeavour, ultimately, is only worth it’s level of recognition, presumably by a body or group of bodies whose opinion one values.  The only truly worthwhile “course”, in my opinion, is one that you get through a reputable university.  Ultimately, it then boils down to what is meant by “reputable”.

In IT, there will always be a market for courses.  At first this statement seems obvious—after all, isn’t the way one learns how to administer an instance of Microsoft Exchange, or SharePoint, or Microsoft CRM is through a course?  Is this the only way?  If you’re a corporate charged with maintaining a level of skill within the org, it sure seems like the only way … it sure is the most expedient (at least at first glance).

The course-peddlers know this, and they exploit it.  Corporates are easily wooed by the promise of the training silver bullet.  It’s money for jam … time away from the office and the shelling out of a few shekels brings you back shiny new Exchange-ready employees the following week.  There’s a problem with this though, it’s subtle—they’re Exchange-ready … yes, but they’re not really Anything-Ready, and that’s what we should really be looking for.

Anything-Ready

Anything-Ready is a meta-concept … it’s not about anything particular that you may know, or that you may need to know, but rather how good you are getting to know what you need to know.  I’m not looking for Master Exchange Certified, or SCRUM Master Certified … I’m looking for Master UPSKILL Certified, All-Technologies-Applicable.  Now that is valuable.

Which brings me back to why I place value in a university degree.  When you go to university, you learn how to learn, and if you’re lucky enough to go on to do a post-graduate qualification, you get first exposure to Anything-Ready 101.  I’m probably being unfair in that this judgement really has been made through the lens of technical degree … can’t vouch for a BA.  Ultimately though, there’s still a whole lot of learning to do before you graduate as “Anything-Ready” and that’s the job of industry.  True Anything-Ready graduates are the product of years of industry-grind-mill, post university degree.

So .. what is Anything-Ready?

Apart from getting your daily dose of mental calisthenics, addressing the root cause means that you’re eliminating a fear—that’s because if you haven’t gotten to the nub, you will always remain in fear of what may pop up later on because you chose to shortcut grokking the problem.

You can spot a course-taker a mile away, she tends to shy away from anything not within the ambit of what has been explicitly learned—face your fear and become a polyglot.

A keen study of Anything-Ready knows that the “Ready” part needs constant attention—regular reading is a must to maintain that status.

I continue to be astounded by the number of programmers who haven’t mastered the bread-and-butter of their tools.  If I had a penny for every “senior programmer” I’ve met who can’t touch-type… Jeff Atwood says it far better than I could.  As for your editor … choose one, embrace it, get to grok it, because if you’re a programmer, it’s the canvas of your craft.

No man is an island.  Programming and Autism—two peas in a pod?   It doesn’t have to be that way.  Humans are computers … just really complex ones, and it does take a while, but every programmer eventually figures out that solving certain problems can be done an order of magnitude faster through a small human interaction than solitary toil.  So keep community, not so much for the selfishness of getting what you need quickly, but for the benefit of your own soul.

There’s no barrier to entry with programming.  Anyone with a laptop and an Internet connection can do it.  More alarming though—anyone with a laptop and an Internet connection can convince Joe Sales Exec that they can do it well.  That’s why the keeper of “Anything-Ready” faith knows how important always beats urgent.  The “sins of the fathers” is a biblical meme that applies everywhere, not least in programming—careful what decisions you make, because they form more of a legacy for those who come after you than you may think.

It’s incumbent on us to be vigilant about what’s important.  If we don’t, we’re in danger of wasting time.  Distilling what’s important is about cutting to the chase and removing waste.  It’s another way of repeating DRY (the only time you get to repeat yourself and feel good about it).  Anything-Ready-practitioners are voracious distillers of what’s important.

Someone who is Anything-Ready knows that it’s tenuous title.  They don’t call it the fire hose for nothing.  Do you know someone who can actually drink from a fire hose? (no spilling).  So keep that ego in check—there’s always someone around the corner way smarter than you.  Really.

Back to Courses

Courses aren’t all bad.  A good course will:

• Provide a formal, structured means to refocus;
• Expose you to people who you wouldn’t normally have had the chance to meet;
• Put your bosses mind at ease that the “skill issue” is being addressed (definitely a soft benefit);
• Keep the paper mills afloat.

Today we assessed our 20^th candidate for position of senior developer, using a small test that I drafted some time ago.  Since then I’ve spoken about how the test was dumbed down since one of the questions was perceived as being too difficult.  The assessment consists of three questions, which I’ve dubbed Fibonacci, Quicksort and University.  I’ve already discussed two of the questions, namely Quicksort and University.

Although not an earth-shattering sample, 20 is a number that we can start to draw pretty graphs with, so I’ve included a summary of the results alongside.

We can speculate about what it tells us:

1. There’s something wrong with University; nobody has come up with a satisfactory answer;
2. In the case of Fibonacci, there’s something wrong with the candidates; only five people have come up with a satisfactory answer.

Ok, my second conclusion above is possibly a little harsh.  I think it’s a little harsh, because upon inspection by others, it’s elicited use of some power words like “mathlete” and “math-wiz”.

Now Fibonacci is the only question that I haven’t posted for all to see, mostly because I thought it was too simplistic and therefore quite uninteresting.  Given these results though, I’ve been prompted to reword it completely.  I mean, if only math-wiz’s have what it takes to do it then I can’t make any assumptions about the background that a candidate may have in the arithmeticmaths department.

What follows is the version of Fibonacci as it was up until now:

I’ve taken great pains to remove anything that may constitute an implicit (read “unfair”) assumption about mathematical background and I’ve reworded the question to provide a lot more hand-holding for the candidate.  This should also help in offsetting the nervousness factor.

Here’s my updated version:

This is really part two of the article I wrote “What is a Senior Developer?”.  I’ve received some shrill feedback on my choice of assessment problem:

• Too math’y!
• Standards too exacting!
• A bit much to ask of your typical commercial developer.

So I’ve taken this all to heart and decided to revamp the Solver question.  Actually, I’ve decided to drop it completely and replace it with something a lot less “math’y” but possibly no more representative of real-world requirements.

A suggestion that was given me: “get them to sort stuff”.  Ok, so what would making someone jump through “sort algorithm” hoops prove?  After all, these days, sorting things amounts to a call to List<T>.Sort—I mean honestly, who ever needs to resort to first principles when sorting these days?  I’m willing to take a different tack—if I’m testing something slightly different, that is, not knowledge, but the ability to absorb and apply … well then that’s slightly different.  Besides … dealing with pointers is generally seen as unnecessary masochism, but some people still regard it as crucial background to being a good developer.

So this time, I’ve actually taken the time to capture the requirements in detail; this amounts to softening things up a little since the general consensus seems to be that the original assessment was too demanding (at least, the Solver question was).

Quicksort, above, doesn’t test ability to perform research independently, and offers a lot of hand-holding, but it is somewhat less daunting than Solver.  I can’t help thinking though that things are being dumbed-down a little too much.

The dumbing-down:

• This is limited to System.String, but could easily be extended to be generic (bonus points if the candidate takes the initiative to do this!);
• I haven’t specified any constraints in terms of efficiency issues (the naïve implementation is, of course, a horrible memory hog);
• I don’t know if I could provide any more hand-holding than this, it’s practically paint-by-numbers.

I have been somewhat vague about one thing, namely choice of pivot.  I have arguably been a little tricky in this question because the example isn’t consistent in how the pivot is chosen.  The astute candidate will quickly realise that choice of pivot isn’t crucial.

Here’s my solution, coded up in approximately 20 minutes:

	public static class QuickSorter
	{
		public static IEnumerable<string> QuickSort(IEnumerable<string> jumbled)
		{
			if (jumbled.Count() < 2)
				return jumbled;
			else
			{
				return
					QuickSort(AllLessThan(jumbled.ElementAt(0), jumbled.Skip(1)))
					.Concat(jumbled.Take(1))
					.Concat(
					QuickSort(AllGreaterThan(jumbled.ElementAt(0), jumbled.Skip(1))));
			}
		}

		private static IEnumerable<string> AllLessThan(string value, IEnumerable<string> others)
		{
			return AllSatisfying(others, s => String.Compare(value, s) > 0);
		}

		private static IEnumerable<string> AllGreaterThan(string value, IEnumerable<string> others)
		{
			return AllSatisfying(others, s => String.Compare(value, s) <= 0);
		}

		private static IEnumerable<string> AllSatisfying(IEnumerable<string> others, Predicate<string> predicate)
		{
			return others.Where(s => predicate(s));
		}
	}

Things to notice about my implementation:

• It’s pretty much declarative, thanks to Linq, of course;
• It’s not very efficient … no in-place swapping; that’s what you get in twenty minutes.

I think that this provides a less jarring assessment experience for a would-be candidate than Solver, especially if our candidate isn’t a math-wiz.

I’m going to say something shocking …

Your degree that you worked for for years doesn’t mean a whole lot.

I can say that, because I was living proof.  I studied for six years and ended up with a masters in electrical engineering.  At the end of it all though, there’s only one pertinent question:

Can you do a good job, consistently?

And, I’m ashamed to say that at that time, as a fresh start in my first job, I could not reply “yes” to that question, at least according to my present-day definition.

So, what does “good” mean? you may ask.  Good point.

It can be argued that it’s in the eye of the beholder, but “good” can be characterised by a range of things and it’s pretty much universally recognisable:

• It’s complete.  The proverbial i’s have been dotted, and t’s crossed.  It’s been checked, and cross-checked.  Complete has an “aroma” to it—you actually want someone to look at it—you’re proud of it.
• It’s timely.  Part of “good” is sticking to your word—you said it would be finished by the end of the week?  Make it so, by hook … or by crook.
• It’s succinct.  There’s been an effort to ruthlessly focus on what’s important.  Just because the annual budget is important, it doesn’t have to be voluminous.
• It’s correct.  That means you’ve proof-read it, and possibly even got someone else to proof read it.  If it’s code you’re writing, you’ve written automated tests to confirm that it behaves the way it’s intended to.
• It has built-in continuity.  Someone needs to extend it or enhance it?  No problem, there’s a Readme that bootstraps you … and the code is “literal” (not necessarily Knuth’s definition, but aspiring to be).  The way to nail this one is to regularly put yourself in the next persons shoes.
• It’s transparent.  It’s built to be absorbed by other human beings—there are no gotcha’s.  If it’s code, it isn’t too “clever”.
• It fails fast.  When you build something, you need to be in the place where you know it’s going to fail.  Good is not perfect.  So when it fails, even when the condition seems innocuous, fail fast and fail hard.  That’s because too much energy is expended on hiding and fretting than failing and fixing.

The thing to notice about all the points above is that they’re not actually taught at school, not explicitly at least.  For me, that’s ironic—because knowing what I know now, I would take someone who’s got those things down pat any day over someone who has a PhD but who can’t deliver.

So, it’s worth ruminating on today: “Am I doing a good job?”.

Today I attended an interview for a prospective VB6 candidate.  Are there serious developers out there who actually want to develop in VB6 going forward, without a prospect of perhaps moving onto a newer platform?  Apparently there are.

Asks I: Can you tell me what polymorphism is?

Says prospect: [paraphrased] It’s when you have a method that takes two parameters, say x and y and another method with the same name that takes three parameters, x, y and z.

Asks I: Say for instance you have an array of strings … a bunch of names of people … but they’re all jumbled up, how would you get them into alphabetical order?

Says prospect: I would do an “order by”.

Asks I: Can you tell me what COM is all about?

Says prospect: It has to do with objects and DLL’s (waves hands in air for effect).

Asks I: Can you tell me what a hash table is?

Says prospect: No [pause] … [a "go-on-give-it-a-go" nod from me]

Says ex-prospect: Aren’t you going to tell me?

Evidently Mort and his cohorts are alive and well and being paid real shekels for propping up those critical business systems.  Afterwards a colleague called me out for asking irrelevant questions like … well, who in the world is expected to know what polymorphism is?  Especially in the VB world (puh!).

This just re-affirms my (reluctant) assertion that it doesn’t really take rocket scientists to keep critical business systems running.  It doesn’t even take people who are mildly interested in furthering their career of choice.  When the proverbial hits the fan, folks may end up running around screaming blue murder for a while, but the problem will be fixed, and more than likely it’ll be fixed with a little duct tape.  Until of course the tape peels off, and a patch over a patch is required.

Let’s say that again–the system will be fixed.  It might be expensive, it might be inelegant, it might take a couple of days off someones life, but things will carry on, and in a year or two the cross will be borne by a new eager victim, one who may even have been lucky enough to have had a mere brush with a computer science text book (we can only hope).

Sadly I have no wise rebuttal right now, because the numbers just aren’t there.  And that is, simply put, why some people can play pied piper to the development masses and lead them into dubious territory.  Our industry is a young one, too young still for the toll of poor practice to be quantified.  My most effective contribution right now then is to continue playing disciple to the way of Uncle Bob and hope that one day, all will be converted.

Anil Dash recently wrote an article on what he calls the Pushbutton Web, a catch-all term not dissimilar to AJAX that describes the popular concept of technology initiatives related to the real-time web.  The assertion is that open protocols like PubSubHubbub are the future means by which we’ll get real-time updates, that is, the real-time web will no longer be the exclusive domain of semi-proprietary protocols like those used with Twitter and Friendfeed.

PubSubHubbub sets out a way for subscribers to be directly notified when there’s an update to a providers RSS or Atom feed.  Here’s how it works:

• Firstly, the content provider (Any RSS source such as FeedBurner) requests a selected hub to respond to future notification pings about updated content (1);
• When a subscriber (choose your RSS reader) asks for content (2), the response (3) contains not only the newest articles but also some metadata informing the client that there is one or more hub that can be used for notifications;
• Subscribers subscribe to content updates by asking the designated hub for updates (4);
• As updates come in (5), they’re broadcast to interested subscribers (6).

Something key that I haven’t mentioned about the diagram above is that the lines represent HTTP requests and responses (blue are requests, green are responses).  The dotted lines represent the “push” part of the protocol.  Something that’s noticeable to anyone who’s ever had to do any networking or administration of a firewall is a rather naive assumption about the connectivity between the hub and the Subscriber.  Now, if by “Subscriber” we’re talking about your typical Internet-connected workstation, then there’s something missing in this picture.  That’s because the vast majority of Internet connected workstations connect via a firewall or a NAT device.

In practice, this translates to:

And trying to hunt down a response by the designers to this little snag reveals:

"I’m behind a NAT. Can I subscribe to a Hub? The hub can’t connect to me."

Anonymous

 

No, PSHB is a server-to-server protocol. If you’re behind NAT, you’re not really a server. While we’ve kicked around ideas for optional PSHB extensions to do hanging gets ("long polling") and/or messagebox polling for such clients, it’s not in the core spec. The core spec is server-to-server only.

Brad Fitzpatrick, San Francisco, CA

 

Looks like the final word on the matter. PSHB is not intended to be used by Internet connected workstation subscribers.  How then does PSHB help us get closer to an open-standard Twitter or Friendfeed?  In the short term, at least, it doesn’t.

So, to conclude, for all practical purposes, our “real-time-web” world is going to be one of piggy-backing the HTTP response, in it’s various incarnations.  To the purist, this spells “hack” and PSHB clearly hasn’t compromised on this one.  Hopefully firewall vendors out there will build in the ability to respect and forward incoming PSHB notifications at some point.  Until then though, the Internet will use what’s simple, available and works—that one I will agree with Anil on.