Desperately Seeking Senior
Recently I chatted about appropriate coding assessment questions for senior developers, and came to the conclusion that Solver was a little too demanding for someone to do in around twenty minutes (under pressure), so I replaced it with Quicksort.
My assessment now consists of three questions:
- Fibonacci. In short, “print out” the first thirty terms of the Fibonacci sequence, any which-way;
- Quicksort. Sort a simple list of names into alphabetical order, applying a crude (read: not very efficient) implementation of the Quicksort algorithm;
- University. Code up a set of objects and/or interfaces that describe a simple domain model, illustrating structural relationships between the domain objects.
Here’s University:
At first glance, this would seem fairly easy. The tricky bit comes in due to the fourth bullet point in the problem description: “a lecturer might also be a student”. Now C# doesn’t support multiple implementation inheritance, and this problem calls for a design involving multiple inheritance.
I’m going to present my solution to the problem. I need to stress though, that I’m not wholly satisfied with it because it involves a StudentLecturer type (you guessed it … a hybrid) which just doesn’t sit well with me.
In any case, this is a classic case of the Diamond inheritance pattern (see the Diamond inheritance problem), and this is how the relationships might look:
I’ve deliberately left Course out because it isn’t really central to the real problem, and as a result, just creates clutter. Within the bounds of the description of this problem, this solution might be acceptable, but it’s pretty tightly coupled and promises to turn into a bit of a nightmare should we need to extend the orthogonal roles to more than just Student and Lecturer (although, off the top of my head, I can’t think of how). Ideally though, this sort of mixin scenario is more suited to a dynamic language like Python that allows types to be defined at runtime. Never-the-less, it makes for a worthy brain teaser when done using a statically typed language that only supports multiple interface inheritance.
All three questions comprising the assessment are required to be completed within an hour. What I didn’t anticipate though, is the response to the questions. After assessing twelve people, no-one has provided a satisfactory answer to University. Is this because it’s particularly hard? Are C# developers, in general, not as strong on the modelling front? It’s a little perplexing.
Senior Developer Assessment Revisited
This is really part two of the article I wrote “What is a Senior Developer?”. I’ve received some shrill feedback on my choice of assessment problem:
- Too math’y!
- Standards too exacting!
- A bit much to ask of your typical commercial developer.
So I’ve taken this all to heart and decided to revamp the Solver question. Actually, I’ve decided to drop it completely and replace it with something a lot less “math’y” but possibly no more representative of real-world requirements.
A suggestion that was given me: “get them to sort stuff”. Ok, so what would making someone jump through “sort algorithm” hoops prove? After all, these days, sorting things amounts to a call to List<T>.Sort—I mean honestly, who ever needs to resort to first principles when sorting these days? I’m willing to take a different tack—if I’m testing something slightly different, that is, not knowledge, but the ability to absorb and apply … well then that’s slightly different. Besides … dealing with pointers is generally seen as unnecessary masochism, but some people still regard it as crucial background to being a good developer.
So this time, I’ve actually taken the time to capture the requirements in detail; this amounts to softening things up a little since the general consensus seems to be that the original assessment was too demanding (at least, the Solver question was).
Quicksort, above, doesn’t test ability to perform research independently, and offers a lot of hand-holding, but it is somewhat less daunting than Solver. I can’t help thinking though that things are being dumbed-down a little too much.
The dumbing-down:
- This is limited to System.String, but could easily be extended to be generic (bonus points if the candidate takes the initiative to do this!);
- I haven’t specified any constraints in terms of efficiency issues (the naïve implementation is, of course, a horrible memory hog);
- I don’t know if I could provide any more hand-holding than this, it’s practically paint-by-numbers.
I have been somewhat vague about one thing, namely choice of pivot. I have arguably been a little tricky in this question because the example isn’t consistent in how the pivot is chosen. The astute candidate will quickly realise that choice of pivot isn’t crucial.
Here’s my solution, coded up in approximately 20 minutes:
public static class QuickSorter
{
public static IEnumerable<string> QuickSort(IEnumerable<string> jumbled)
{
if (jumbled.Count() < 2)
return jumbled;
else
{
return
QuickSort(AllLessThan(jumbled.ElementAt(0), jumbled.Skip(1)))
.Concat(jumbled.Take(1))
.Concat(
QuickSort(AllGreaterThan(jumbled.ElementAt(0), jumbled.Skip(1))));
}
}
private static IEnumerable<string> AllLessThan(string value, IEnumerable<string> others)
{
return AllSatisfying(others, s => String.Compare(value, s) > 0);
}
private static IEnumerable<string> AllGreaterThan(string value, IEnumerable<string> others)
{
return AllSatisfying(others, s => String.Compare(value, s) <= 0);
}
private static IEnumerable<string> AllSatisfying(IEnumerable<string> others, Predicate<string> predicate)
{
return others.Where(s => predicate(s));
}
}
Things to notice about my implementation:
- It’s pretty much declarative, thanks to Linq, of course;
- It’s not very efficient … no in-place swapping; that’s what you get in twenty minutes.
I think that this provides a less jarring assessment experience for a would-be candidate than Solver, especially if our candidate isn’t a math-wiz.
Exposing your Applications Guts using IronPython
Application guts (or indeed anyone’s guts) isn’t typically on ones “list of things to see”. Quite often though, when presented with some perplexing behaviour on live, you end up wishing that you’d added a key piece of logging code to get you to the point where you had just enough visibility to be able to solve the problem.
As it turns out, if you’re writing a .NET application, there isn’t a tremendous amount of difference between a Debug and a Release build. That’s because, of course, the compiler isn’t spitting out real machine code, but rather MSIL, and if any kind of optimisation ever happens, it happens at JIT time. At the end of the day, if you’re talking .NET, the difference between being able to debug your application boils down to the availability of some .pdb’s and an INI file … that’s it.
Never-the-less, you may not actually have a copy of Visual Studio or WinDbg with Son-of-Strike installed on the machine that you’re interested in poking around your badly behaving live application with. That’s why you’d be particularly interested in employing the services of a worthy logging framework like Log4Net or the logging block from Entlib at the outset of your enterprise development stint. Just increase your log level to “debug” and you’re a-for-away … right?
There are a couple of issues with this:
- Typically, you need to restart your application or service to get the higher logging level into effect—maybe you have a situation where you don’t want to do that; you just want to view state?
- What if, even on debug level, you’re not emitting the detail that you need? At the end of the day, you’re at the mercy of the vigilance of the developer who wrote the debug entries—they may just not be enough.
How about bundling a little back-door into your app? You could:
- Get at it using something available on any PC, namely telnet;
- Do anything that you would be able to do using code, only dynamically.
Enter PythonServer, available on github. Let’s take it for a run using the (very) simple sample application:
static void Main()
{
Console.WriteLine("Starting...");
var theFibber = new Fibber();
var pythonServer = new TheLimberLambda.Utils.PythonServer(2323,
new [] { new NameBinding("fibber", theFibber) });
pythonServer.Start();
Console.ReadLine();
}
SimpleSample fires up a console, instantiates an instance of Fibber and starts an instance of PythonServer, listening on port 2323.
Fibber is just an implementation of IEnumerable<int> that spits out Fibonacci terms, but the key point is that it was instantiated in and lives inside the SimpleSample process. We keep SimpleSample from exiting by waiting for input on the console.
Telnet’ing into localhost on port 2323 gives us an interactive Python command-line, so legal Python will execute as expected:
The real kicker here though is that we have access to our SimpleSample process, and anything that we decided to publish is available to us (in SimpleSample’s case, that would include our instance of Fibber). Since Fibber implements IEnumerable, we can benefit from IronPython’s automatic recognition of anything IEnumerable as a Python iterator:
Here we’re using the itertools package that comes with Python (or in our case, IronPython) to grab the first 10 items of the Fibonacci series.
Because we’re referencing a single instance of Fibber, and because the state of “where we are” in the series is maintained, we can telnet in from a difference spot, and ask for the next two items:
Thus, we have a Python interface into potentially any .NET application.
Name Binding
Now, how did the name “fibber” become available to us, you may ask? The key is the IEnumerable<NameBinding> that we passed to the PythonServer constructor. At some point we need to provide some translation between the python namespace and the object instances of interest. Presently, PythonServer does this using a dead simple string-to-reference map provided up-front.
Going under the bonnet and taking a squiz at the code that gets executed when a connection is made to the server, we notice the introduction of a ScriptScope instance:
private void InitialiseScriptRuntime(Socket socket)
{
_ScriptRuntime.IO.SetOutput(new SocketConverserStream(socket), Encoding.ASCII);
_ScriptRuntime.IO.SetErrorOutput(new SocketConverserStream(socket), Encoding.ASCII);
_ScriptScope = _ScriptRuntime.CreateScope("py");
}
… and binding names is just a matter of setting ScriptScope variables, thusly:
private void BindScriptScopeNames(IEnumerable<NameBinding> nameBindings)
{
foreach (var binding in nameBindings)
_ScriptScope.SetVariable(binding.Name, binding.Target);
}
I will admit that PythonServer has a way to go, and could do with a whole bunch of things, including:
- A solid security model. At the moment PythonServer should really only be used in controlled environments since of course there is no authentication (or encryption) to speak of—SSH should fit nicely here;
- Integration of the name binding interface into your favourite IoC container.
As a start though, this provides a great means of getting into your process in a relatively hassle-free way.
What is a Senior Developer?
So, at work we’re in this recruitment cycle again. This time it’s aggressive, and we’re really after the cream-of-the-cream. Those hard-to-find coding ninjas who generally don’t ever need to approach a recruitment agent, because of course, the second someone sniffs that they’re on the market, they’re wooed with shares and options and Wii’s and iPads and rubdowns. It’s a mad scramble. Did I mention that I’ve never had to approach a recruitment agent? ;)
The best way to finger these types is through someone you know who’s really good, who knows someone they worked with at some point who blew their socks off. Unfortunately the network method has failed us—sadly it looks like all talent has gone deep underground, or left the country. I’m partial to the latter because quite frankly, the quality of the meat that our corporate designated agent is passing our way has been found wanting … repeatedly.
But before I get ahead of myself—let’s approach this methodically, like we should a new project.
Step 1: Clearly define what we require
Enter the Senior Developer. As expected, this is all too subjective … a quick zoot over to SO confirms our fears and leaves us unsatisfied: it depends. I wish it was as easy as “Spanish male developer” (si señor!).
Being the “main technical peanut” using our PM’s terminology, the responsibility of defining the standard falls on my shoulders. I do subscribe to Joel Spolsky’s “smart and gets things done”, but the definition somehow falls short … it just isn’t complete.
Drawing from that deep unknowable über-developer essence that I supposedly have access to, I therefore decree:
Senior Developer Quality 1: Professionalism
Now we all know that you can’t distil what makes a senior developer into one simple thing, but this is one of those undeniably big differentiators. Can you do a good job, consistently? If yes, then proceed to next assessment gate.
Senior Developer Quality 2: Intelligence
Where I come from, there seems to be this unspoken rule: “one doesn’t explicitly talk about smarts”. Because of course, smarts is one of those things that if you don’t have, no amount of experience is ever going to improve the situation. Let’s square up to this, for crying out aloud—if you want to be a senior developer you absolutely must be smart. Preferably, very smart.
Senior Developer Quality 3: Passion
Another big differentiator from the unwashed Mort-cast. You gotta wanna learn, all the time, during meals, on the can, driving to work, driving from work, on the treadmill, aside the water cooler and anywhere else you care to mention. Senior developers have technology in their veins, they live it and breathe it. Excitement isn’t derived from the promise of a ticket to the Super 14 final, but rather the appearance of a postal collection note for that 200MB/s write-rate SSD from Newegg.
Senior Developer Quality 4: Humility
Getting to the point of truly grokking that no matter how good you think you are, there’s always someone else out there who’s better than you are is a watershed moment. In all honesty though, no-one enjoys an arrogant git … it just isn’t conducive to greasing the cogs of the team dynamic. The more numerous the alpha-geeks in a team, the more critical the quality of humility becomes.
Senior Developer Quality 5: Experience
There’s a bit of cross-over here with quality 1, so let’s say that in this case we’re particularly interested in the sort of experience that gives you that technical “gut feel”. After a number of years, the neural pathways have been set up so that you can generally “smell” whether something sounds right or it doesn’t (when interacting with colleagues), and your hunch about where problems may lie tend to be more often right than wrong.
Step 2: Screen ‘em
When it comes to finding good people, and when you don’t have the luxury of a network-enabled direct route, it boils down to a numbers game.
Spolsky advocates the phone screen, but we’ve opted for a technical assessment. Do we really want to waste our time sitting down to chat with someone if they don’t make the bar? So to be sure, this is an effort to weed out those who think they represent our definition of Senior Developer, but who don’t.
A simple test should suffice. I drafted one this morning, and I’m going to publish one of the questions (with sample answer). Now you’ll notice that the problem posed isn’t very challenging (although some of my colleagues beg to differ), but you’d be surprised at how many people who sell themselves as senior developers who can’t do it.
My lazy side originally opted to go for one of those shrink-wrapped multiple choice online assessments ala Brainbench. I’m not going to mention the brand of assessment that is our corporate standard because I have nothing good to say about it—typographical errors, and code that wouldn’t compile in almost every single question? I wasn’t impressed either. Suffice it to say, it’s not Brainbench. Whatever the choice of assessor though, all of these tests suffer from the same problem; they tend to test stuff that we would naturally expect to Google these days. I don’t rate that as being particularly useful at all.
Solver, above, may seem overly mathematical and unrepresentative of typical “throw-stuff-in-a-database-and-pull-it-out-again” business requirements, but what it does do is very quickly highlight the sort of person we don’t want.
- Do we want someone who can’t understand the question because it contains “nasty unknowable symbols”? Even if you didn’t do maths at university, surely you did algebra at school? That’s all you need to know.
- Do we want someone who can’t do research on the Internet? Jeepers, I even provided the exact Wikipedia search. The corresponding article, predictably, contains pseudo-code for various algorithms—would you honestly need more than that?
- Do we want someone whose brain can’t be stretched to understanding an iterative algorithm to which they haven’t previously been introduced?
- Do we want someone who doesn’t know enough to fire up the browser and at least try “+”root-finding” +C#” on Google? (Yes, I do check the browser history afterwards :)
Of course I’m not expecting Newton’s method here, simple bisection will suffice. Even if our prospective senior has never attended a calculus class, I would expect her to be able to fathom this one unassisted.
Here’s my bisection code, written up and tested in 20 minutes—too much time, I might add, for what I would consider a senior who lives and breathes code:
class Program
{
static void Main(string[] args)
{
var lhs = (Func<double,double>) (x => x * x - 3);
var rhs = (Func<double, double>)(x => x * Math.Log(x));
var diff = (Func<double,double>) (x => rhs(x) - lhs(x));
const double threshold = 1e-5;
Func<double,double,double> findRoot = null;
findRoot =
((l, r) =>
{
if (Math.Abs(l-r) < threshold)
return l;
var mid = (r+l) / 2;
return (Math.Sign(diff(l)) == Math.Sign(diff(mid))) ? findRoot(mid, r) : findRoot(l, mid);
});
Console.WriteLine(Math.Round(findRoot(1,50), 2));
Console.ReadLine();
}
}
Now the big problem with bisection, of course, is local minima, but that is a non-issue because I even provide a graph illustrating that there aren’t any local minima.
It’s time to wrap this post up, and defer description of the how testing against qualities 1 to 5 should be done to another one. Today we presented our first senior candidate with Solver, but you can probably guess what the outcome was when I tell you that he couldn’t nail Fibonacci which was to print out the first 30 terms of the sequence of the same name. Sad, indeed.
11 comments