Not so long ago I had an occasion to take a test created by IKM. I’m not sure if it’s a very common way to assess programming skills but certainly the process itself looks interesting. Sadly, it seems they don’t have Erlang suite but as I developed a few things in C++ and Java before I learnt Erlang, I thought “why not?” and decided to check how much I have forgotten and how much still lingers in my skull after 2-3 years.
I have to admit – at the beginning I wasn’t sure what to expect. I was told just “IKM”, of what I’ve never heard earlier. I was wondering whether will it be problem solving, knowledge of the language core or some fancy libraries? It turned out it was the second option with a bit of the third one. I was a bit disappointed, since language is a tool (of course it is important to know your tool!) but what use is of a developer who kicks ass in C/Java/Erlang/etc. and is unable to understand the project as a whole or design a simple state machine and is just programming robot?
Anyway, probably problem solving is a difficult class of skills to be judged by computer algorithm. The tests in case of both languages consist of questions checking the understanding of language mechanisms plus some aspects from core libraries like java.io or STL. The libraries were the most difficult part for me, since I always had a problem with remembering all these BufferedFileWriterReaderShredderDiaperChanger classes (e.g. it took me a really long time to remember argument order for lists:foldl/3 :)). There were some tricky questions where I was asked for a predicted result of executing the snippet and there was just one semicolon missing or lack of int in front of function declaration. I especially liked the question about finalize method in Java; I still remembered it is not guaranteed for it to be called ever. 🙂 The frustrating one was about diamond inheritance in C++. I’ve used it once about 10 years ago when I was learning C++ and still knew the concept but even with utmost effort I couldn’t recall the correct syntax for it.
The creators of the test algorithm state that the software can adjust questions difficulty so to the assessed folk the questions will always seem challenging but not impossible. I must admit this is true. There were few questions I could answer without hesitation but I also haven’t utterly failed the rest.
I haven’t mentioned it’s a multiple choice test and bad answers lower your final score. This means sometimes it’s just better to skip the question than pick random answers (and so did I in a few cases). Answers are weighted, which means some correct ones are more obvious than others, which means less points for them. The same goes for wrong ones – completely stupid option means a considerable loss of points.
In general, both Java and C++ tests took about 1 hour each and I got about 80% in both. Perhaps for someone it would be a complete failure but given my long break and using mostly Erlang on daily basis, I’m very pleased with such results. I clearly forgot java.io, never learnt STL too well but at least my results on core language mechanisms are more than average.
… and yes, I decided to be fair and didn’t have any handbook on my desk or Google opened on laptop. Most of all, I was curious and eager to try my actual knowledge. I didn’t care about getting 100% result. I even dared to have a beer when I had only a handful of questions left. 🙂
In overall, I consider it a nice experience. I understand, that for an average company it is a good way to judge if the candidate has certain skills without spending a lot of time on “manually” checking everyone. The big mistake that some probably make, is relying only on this tool or treating it as basic sieve. It’s good that final report actually divides the questions in certain classes like “memory management” or “Java IO”, so someone who understands memory management in C++, won’t be at much disadvantage because of poor skill in STL.
… but the best assessment ever will always be:
‘Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.’