It is almost obvious that “new” not always meets expectations. Some might not agree with me and some might think it is an example discussed so many times, nobody cares already – but I just have to mention Windows Vista. Released 6 years after Windows XP, was especially loathed for its low performance and became usable after serious tweaking.
Fortunately, OTP releases are much more frequent; a few days ago R15B03 version was released. One doesn’t expect major changes from minor release but what about major versions? R13, R14 and R15?
Of course, R15 is most feature-rich, there is no doubt. It’s a shame we had to wait until R15 to get something as basic as line numbers in stacktraces. But has anything changed in terms of performance?
The case I would like to present is Erlang Jabber server. Luckily it can be compiled under R13, R14 and R15 with no changes in the code necessary. What were the results of load tests I conducted?
Well, I was quite surprised. There was no difference in memory usage but let’s say that no change is sometimes a good news. On the other hand, CPU usage was 10% higher in case of R15! I didn’t have time to inspect what could have caused such a difference, but I must admit I’m a bit afraid of R16 release. 🙂
I haven’t done any tests for R13 but I’ve heard from someone else that he could observe the same difference between R13 and R14. In total it gives 20% higher CPU usage in R15 compared to R13.
I just hope this is not a pattern and we don’t get 30% score with R16. Right now, it is difficult to resist a thought, that maybe we should do development and testing (line numbers!) with R15 but deploy our systems on R14? 🙂
Thanks to Henrik, I googled a blog post about this behaviour and also original Rickard Green explanation. It turns out the cause is schedulers working “just in case” before they go to sleep. Fortunately it can be managed with
+sbwt switch for
My apologies, OTP team!