Are too many units tests bad for your project?

Perhaps to some this might sound like heresy and to others it my seem like common sense.

I am a fan of unit testing but have recently been thinking about the right number of tests prompted from observing an over zealous attempt at creating unit tests.

During this time I was reminded that tests are not free, they take time to create, maintain and run. The idea is this time is recouped during development by catching errors early and acting as a sanity check. This kind of works out quite well most of the time.

There are times when it can be a drag.

The first of these is small prototype projects. The sort of ones I talk about on this blog. Things that are knocked out on a weekend or at most a month. They are mostly for personal consumption, occasional I do do unit tests but often only start adding unit tests after the first release. If I start on a second iteration it probably means I will work on it on and off for an extended period and unit tests will pay for themselves. I doubt people will disagree with this one as it is not exactly controversial.

The point of this post is I recently observed someone create many hundreds of unit tests that were all permutations of each other. It took him quite a while. All these permutation represent  different options that can be given to a system. The code is programmed such that there are many similar functions (this part of the code if old and by that I mean decades) so he used that as the reasoning for writing the test while altering the code.

The trouble is the possible combinations is so large even this guys herculean attempt could not cover everything and general testing revealed more bugs.

This led me to thinking someone sitting down and attempting to write lots of unit tests should trigger warning alarms. Personally I felt it is more appropriate to spend time going over the code to reducing the number of branches/functions in the system meaning we could cover the code with less unit test.

To answer the obvious re-action. there are many layers of testing so it is possible to refactor code fairly safely without unit tests in place.

TL/DR Sometimes it is better to simplify code rather then write unit test for complex code.

A Journey into Emacs is a Long One.

I never really intended to get pulled into emacs  I admit I have at points in the past liked it and at other points found it frustrating  My return to it from vim for simple editing and lots of eclipse for Java was initiated when I wanted to play with Clojure. Doing lisp and not using emacs just did not seem right :)

My adventures with Clojure did not last long and I move on to revisiting JavaScript. However I stuck with emacs and use the official js-mode along with the js2-mode thing originating from Steve Yeggie. I think I prefer the Steve Yeggie one at the moment. As there does not seem to be much development on either of these two modes I doubt my opinion will change in the next year of so.

I have written a couple of noob elisp functions but not anything that I use daily. Usually I myself coming up with an idea fiddling a bit and then realising that core emacs has it build in or someone else has already added the idea to some extension. This is good as it is not really a goal to become really good at hammering out elisp. I will just learn as I go.

Some things that I have found useful are:

Book Marking
This is basically a list of places on your drive that you want to jump to. Say different project directories and so on.

The main keys are

•  C-x r m                     - Make bookmark, you will be prompted for a name
•  C-x r l                       - To list them
• M-x bookmark-save  - To save them so they persist past the session

When in a book mark list buffer

•  D - delete a bookmark
•  r  - to rename it

I use this everyday.

Register Bookmarks
This one is more for jumping around a file. It lets you drop bookmarks and return to them. The keys for this are

• C-x r SPACE [a-z]  - Store a position
• C-x r j [a-z]      - Jump to position

I am guilty of not using these enough. They have not made it into my muscle memory just yet. I am using them more and more so they will be soon.

Navigation Stack
Again another one I am just getting used to and am not 100% on how exactly they work. But they let you go back to position in a buffer or between buffers

• C-u C-SPACE - Back to a previous position in a buffer.
• C-x C-SPC      - Back to a previous buffer.

Alignment
I really like this one and nearly wrote some elisp to do this before I found out it existed.

Say you have some code

x = 90
myvar = "the"
thisIsAReallyLongVariableName = 1

and you want to line up all the equal signs to make them a little clearer. It is simple, you just highlight the region and then do

M-x align-regex press return;
then type = and press return
Hey presto we have

x                             = 90
myvar                         = "the"
thisIsAReallyLongVariableName = 1

I really like this as I mentioned before.


Other stuff,
I followed some tutorial when I was first setting up emacs to get things like ido mode working and using them makes the whole experience a lot better. There is an lot of setting up emacs articles on the web.

I will not be doing any proper Java coding in emacs, it would be significantly slower but perhaps on cold winter evening I might give it a go. I also suspect there are better editors for JavaScript but for now I am quite happy.

I think I am over the hardest part of the learning curve and emacs is starting to become enjoyable to use. If I learn elisp it will be a gradual thing done on an as needed basis and I feel like I am now into the slow customisation phase that will slowly boost my productivity.

Do I code faster in emacs? Saying yes would be nice but I don't honestly know. I suspect coding JavaScript in eclipse or some other ide would be just as fast but I can't be sure as a lot of productivity is subjective. At the moment nothing gets in my way when using emacs and it is fun to use, I don't mind firing it up late at night when I am tired and that helps.

Coding using emacs and JavaScript is so distant from what I do at work it just feels like fun. Hopefully as my JavaScript code grows in size my skills in emacs will also grow to accommodate the added complexity.

It Seems To Go In Circles

A long time ago I wanted to create and sell a programming language (this was like 12 years ago), like in the old days of AMOS. At the time looked just about possible but I realised that a spare time hobby it would take me years to get anything descent working. The market was also falling out of being able to sell software like that. Yes some some where surviving but if was getting harder and hard to sell a language and its associated IDE.

Fast forward to now and it looks like it could be a good time to be coming to market with your own programming language.

So what has changed. Well are a few things.

First the web or should I say the cloud. People want to do stuff on it including programing. Programming as a service is what they probably call it. For a monthly fee you can get all your programming needs hosted on the web.

People are happy to pay to have access from anyway and have it all stores on the web. Yes current one are using existing languages but it is only a small step to creating you own one. Compiling down to Java Script and using the HTML5 goodness is an obvious choice.

The seconds is while 12 years ago the only open platforms we had to target were Window, Mac and perhaps Linux not there are many open-ish platforms. The three desktop are still there and then there is android, ios, blackberry, html5. A programming language that abstracts that away and lets people get on with making say games (where this abstraction is easiest) is something people will pay for. Monkey is an example of this.

Finally the new platforms other you a chance to gain a foot hold. Many programmers would love to code on their tablets. While this may, or may not, be allowed on the ipad it is definitely allowed on android devices. Creating a good IDE that works on tablets is only just starting to be done and is still pretty open.

I am not about to jump into this field much as it would be enjoyable I could not devote the time required to get something working as well as I would want to. I do look forwarding to watching these thing develop.

For some reason I had the urge to blog about this today.

Simple Dungeon Generation

In my last post I talked a little bit about Maze/Dungeon generation. As I was typing out the post it became clear I needed some pictures. So this post is going to repeat a lot of what I said in the previous post but with pictures.

As a learn Dart project I decided maze generation would be a nice thing to fiddle with. To be honest it is not really a learn Dart more get a feel for whether I enjoy the optional typing and other features Dart provides. Dart is pretty easy to learn.

I started with a couple of algorithms that I picked up from Wikipedia. That is a depth first algorithm and Kruskals algorithm for maze generation. Wikipedia outlines these algorithms and I don't feel the need to explain them. Long story short mazes are generated and here are a couple of examples

Depth First Maze Generation Algorithm

Kruskals Maze Generation Algorithm

Yay! We have mazes. The little blue lines represent doors. I randomly place them in places where doors can be, that is cells with only two exits. The black dots are just the top left and bottom right. finally the yellow circle is the little player guy who can traverse the maze. Any way the important thing is Yay a maze!

Next up I wanted to carve out the maze to create caverns as they are way more interesting then mazes. Well depending on you point of view. I wrote an algorithm that works the rooms in the maze. A room is all the cells on the same side of a door. I then remove what I termed Dangling walls. That is a wall that can get from one side to the other by taking only 3 steps. I actually write the algorithm in an iterative way so I could gradually remove the wall if I wanted to. Here is how it looks

Hollowing Out The Rooms

Finally I want to do something to make the walls look a little less random. I to do that I wrote another algorithm that would look for cells in a room that only have one entrance/exit and give them to random neighbouring rooms. This actually work quite well, see the image below.

Trimming Cell That Stick Out (Where Possible)

For reference here is a picture before I trimmed the nodes, notice how a number of the rooms have been squared off a little bit. I don't move doors so there is a limit to the amount of squaring off I can do with this algorithm.

Original Maze Before Cell Trimming

Overall I am quite happy with the results. The caverns or rooms produced are not exactly the most awe inspiring designs but the are suitable for the next topic I wish to explore. More on that in a minute.

First some more thoughts on Dart. Dart still feels pretty nice to code in. Just recently the IDE gained the ability to optimise the Javascript produced so the output file is measure in Kb rather than Mb for the canonical hello world example. This means people can actually deploy some of their Dart experiments.

As a language it feels like coding at a slightly higher level than Java. It is comfortable and I rarely think about the actual language. I do tend to find myself putting in the types a reasonable bit most to get code completion to work in the IDE. I am quite happy to use var or say List instead of List and so on. I guess what I am saying is there is no problem in my mind in jumping between typed and untyped parts of my code. I certainly appreciate the arguments that types act as a form of documentation. 

The IDE is simple at best but functional and is productive to use. Give it another year an it should start to actually be pretty. At the moment there is not real debugging support so you are left printing to the console or trying to make sense of the outputted Javascript.

So far my bugs have been simple enough that printing to the console was all that I needed to do. I did use the JS profiler built into chrome as many of my algorithms are written to be simple and slow. Currently generating a maze takes a reasonable number of seconds. It is pretty easy to use the profiler and while the function names have been mangled a fair bit they are not beyond use.

So my next stage is speed up my maze generation with the aim of it taking less than a second on my laptop. Then I want to experiment with dungeon generation a little further by creating some algorithms that will lock doors and hide keys the player has to find to progress. Perhaps levers to pull and so on.

Early Nov 11 Update

Another small update....

It has been a bit of a strange few weeks as much of my time has been consumed by family stuff. So my coding time has been fairly limited. I have also been jumping around a fair bit on topics.

I have taught myself a little bit of OpenGL, The modern type, I used to know some way back when using GLBegin and GLEnd were all the rage. I am happy that it all turns out to be quite simple. Not being scared of matrices and vectors probably helps this.

Days Until has had another little iteration. Nothing spectacular and I have squashed the final bug that was holding up a release. So that should be posted soon.

Finally I having been taking a look at Dart. The new language from google. I quite like it even though it is in a very early state. I only took a look when they released the first iteration of an IDE for it.

I don't understand why they have a stand alone IDE rather than a plugin for eclipse. It is obviously based on eclipse. Perhaps there is a plan for the IDE that involves doing stuff that you cannot do as a plug in. That is perhaps they need to alter the core in some way. I am not to worried about it either way. Being in an Alpha state it does not contain much in the way of features.

So how have I found Dart? As I said before I kind of like it. I think this is because if you know Java you kind of already now most of the language. Well except for lambda functions that Dart has which is a definite plus point. There are probably other features I could think off that would be cool to add but then that would add complexity to the language, at the moment it is quite simple.

A lot has been said about Dart's type system most not nice. I have used enough different languages to not really be concerned either way. I perfer to have types if possible but can see why people prefer to not have them. I suspect once the tooling for the language get there it could be a nice solution.

I have not looked at performance or deployment but I imagine both will be OK once they get a few iterations into the life of Dart.

I guess so far Dart gets a B+ in terms of my feeling about it. I like it but it's not yet amazing. I get the feeling it has the potential to be quite a productive language.  Time will tell I guess.

Early Nov 2010 update

I appear to have two strands going on at the moment in programming. Both are web based and involve JavaScript. Ok I also went on a 4 day Stag do plus travel time so the past couple of weeks have not be the most productive in terms of get code produced.

The first little project is a little game (Hamla) using JavaScript and HTML5 canvas. In all honesty the game is more about developing some technology to write other games in. The game is actually finished in terms of game play and features but I want to add at least a little bit of polish to it. So I have started to write a little SVG loader/render to allow then them to be drawn on the canvas. I am not planning on implementing the full SVG spec just the bits I need to load some sprites.

Given my lack of experience in JavaScript it feels like it is actually going quite well. At some point I will publish it a website somewhere.

My synth app may look like it has ground to a halt but that is not the truth. So far I have just been adding features without much concent for code quality and that is gradually going to have to change. Part of this work was getting a very simple graph library put together so I can handle gradually work toward users being able to add nodes and links. 

As this was an independant bit of code I have designed it as such. It means the minimal requirements for what I need and is a simple as it can possibly be. The means it is hugely ineffiecient but is back up by a reasonable number of unit tests so should performance ever become an issue I can fix that. Of course if the profiler never highlight this module as being a problem I won't have wasted time. I expect the graphs to be pretty small so simple, bug free, code is currently prefered.

Android Update

I have found my focus again when it comes to android programming and started to work on a simple little game. It is a clone number puzzle game I used to play as a child on an amiga. At the time I sunk a fair amount of time into it so I figure there must be some addictive element into it. That said I will be happy if it get 3 star ratings and a 1000 downloads. I suspect my visual standards will not be that great.

I also put out a small update to DaysUntil and promptly added a Force Close to the code. For those that don't know a force close is the android name for it crashes. Oops!

I also continue a journey into lisp by continuing to watch the original structure and interpretation of computer programs over on you tube. It is fascinating to watch just to see how little we have progressed over the past 24 years. Perhaps monads to help solve the issues with pure functional languages and perhaps we empahsise more unit testing where they prefered incremental testing but I suspect in bigger systems they would have used the concept of unit testing even if they don't mention it in the lectures.

The alternative view point is perhaps they were just that far ahead of the curve. In which case I would love to know the ideas that are being taught at MIT at the moment.

C++ And Bug Finding Tools

I work as a C++ programmer and one thing that always amazes me is the difference in compilers, we use compilers as a way to catch some of our errors along with the obvious use of compiler! We compile on Windows (VC and MinGW), Linux and Solaris. Each one seem to have different analysis tools built into the compiler that picks up different bugs/warnings, sometimes only the Solaris catches shadowed variables and so on.

Yes we have all the compilers set up to some pretty high warning levels and warnings are treated as errors. As we have a large test suite we can run against our code we also get to catch a lot of edges cases from compiling and running on different platforms. The benefits to Windows (where our main user base is) users is reasonable and certainly worth the programmer time spent maintaining our own internal build of the alternative operating systems.

Ok the company I work at does ship on Linux and Solaris but the builds we do locally are internal to our office, the main builds are handled elsewhere.

We also run over the code with the Microsoft static analyser, Coverity and recently we have started to use valgrind. They all catch bugs that would be difficult to find otherwise.

We obviously have lots of other bugs to deal with but this set of tools certainly helps. Some may think we are excessive but once the systems are in place compiling and running our tests is not a time consuming task. Even valgrind can be left to run over night so time is not wasted.

So if you work in on C/C++ application and the code you work in standard C/C++ (not gui stuff) cross compiling help you catch bugs and gives you access to valgrind. Plus running linux is so easy now you have virtualisation.