Sunday 8 December 2013

The Growing Pains of TDD : Developing the Bare Minimum to Pass a Test

Hello Readers!

The first post in my TDD blog series is about developing the bare minimum to pass a test!

According to the TDD paradigm, you must only develop the absolute bare minimum required to get a single test passing.  The motivation of this being that you never write anything that is left untested.  Edge cases and specifics should be dictated by a test before they are developed on.  In many cases, this can feel trivial and nonsensical.  Say you're writing a boolean function 'isUserAuthorized(UserInfo userInfo)' .  Your first test might be to check that the user is authorized to perform some action because their role is 'top dog'.  However, the absolute bare minimum to do this is basically just to return 'true' without any other logic.

This kind of thing leaves me itching all over.  I can appreciate that maintaining this now passing test during further iterations of TDD is at some point going to result in proving some meaningful functionality.  However, I can't help but implement something reasonable before I can truly say this test has 'passed'.  In the example above, I might actually look into the userInfo object for the role 'top dog', return 'true' if it is present and false otherwise.  It frustrates me to deliberately write code I know is actually useless, just for the sake of passing a test.

Rather than getting a test to pass for the sake of getting a test to pass, I'd rather have the test pass knowing that the actual logic that the test is supposed to prove is working is in fact working.  In more complex scenarios, I have found huge value in developing as per this more pragmatic definition of the 'bare minimum'.  It allows you to understand many of the technical details you will need to know about in order to correctly implement the rest of the requirements of the class/interface you are testing.

To illustrate my point, lets say you're auditing a web service query that includes a SAML assertion.  Your auditing feature needs to record all relevant information about the User performing the query as well as the query itself.  Your first test might be to ensure that a query with a single required parameter, 'dataId' is audited.  In order to implement bare minimum required to pass this test, you need to address the following concerns:
  • How do I get the relevant User information from the incoming query?  Generally, SAML headers are used to create a Session for the current web service transaction, so how do I get access to this Session?
  • How do I go about retrieving the 'dataId' query parameter from the request?
  • How do I appropriately format this dataId query parameter in my audit log?  Perhaps in its present state, it is not very readable to someone viewing the log.
  • How do I even audit any of this information once I've gotten it?  Is there some library or service I can make use of to do this for me?
Complex situations like this are where you see the greatest value in developing the minimum.

A great way to solve a big problem is to narrow it down to the smallest possible sub-problem and then to solve it in a quick and dirty manner before moving on.  In this case of this auditing feature, the smallest sub-problem addresses so many issues that are relevant to the remaining set of requirements.  Attempting to write the whole feature set at once while also trying to address these concerns would be exponentially more difficult.

Thus, I have decided to let pragmatism dictate how I follow the 'develop the bare minimum' step of TDD.  I prefer writing the minimum amount of code that I know will become a part of the solution, rather than a bare minimum that I know is destined to be thrown away.  In difficult scenarios such as the above, there is huge value in developing this way as we focus development on the simplest version of a complex problem.  What are your thoughts, readers?  I'd like to update this post with any interesting discussion points.

Next up is a post on the Refactoring pains of TDD!

Shrek

The Growing Pains of Test Driven Development

Hey all,

Earlier this year I realised I had been struggling with a few development concepts that seem pretty mandatory to the market.  I've since made it a point to look out for design patterns, think about writing 'testable code' and applying sensible refactoring.

One concept that I feel is surrounded by a lot of disagreement and confusion is TDD.  There are developers who love it and can't agree on a TDD approach to follow and there are those who just out right hate it.  The general development market seems to think of it as a god-send and that every developer needs to be an expert at it, but there are developers out there who follow a develop-then-test methodology and still end up with very robust, well tested code.

Naturally, I began to ask the "why does this matter, seriously?" questions and not long after decided that the best way of figuring it all out was to follow an uncompromising TDD approach for all of my development.  The only sensible way of making my mind up was to throw myself to the TDD ocean and to see if I drowned.  Barring some annoyances, I'm actually handling it quite well and I've come to see a few huge benefits from using Test Driven Development.  Working directly from requirements, constructing testable interfaces and the huge step up in knowledge of testing libraries like Mockito and Hamcrest have been some of the advantages I've witnessed from using a TDD approach.  However, these advantages haven't come without serious frustrations.

In a series of posts, I'll glace over some of the hurdles I've had to overcome in my TDD adventure, and the lessons I've learned as a result!

The first post, coming very soon, is about 'Developing the Bare Minimum to Pass a Test'.

Stay Tuned!

Shrek

Saturday 9 November 2013

John Cleese on Pair Programming

Hello Readers!

I'm feeling philosophical today, so rather than sharing with you some advice on solving a particular problem, I'm going to touch on why I think Pair programming is fantastic, and how English Comedian John Cleese helped convince me.  As you are presumably aware, Pair Programming involves one programmer who drives and one who observes, critiques and thinks ahead.  The advantages of this style of development are said to range from increased morale to increased code quality at the expense of development time.  But why?

I had always attributed the increase in code quality to the simple explanation of "two sets of eyes are better than one".  Perspective surely helps you spot mistakes and come up with more optimal solutions; a common sense anecdotal conclusion that lends itself to encouraging the use of Pair Programming.  However, after watching John Cleese - a lecture on Creativity I came to realise that the Pair Programming dynamic is in fact rooted in the fundamentals of Human Creativity, and that it is to an extent, a necessity for the development of great software.

I'd encourage you to go ahead and check out the video as I think it is an essential watch for anyone.  However the point I'd like to hone in on is Cleese's idea of the "Open" and "Closed" mental modes.  The Open mode being a state of mind in which creativity is possible and the Closed in which is it impossible.

Cleese describes the Closed state as a mode in which we are focused on a task at hand.  In this mode we are purposeful and determined but cannot be creative.  He goes on to describe the Open mode in which we are less focused, more relaxed and as a result more inclined towards curiosity, playfulness and as a result, creativity.

In his discussion he concludes that both modes are a necessity for any creative pursuit:
We need to be in the open mode when we're pondering a problem but once we come up with a solution, we must then switch to the closed mode to implement it. Because once we've made a decision, we are efficient only if we go through with it decisively, undistracted by doubts about its correctness.
The truth of Cleese's take on creativity can easily be observed in the creative pursuit of Software Engineering.

I often find myself completely zoned into a task, eyes glued to walls of code and headphones blasting my favourite Meshuggah track.  Although this state of mind is fantastic for productivity, I've often found that being so focused has allowed me to lose awareness of the big picture and how my changes propagate through a code base; sure signs of being sucked into Programmer Tunnel Vision.  In this state we can quite easily make questionable design decisions or introduce bugs that are overlooked in the shadow of a successful current task.

Thankfully at Orion Health, we're pretty strict with Code Reviews.  I've often shown a supposedly complete task to a peer only to find my relief shattered by their observation of a new edge-case to test, a scenario I didn't think to cover or the potential for a bug, leaving me wondering how in the world I even began to think that my work was complete.  Similarly, I've often found myself dishing out a grilling at code reviews, often spotting issues that seem very plainly obvious.

Going back to Cleese's mental states, we can see that the concept of a Closed mode marries quite well with the idea of Programmer Tunnel Vision.  Staring at a screen and blasting your ears with Death Metal seems to work wonders for productivity at the cost of not always asking the "what if" questions and considering the alternatives.  Being in a mindset of productivity makes it difficult to be creative.  However, casually strolling over to a colleague's desk and discussing their latest code commit seems to fill you with a million constructive questions and thoughts about what their solution may lack.  Being in a relaxed mindset free from pressure allows creativity to flow.

It seems the ideal Super-programmer would be able to zone in to a problem and zone out to the big picture at will, switching between the Closed and Open mental states to facilitate the optimal balance between unadulterated focus and playful creativity.

From here, we finally arrive at the point.  Pair Programming facilitates exactly this by separating the zone in from the zone out using two developers.  The Driver is afforded the freedom to focus on the implementation of a problem, while the Observer is afforded the freedom to consider the big picture.

As a driver I've found silly mistakes and misunderstandings cleared rapidly with a second pair of eyes looking over my shoulder.  I've found the observer realising issues with my designs that result in complete changes in direction.  As an observer I've found myself constantly considering how the driver's changes propagate and what alternatives to consider.  Where the driver focuses on the implementation of the solution, the observer can sit back and let their curiosity do its work.

Pair Programming allows for the creative pursuit of Software Development to occur simultaneously in the Open and Closed mental states.

In addition to now having a deeper understanding of my many creative pursuits, watching John Cleese's presentation has solidified my advocacy for Pair Programming.  I cannot speak for the exact situations in which it is practical and which pairs of colleagues would make great Pair Programming teams.  Perhaps that's a discussion I'll return to in a future blog post.  However, understanding its effectiveness in terms of the fundamentals of Human Creativity makes me think it should be exercised whenever possible.  At the very least, Code Reviews should be mandatory.  Although they may not be as thorough, Code Reviews do allow for the same relaxed mode of thinking for an onlooker.

That's it from me today!  Be sure to check out  John Cleese - a lecture on Creativity in full for more invaluable perspective on creative and professional environments,

Cheers,

Shrek

Wednesday 10 April 2013

Notes on Remote Debugging in Java

Hello Readers,

Been a while!  This post will cover how to enable remote debugging in a java application and hook Eclipse's debugger onto it.  It'll also cover the explanations of all those little details that can really bite you if you don't understand them.  If you're looking to debug issues in a testing or production environment and are feeling blind without being able to snoop through your code then you've hopefully come to the right place!

Enable Remote Debugging


Remote debugging of a Java application isn't enabled by default.  You need to run the application (or Target Java VM) with certain Java properties that enable and expose the debugging functionality.  There are many properties we can use to configure Java debugging on the VM's end but for this post, we'll stick to one standard set of properties that suffice for allowing the Eclipse debugger to connect.  When running the Target VM, these properties look like:

java myApplication -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=9000,server=y,suspend=n

What these properties mean (found from around the web) are as follows:
  • -Xdebug : Enables debugging support in the Java VM
  • -Xnoagent: Disables oldjdb, a legacy Java debugger that has now been deprecated
  • -Xrunjdwp: loads libraries required for Java debugging
    • transport: specifies the manner in which the VM and debuggers will communicate.  For instance, setting transport=dt_socket means that the debugger and the VM will communicate socket to socket via a TCP/IP connection.
    • server: specifies whether or not the Java VM will act as a 'server'.  If server=y, the VM will expose a port for debuggers to attach onto.  If server=n, the VM will instead attach onto a specific debugger application.
    • address: If server=y, this is the address that the VM exposes to debuggers.  If server=n, this is the address of the specific debugger that the VM attaches to.
    • suspend: specifies whether or not the VM should wait for a debugger to attach before starting
Thus, in our example:

java myApplication -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=9000,server=y,suspend=n

What we have done is:
  • enabled debugging for the Target Java VM
  • exposed it as a server at port 9000, so that any debugger can attach to the VM at this port
  • ensured that the VM will not suspend while waiting for a debugger to attach

Ensure that the Classes of Interest in the Target VM have Source Attached


Java debugging seems to work on a file line or method signature basis.  If the environment you are running happens to be in a production or testing environment with Obfuscated source, then it will be near impossible for your debugging application to break when you want it to break.  The approach I've seen used at work has been to:
  • Stop the target VM
  • Replace any obfuscated classes (in the form of a .jar library), with non-obfuscated classes
  • Start the target VM
This way the source for your class files in Eclipse will match those in your Target VM.

Hook the Eclipse Debugger onto Target VM


To use Eclipse as our debugger:
  • In Eclipse, open 'Debug Configurations' (found in the drop down menu next to the bug icon)
  • Create a new 'Remote Java Application' configuration
  • Set the 'Project' to the Eclipse project that contains the classes you want to debug.  I believe this project determines which classpath the debugger will be looking through.  Thus, classes from projects referenced by this project should also be able to be found.
  • Set Connection Type to 'Standard (Socket Attach)'.  This matches our dt_socket option set on the Target VM.
  • Set the Connection Properties:
    • Host: the host of the Target VM (e.g. localhost).
    • Port: the port that the Target VM has exposed for debugging.  This should match the address option we set for the Target VM earlier.
  • Hit 'Debug'

Set Breakpoints on Lines... not on Method Signatures!


Setting breakpoints on Method Signatures then attaching the debugger seems to cause the Target VM to slow to a crippled pace, pretty much killing any chance you have of investigating your issue.  Apparently this happens because Method Breakpoints are "expensive to evaluate" (http://devnet.jetbrains.com/docs/DOC-23).  Avoiding Method Signature breakpoints can save you a lot of pain :).

References


Check out the following sites if you want to learn more details: