Break away from the debugging cycle of doom in ASP.NET MVC with unit tests

Tags: Patterns, MVC, Unit Testing, Software Development, Patterns & Practices

Aug 15 2011 12:15 PM

The process of using F5 to start a debugging session is a ubiquitous practice in .NET development with Visual Studio. Despite that, developers are noticing and employing other methods of testing and writing software as these methods can provide better results and a more smooth development experience.

The debugging cycle of doom

The entire process of writing some code, compiling, then pressing F5 to walk through code, can be a tedious way to develop software, especially in larger applications. In general, many consider this type of testing as a smoke test. Smoke tests are fragile, error prone, and nowhere near as comprehensive as they should be, except in cases of automated testing (e.g., record/playback).

Consider the following common development scenario: You need to fix a bug in your code that calculates the total for an order. You would normally start debugging via F5, and try to reproduce the problem, so you can get to the code to fix it. This involves only a few steps; however, they're very time consuming, interwoven, efforts:

Reviewing the steps...

  1. F5
  2. Navigate to the section of application as described by the bug report, using the UI as a user would.
  3. Attempt to reproduce the problem by interacting with the UI. This means filling in forms, submitting data, and using links and other page UI elements, again, just as a user would.
  4. Step through and investigate code with debugging tools.
  5. Return to code editor, locate source with error, and modify code. 
  6. Repeat steps 1-5 several times.

Because this is the customary way to debug using Visual Studio, this may not seem time consuming, especially in a small application. However, this process often takes much more time than is needed to write code, fix bugs or resolve issues. It's also rare that developers - even ninja developers - will actually fix the bug on the first shot without causing other bugs, so the process continues over and over again, except now, you're aiming at a moving target. As often as not, it's easy enough to find yourself "going down the rabbit hole", which usually ends up with a rather unpleasant undo checkout command at the end of it all.

You look like this when you use F5

(Yep, it feels like this, doesn't it? Img src: http://www.seussville.com/)

Let's not forget, after you fix the bug, you still need to update the requirements documentation. This step is usually brushed off as a low priority, and then causes the requirements to become out of sync with the code base. As you can see, The F5 Way does nothing to help you code understand requirements, produce documentation, or add any extra business value.

Break away from debugging tedium with unit tests.

Now, consider the same development scenario: You need to fix a bug in your code that calculates the total for an order, but this time, going the unit test route.

The steps...

  1. Run tests.
  2. Inspect failed test and fix the bug in the code it tests.
  3. Repeat steps 1-2 if red. If green, your bug is fixed. [1]

Note/Disclaimer for purists: Depending on whether you're doing TDD or not, the exact process (e.g., test first vs.test later) will be different. This blog post in particular is trying to demonstrate what I believe is more likely to be seen in real world apps based on personal experience - i.e., test later, rather than test first, if they're testing at all! Your mileage may vary.

Not only are there less steps involved with unit testing, but you're also saving a great deal of time. Unit tests also serve as documentation, not only saving even more time, but also while adding value by using unit tests as living, up to date, requirements documentation.

Additionally, try to ensure your tests are isolated to one unit of code (one primary/entry point method a maybe a few supporting ones) as it’s easier to find the defect since you aren’t looking through n layers of architecture, classes and methods to find where the defect is occurring. This helps you test easier, as well as leads you toward writing cleaner code.

Fixing bugs with unit tests

After a user files a bug report claiming that the the program calculates the total incorrectly, you investigate the code at question, which might not appear out of order at first glance.

public decimal CalculateTotal(decimal subtotal, decimal tax, decimal discount)
{
    var tempTax = (subtotal - discount) * tax;
    var total = (subtotal - discount) + tempTax;
    return total;
}

But, how can you know for sure? You can run the application from Visual Studio, using breakpoints and other debugging tools. Since this code doesn't throw an exception, the app appears to be doing what it should. However, an important calculation is incorrect! We can know that the code is correct with unit tests.

A unit test that will verify if the code is indeed, doing what it should be doing. Inside the unit test method that you specify the exact parameter values to test as inputs, and exactly what you expect as a result. Once the test is setup, call the code in question, and then verify the results. The best way to structure the tests by using the fashionable "Arrange, Act, Assert" technique, as described here:

  • Arrange: Declare variables and instantiate objects or mocks needed for the test.
  • Act: Execute the code in question, by calling a method or property of the object.
  • Assert: Use the Assert object to determine if the results are what you expect.

The unit test code below demonstrates testing the calculation for an order using Arrange, Act, Assert.

[TestMethod()]
public void Calculate_The_Total_Of_An_Order()
{
    //Arrange
    Order order = new Order();
    OrdersBLL target = new OrdersBLL(order);
 
    Decimal subtotal = 10M;
    Decimal tax = .54M;
    Decimal discount = 1.0M;
            
    Decimal expected = 9.54M;
    Decimal actual;
 
    //Act
    actual = target.CalculateTotal(subtotal, tax, discount);
 
    //Assert
    Assert.AreEqual(expected, actual);
}

The [TestMethod()] attribute denotes that this particular method is only meant for unit testing. Aside from the [TestMethod()] attribute, the code inside unit tests is the same type of code you would normally write any other day. Code that creates instances of objects, calls methods, and uses assertions and logic.

Unit test classes live in their own code files and projects, separate from other projects in the solution, so they won't get in your way, and you can run them separately from your core projects.

When you run the test(s), you can see right away, which ones passed and which ones failed.

Visual Studio 2010 & MS Test are awesome for unit testing

Note: This sample demonstrates using MS Test, however any good testing framework will provide a test runner that does the same basic thing - run tests and output pass/fail statuses.

Since the test failed, you then need to investigate the OrdersBLL.CalculateTotal() method in the OrdersBLL class, since that's the method this unit test is testing. 

public decimal CalculateTotal(decimal subtotal, decimal tax, decimal discount)
{
    var tempTax = (subtotal - discount) * tax;
    var total = (subtotal - discount) + tempTax;
    return total;
}

The original developer is using the tax argument as a percentage, rather than just adding the pre-calculated tax amount, which is the original intent of this method. Now that you've found the bug, you can modify the code and re-run the tests. 

public decimal CalculateTotal(decimal subtotal, decimal tax, decimal discount)
{
    var total = (subtotal - discount) + tax;
    return total;
}

The unit test passed!

Now that the test(s) show green, you can feel safe knowing that you've fixed the issue.

Although you may have to modify and run the tests a few times, there will be no UI interaction, as the testing framework will output the results quickly and clearly. To reiterate, the time saved compared to F5 debugging are huge, as most tests can complete running before Visual Studio can even load a web application. This means you'll already be in the code making the right modification rather than wasting time fishing around in the UI and guessing.

A programming kitty who uses unit tests on his nap break

(Development feels more like this when unit testing. Img source: cattearoom.com)

This doesn't mean you'll never use F5 again, since you'll still need to run smoke tests, but with far less hunting, pecking, and changing than with F5 debugging. Plus, most developers just like to poke around in the code, tweak, and refactor as well.

Now that you've seen the benefits of unit testing, here are some popular frameworks you can use to get started.

Summary

MS Test: Built into Visual Studio 2010, MS Test [2] is a nice testing framework, especially for those getting started. There's no extra installation required and, of course, it's fully integrated with the IDE, so there's less friction for testing novices.

xUnit: xUnit is a unit testing framework written by ASP.NET team member Brad Wilson [3], and Jim Newkirk. xUnit is easy to use, and works with many Visual Studio add-ins and 3rd party libraries. xUnit also has built in MVC test project support, and works with .NET versions 2.0 and later.

nUnit: nUnit, like xUnit, nUnit is a testing framework that is also easy to use, and like xUnit, integrates with Visual Studio add-ins and 3rd party libraries. nUnit is one of the more mature unit testing libraries available for .NET developers.

Want to compare these frameworks? Check out the unit testing features comparison at the xUnit site on CodePlex.

There will always be at least some learning curve and friction to those beginning anything new and unit testing is no different. However, most developers are quick to get the hang of it, and quick to start enjoying writing tests and seeing results quickly. It does take a shift in mindset, but the payoff is more than worth it, as unit testing is one of the best ways to bake quality into your application from the start.

Resources:

James Bender & Jeff McWherter have paired together to write an excellent book, "Test Driven Development with C#", and at 305 pages, it shows that unit testing and TDD are not rocket surgery!

James & Jeff's awesome book!

The book is an enjoyable read, with great explanations and well written code.

Footnotes:

[1] It is possible that you have written your test(s) incorrectly, but for the purposes of demonstration, this sample assumes the tests are well written.

[2] See the Visual Studio 2010 features to determine if your edition of VS works with MS Test.

[3] Brad is my BFF. :-D

18 Comments

  • Andrew said

    I've had enough conversations with management type people about Unit Tests v. Debugging to last me a second lifetime. Nobody seems to really factor in how much time it takes to debug an app when estimating build time.

    I've written many presentations just like this blog, and to be honest, I've pretty much given up on getting the .NET world to embrace testing. Microsoft doesn't push it enough, current "senior" .NET people (re: VB6 people that moved straight into .NET) discount it and it just doesn't seem to be of an urgency any where I've worked.

    Kudos for a good blog posting though, it's a shame that in 2011 this even needs to be written.

  • Andrew said

    James,

    I'm not sure if there is anything to "help" I'm afraid. There have been a few white papers on the benefits of Unit Testing, but when it comes to pure financials (i.e."how much is this going to save me?") there just isn't much data out there. So when you need to convince a big company that they need to do Unit Testing, there aren't many bullets for the gun.

    Rachel,

    I just saw that you work for Microsoft (I was directed to your blog post directly via Twitter and didn't see your home page). It's nice that you've made this post, but seriously, how is this not smack dab in the center of MSDN? The hardest part about getting .NET shops to Unit Test is the fact that Microsoft doesn't seem to care much about it. Don't get my wrong, I appreciate the posting, but testing is at the core of every other IT ecosystem (well most anyways), why can't it be for .NET as well?

    I know Alt.NET has all but fizzled out, but trying to teach people to code the "right way" is getting more and more difficult without someone taking the ball and running with it. I interviewed 3 "senior" developers today who actually were quite smart about most things Microsoft, but not one had written a single unit test in their career. That's just so wrong.

    Anyway, my apologies for the rambling rant.

  • Rachel said

    Andrew,

    Yes, I totally agree with your seniments. There's not much, if at all, anything with hard numbers. I started looking in the Java space a while back, thinking they might have some numbers, but became distracted with other things at the time.

    Believe it or not, Scott Hanselman recently tweeted that 23% of devs do NOT use source code control, now, in 2011. That certainy gels with my personal experience, and also, is quite a shame.

    This should pop up on MSDN tomorrow or this week sometime - and I'm working with the good folks there to get as much of this type content out as possible. Of course, we still have to balance that with talking/writing about our core products too.

  • Andrew said

    Rachel,

    23% of .NET/MS Devs, or 23% of all devs? I can see within .NET shops, but outside everyone I know uses github.

    I understand that Microsoft is a business, but what I don't get is why they don't want their developers to build as robust software as possible. MSDN is full of horrible, half written examples that contain bugs that developers gladly copy and paste directly into their code. That only makes MS look bad when the thing blows up in production a few weeks later.

    I know it's not Redmond's job to make sure everyone is doing everything "right", but as a customer of Microsoft (and an employer of over 200 developers on Microsoft), I'm disappointed at the lack of good direction they give for anything beyond a Hello World app. Be that by the lack of good examples, or the bizarre release of horrible sample projects like Oxite or the recent Spanish DDD project on Codeplex.

  • Andy Hochstetler said

    Nice post! I've seen a lot of developers that have a hard time transitioning away from the F5 key to real unit testing. It's a shame because the benefits are so many!

    I particularly like your olive branch that you extended to the TDD purists. Like your personal experiences, I've also found that many real world projects are either "Test Later" or "No Tests At All". The latter are usually a mess.

  • Rachel Appel said

    Andrew,

    You make an excellent point. I don't know if it was MS or non-MS Devs he was referring to.

    I can say there are many of us here at MS who do want developers to build as robust of software as possible, and I'm seeing a gradual rise in interest of that persuasion. We do need to promote good practices much more though.

    The product teams are very open to feedback, and would love to hear thoughts like yours. I will pass what you've said along to the VS/ASP.NET folks.

    I'm on this project, which I think you'll like. It's an ASP.NET MVC project created specifically for guidance on how to do things "the right way". We've had multiple drops with lots of community involvement to get as pure as possible.

    Project Silk
    http://silk.codeplex.com/

  • Ian said

    I'm glad to see anything that's going to convince more people to write unit tests. However, they're not going to get very far without a mocking framework and dependency injection. I know these are relatively advanced topics which could scare off a newbie to unit testing, but some introduction of the concepts and benefits, and possibly pointers to further information, would be useful.

  • Eli said

    There are some resources out there that show the benefit, they are just difficult to find.

    Caper Jones's "Software Assessments, Benchmarks, and Best Practices" offers numbers based on incidence of success across > 9000 projects and what factors affected it the most (a subset of the numbers can be found in Code Complete). Rather than showing how much time it saves or how much money it simply says that it is proven to increase odds of success.

    There is also an NC State paper out there called "An Initial Investigation of Test Driven Development in
    Industry" (didn't save the URL, just the PDF, sorry). It has a smaller scale experiment comparing TDD vs non TDD approaches.

    Speaking of Code Complete, there is an excellent section on a debugging experiment in there which could help make a case on the costs of debugging. Basically it showed much, much poorer results than you would expect for programmers finding bugs, number of cycles it took, and the number of bugs introduced while correcting others.

    I also did a small series of posts last year on some of the popular misconceptions of Unit tetsing (Starts here: http://blogs.lessthandot.com/index.php/DesktopDev/GeneralPurposeLanguages/unit-testing-costs-too-much) and the one I usually focus on the most is the comparison between catching N% of bugs at the developer compared to the costs involved in a bug being found at QA or at the customer. Unit Testing isn't just about saving time at the developer, that rework reduction and utilization of less 3rd party time is more valuable.

  • Rachel said

    Ian - Great comments. Definitely, I plan to write more about this. If only I could clone myself - but that would be weird for everyone else. :)

    Eli - Thanks for the excellent information on where to get some numbers from, as well as the links to your posts, that's some good info there!

    The moment I read your second paragraph, I instantly recalled reading that same paper, which I had completely forgotten about. :(

    Here's the URL (opens a pdf) http://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf

    Readers,
    Check out that URL and Eli's blog for more great info on this topic!

  • Raj said

    Well written. This is one of the side benefit you get from Unit Testing. It is important to keep in mind that the intention of writing Unit Tests is not to fix bugs, but to improve the design.

  • Andrew said

    Eli - Yes, I've seen some of those, they're still not great since they don't do apples to apples comparisons (although, it's easy to argue that apples to apples isn't possible in software).

    Rachel - I took a brief look at Silk, but didn't have much time with it. One of the first things I saw was ICountryRepository and got a bit nervous. I fall firmly into the "the repository pattern is almost always overkill" camp, but I'll hold of judgement until I've had a chance to look at it a bit further.

    Sample apps are hard, because they are simple but meant to show off complex problems, which is not an easy thing to balance.

  • Patton said

    "The steps...
    1.Run tests.
    2.Inspect failed test and fix the bug in the code it tests.
    3.Repeat steps 1-2 if red. If green, your bug is fixed. [1]"

    I think this is an over simplification of what is involved and does not give a balanced view on the two processes. I agree unit testing is the way to go but the argument must be balanced. Here you are suggesting that when a bug is reported unit tests will make a big difference to how long it takes to fix that bug. In reality they won't help much with an individual bug but improve the quality of code over the long term and help to prevent the introduction of new bugs.


    If you had the unit test in place for step 2 - the code would never have been released. Following on from that - a bug was reported means their is no unit test to fail so you are still going to have to debug through the code in the usual way until you know what part of the code is causing the error. Is it the method or the parameters been passed in.

    So the proces is more like:

    1.F5
    2.Navigate to the section of application as described by the bug report, using the UI as a user would.
    3.Attempt to reproduce the problem by interacting with the UI. This means filling in forms, submitting data, and using links and other page UI elements, again, just as a user would.
    4.Step through and investigate code with debugging tools.
    5.Return to code editor, locate source with error, and WRITE A NEW UNIT TEST.

    6.Run tests.
    7.Inspect failed test and fix the bug in the code it tests.
    8.Repeat steps 6-7 if red. If green, your bug is fixed. [1]
    9.Test final outcome to make sure TEST and UI match. Now your code is good & your test is good for future regression testing.

    The benefit of unit testing is that you now have a test so when changes are made to this code in the future you have a set of regression tests & development can be done quicker and confidently.

  • Andrew said

    @Patton,

    You're making a pretty big assumption there, that you won't be able to find the issue without stepping through the code.

    What if the bug is "don't allow the user to make property X negative"? It probably wasn't part of the original requirement list, so it was never added in the first place. Let's face it, boundary conditions are where a lot of code fails, so it'll be a common enough problem.

    Sure, there'll be code that you need to step into, but the benefit of Unit Tests are that it's very rare that I ever need to.

  • Rachel said

    Patton,

    Since I couldn't go over every possible outcome, some simplification is needed. However, when you say "Here you are suggesting that when a bug is reported unit tests will make a big difference to how long it takes to fix that bug" I say - Yes! I most certainly am.

    The point of this post is to show folks that as individuals, yes, unit testing can help you save that much time, without being pedantic and getting off track with paragraphs of details.

    Other than that, see what Andrew said in the above comment.

  • Patton said

    @Andrew.. I agree it reduces the amount of time you need to spend in the debugger - its just I usually get defects in the form -> User was doing X.. they got wrong result - investigate... I'm not sure any QA or real user was as specific as "don't allow the user to make property X negative".. Hence more often than not even with unit tests you still end up doing a long investigation which may or may not include debugging just to find the area of code. As @raj said its more about improving design than reducing time in the debugger - for me anyway.

    @Rachel - I'm not against unit testing its just I think their is a danger that if you oversell a particular benefit people will be disappointed when they try it out. Doing away with debugger is not the main benefit of Unit Tests and is not even mentioned here (http://en.wikipedia.org/wiki/Unit_testing)

    Still I like the article, and its goal to make more .Net developers take up unit testing, and hope more is written on this topic in the .Net space.

  • dnn said

    I've found that many real world projects are either "Test Later" or "No Tests At All". I've seen some of those, they're still not great since they don't do apples to apples comparisons.

  • Rachel said

    dnn,

    Yep! That's why I decided to mention that. One day, maybe we'll see more "Test first". Today's not that day though :(

Comments have been disabled for this content.