Hacky Haskell monadic testing

Question

Edit for posterity: The library you're looking for is Hspec.

I recently started writing Haskell tests with HUnit. My general setup looks something like this:

import Test.HUnit

import Widget (foo, bar)

tests = TestList [ "foo" ~: testFoo
                 , "bar" ~: testBar
                 ]

testFoo :: Test
testFoo = TestList
  [ "with even numbers" ~:
    4 ~=? foo 4
  , "with odd numbers" ~:
    0 ~=? foo 5
  ]

testBar :: Test
testBar = TestList [ {- omitted -} ]

The semantics of this are fine with me: I'm trying to do expected–actual testing, not property testing like Quickcheck does. However, there are two things that I don't like about the syntax:

• the list syntax feels really clunky, and
• I have to write "foo" once and testFoo two times (three counting the type annotations); while I could inline this, that would make the lists even clunkier.

My goal was to be able to write tests like this:

import Test.HUnit (Test, (~=?))
import Describe (toTests, (...), (~:))

import Widget (foo, bar)

tests :: Test
tests = toTests $ do
  "foo" ... do
    "with even numbers" ~:
      4 ~=? foo 4
    "with odd numbers" ~:
      0 ~=? foo 5
  "bar" ... do
    "with true" ~:
      10 ~=? bar True
    "with false" ~:
      -10 ~=? bar False

I managed to accomplish just that! But to do so I had to resort to what I consider to be a pretty ugly monad. Here's Describe.hs:

module Describe(group, describe, toTests, (~:), (...)) where

import qualified Test.HUnit as H

data LeftList l r = LeftList [l] ()
                    deriving (Show)

instance Monad (LeftList l) where
  (>>=) = error "LeftList does not support binding; use (>>) instead"
  (LeftList xs a) >> (LeftList ys b) = LeftList (xs ++ ys) b
  return x = LeftList [] ()

group :: String -> LeftList H.Test () -> LeftList H.Test ()
group s (LeftList xs ()) = LeftList [s H.~: xs] ()

(...) = group
infixr 9 ...

describe :: String -> H.Test -> LeftList H.Test ()
describe s x = LeftList [s H.~: x] ()

(~:) = describe
infixr 0 ~:

toTests :: LeftList H.Test () -> H.Test
toTests (LeftList xs _) = H.TestList xs

Obviously, the ugly part is creating this data type that has an unnecessary second type parameter and isn't actually a monad! That is, my implementation of the do-notation is incomplete; sequencing works fine, but if I wrote x <- "foo" ... do { } I would get the error here.

I don't really mind that I'm shadowing HUnit.~:, although it's not optimal.

So, my question: what's the cleanest way to get the desired test case syntax without making the monads cry?

Answer 1

Firstly, I would advise you to look at how blaze-html implements their monads for HTML templating. They are doing something very similar to what you want to do.

You can simplify your definitions a bit by removing the second field from the LeftList constructor:

data LeftList a r = LeftList [a]

Then every place where you use the constructor LeftList you can omit the (now) extraneous (), e.g. the >> definition simplifies to:

  (LeftList xs) >> (LeftList ys) = LeftList (xs ++ ys)

You should also make group, describe, and toTests more general by using a type variable instead of ():

group :: String -> LeftList H.Test r -> LeftList H.Test r

This makes these functions valid for any return type r - not just (). Of course, you don't care what the return type is anyway. But GHC cares - and allowing a general return type might help with type checking.

It actually is possible to define bind for the LeftList monad -- just pass in undefined or error "...":

  (>>=) (LeftList xs) f = let LeftList ys = f (error "LeftList does not support binding")
                          in LeftList (xs++ys)

An error will occur only if the parameter to f is actually evaluated.

For instance, this will not throw an error:

"foo" ... do
   x <- "with even numbers" ~:
           4 ~=? foo 4
   "with odd numbers" ~:
     0 ~=? foo 5

because the value x is never evaluated.

Finally, with GHC 7.10 you will also have to define Functor and Applicative instances for your monad:

instance Functor (LeftList a) where
  fmap f left = left

instance Applicative (LeftList a) where
  pure _ = LeftList []  -- should be same as return
  (<*>)  = undefined