Evaluating Hacle

We evaluate the usefulness of Hacle over two criteria. Firstly, we measure its completeness by translating a range of Haskell programs (mostly from the nofib benchmarking suite). Secondly, we measure the performance of these programs as compiled by (1) the GHC compiler and (2) the pipelined Hacle then Clean compilation system.

In evaluating Hacle under the first criteria, we draw your attention to the result below which shows that Hacle can translate itself. Hacle is written in approximately fifteen and half thousand lines of Haskell code and includes a full Haskell'98 parser and Clean pretty printer.

In evaluating Hacle under the second criteria, we draw your attention to the results below which show the Hacle then Clean compilation system producing faster executables that GHC. However, there are also severals results showing GHC producing faster executables.

Program Instance Ex. Time (GHC) Ex. Time (Clean) Notes
Hacle mate 1.81s 2.67s ovl,getArgs,rat
digits-of-e2 2000 5.41s 1.23s ovl,getArgs
queens 11 3.23s 1.32s getArgs
puzzle default 0.76s 1.07s  
wheel-sieve1 100000 2.66s 6.91s getArgs
mate shinkman 2.88s 1.51s getArgs
cryptarithm1 default 3.41s 5.62s  
life default 0.9s 0.45s  
last-piece default 8.3s 15.82s rat,heapsize=16mb
classify default 0.16s 0.23s ovl
infer default 0.32s 1.4s ovl
reptile default 0.066s 0.143s heapsize=16mb

Note: the choice of programs used above may seem quite random. I have no real explanation for this other than the fact that some nofib programs aren't suitable for hacle, i.e. those programs which even hugs doesn't recognise as standard Haskell'98, and those which don't conform to the limitations of Hacle. By no means have I attempted to translate every program in the nofib suite!

In the table above we have a field called ``Notes''. Notes represent the modifications required to a program so that it conforms to the subset of Haskell'98 that Hacle supports.

ShorthandMeaning
ovlManually resolve internal overloading of numbers.
getArgsManually remove requirement for getArgs.
ratManually remove requirement for rational numbers by using doubles instead.

Test Conditions

  • We used version 5.04.2 of the GHC compiler. We compiled programs with the command ghc --make -O2 prog
  • We used version 2.1 of the Clean compiler. We compiled programs with the command clm prog after, of course, prog had been translated by Hacle.
  • We used the UNIX time command to time the execution of programs (i.e. the ``real'' time)