More emacs craziness

I was going to write a GreaseMonkey script for emacs keybindings in gmail until I came across this. I'll have to try loading gmail in emacs instead!

EmacsVM

Thomas Guest has a great blog article on Emacs - the kitchen sink. It got me thinking...

I usually think of Java and Perl being portable languages. In that I mean that there is an interpreter/machine for most environments that will allow the languages to be written in a single dialect. That is contrasted to a C flavor where the source might have to be modified/generated for the architecture.

Starting today, I am adding Emacs/elisp to the list of "Platform Independent VMs" :) It runs on many popular OSes.

Autotest

Autoconf/Automake tools have a utility called Autotest. The software is alpha and subject to change. The disclaimer reads:

     "N.B.: This section describes an experimental feature which will
     be part of Autoconf in a forthcoming release.  Although we believe
     Autotest is stabilizing, this documentation describes an interface which
     might change in the future: do not depend upon Autotest without
     subscribing to the Autoconf mailing lists."

There is also a blurb on that page discussing reasons for autotest over DejaGNU, which is worth the read. The rest of the manual is linked off there. Additionally, there is a related, but different version of the manual available at EPITA Research and Development Laboratory that has more examples.

Between the two sources, and after looking at GNU tar source, which uses autotest, I was able to reverse engineer the quick way to hook this into an Autoconf package. Here is how I am doing it; 99% canned and "borrowed" from GNU tar source:

1) Create a package/tests directory.

    ~/myPackage/
                configure.ac (required by autoconf)
                Makefile.am  (required by automake)
                tests/
                      atlocal.in  (environment definitions)
                      testsuite.at (test execution definitions)
                      Makefile.am  (bindings to make check)

2) The top level configure.ac needs to contain the following macros:

    AC_CONFIG_TESTDIR(tests)
    AM_MISSING_PROG([AUTOM4TE], [autom4te])
    AC_CONFIG_FILES(tests/Makefile tests/atlocal)

3) The top level Makefile.am needs to contain the following:

    SUBDIRS = tests

4) tests/atlocal.in defines the environment, and should have at least:

    PATH=@abs_builddir@:@abs_top_builddir@/src:@abs_top_srcdir@/build-aux:$top_srcdir:$srcdir:$PATH
    XFAILFILE=$abs_builddir/.badversion
    trap "test -r $XFAILFILE && cat $XFAILFILE; exit $?" 1 2 13 1

5) tests/Makefile.am is used to bind testsuite to the make check rule. It contains:

    EXTRA_DIST = $(TESTSUITE_AT) testsuite package.m4
    DISTCLEANFILES       = atconfig $(check_SCRIPTS)
    MAINTAINERCLEANFILES = Makefile.in $(TESTSUITE)
    
    ## ------------ ##
    ## package.m4.  ##
    ## ------------ ##
    
    $(srcdir)/package.m4: $(top_srcdir)/configure.ac
            {                                      \
              echo '# Signature of the current package.'; \
              echo 'm4_define([AT_PACKAGE_NAME],      [@PACKAGE_NAME@])'; \
              echo 'm4_define([AT_PACKAGE_TARNAME],   [@PACKAGE_TARNAME@])'; \
              echo 'm4_define([AT_PACKAGE_VERSION],   [@PACKAGE_VERSION@])'; \
              echo 'm4_define([AT_PACKAGE_STRING],    [@PACKAGE_STRING@])'; \
              echo 'm4_define([AT_PACKAGE_BUGREPORT], [@PACKAGE_BUGREPORT@])'; \
            } >$(srcdir)/package.m4
    
    TESTSUITE_AT = \
     additionalTestsM4IncludedInTestsuiteDotAt.at
    
    TESTSUITE = $(srcdir)/testsuite
    
    AUTOTEST = $(AUTOM4TE) --language=autotest
    $(TESTSUITE): package.m4 $(TESTSUITE_AT)
            $(AUTOTEST) -I $(srcdir) testsuite.at -o $@.tmp
            mv $@.tmp $@
    
    atconfig: $(top_builddir)/config.status
            cd $(top_builddir) && ./config.status tests/$@
    
    clean-local:
            test ! -f $(TESTSUITE) || $(SHELL) $(TESTSUITE) --clean

    check-local: atconfig atlocal $(TESTSUITE)
            $(SHELL) $(TESTSUITE)

    check-full:
            FULL_TEST=1 $(MAKE) check

    # Run the test suite on the *installed* tree.
    installcheck-local:
            $(SHELL) $(TESTSUITE) AUTOTEST_PATH=$(exec_prefix)/bin

6) Lastly, tests/testsuite.at contains:

    m4_version_prereq([2.52g])
    AT_INIT
    AT_SETUP([myApp version])

    AT_CHECK([myApp --version | sed 1q],
             [0],
             [AT_PACKAGE_VERSION
    ],
    [],
    [cat >$[]XFAILFILE <<'_EOT'

    ==============================================================
    WARNING: Not using the proper version, *all* checks dubious...
    ==============================================================
    _EOT
    ],
    [rm -f $[]XFAILFILE])

    AT_CLEANUP

The once all those files exist, then an automake; autoconf; ./configure from the top directory will generate all the necessary scripts. And lastly the 'make check' rule can be run and all the tests defined in testsuite.at will be run. (Thank you GNU tar team and contributors and supporters for that bit of code. It follows the currently existing autotest documentation enough for a dev to fill in the missing pieces.)

As to the pieces that the individual developer needs to complete:

1) List your individual test dependencies (additional test files) in tests/Makefile.am in the following variable:

    TESTSUITE_AT = \
     additionalTestsM4IncludedInTestsuiteDotAt.at

2) You are going to want to write your own tests in your own test files then, following the pattern:

    AT_INIT
    AT_SETUP(keyword)
    AT_CHECK(...)
    AT_CHECK(...)
    AT_CLEANUP

3) Each of those additional test files will need to get m4_include'd into the testsuite.at file with:

    m4_include([additionalTestsM4IncludedInTestsuiteDotAt.at])

And away you go!

Now, I assume that you have read the docs at this point, so here is a quick synopsis of some of the useful features:

• 'testsuite -h' will give you a help message on how to run tests outside of 'make check'.


• 'testsuite -k myApp' (in the code above) will run only those tests flagged with the 'myApp' keyword.


• 'testsuite -l' will tell you the list of tests that will get run prior to running them.


• 'testsuite 10' will run just the 10th test.


• 'testsuite 1 4' will run tests 1 through 4.


• Once the test executes, any test that fails will be in the testsuite.dir/N directory where 'N' is the (ordered) number of the test. In there will be a .log file showing the command run and a diff of expected/actual outputs + exit code. Also, there is a testsuite.dir/N/run script that will rerun just that failed script.