Org-mode has grown into a popular large and feature-rich application,
on which development often happens at a great pace. Indeed, feature
requests posted on the are frequently developed and
published within hours of the request.
The down-side of such rapid development is that on some occasions, one of the many patches, features, or refactorings being applied to the code-base may unintentionally cause unexpected behavior within Org-mode. This is known as a software regression. The larger and more complex Org-mode becomes, the more you may expect to see such software regressions.
One concern with introducing software regressions is that the regression may not be immediately obvious to users. Indeed it can be hours, days, or even months before an Org-mode user moves to export an important schedule, when "bang" - the regression has reared its ugly head.
The solution to avoiding software regressions? Regression Testing! With a regression test suite the developer can run the tests prior to committing their change, to ensure their additions do not break the regression tests.
Naturally, a regression test suite is only as good as the number, quality, and coverage of regression tests within it, and this is where Org-mode users can help.
So long as you are on an up-to-date release of Org-mode you may run the tests on your local setup. To update Org-mode check out the org FAQ page. Note, the test suite will run on versions of Emacs back to version 22.
A pre-requisite for running the tests is the ERT test library. This may be obtained easily as follows.
In Emacs-22 and Emacs-23 you will need to download the test files _ert.el_ and ert-x.el to your testing directory. This may be accomplished with the following commands entered on the command line.
cd /path/to/org-mode/testing curl -O https://github.com/mirrors/emacs/raw/master/lisp/emacs-lisp/ert.el curl -O https://github.com/mirrors/emacs/raw/master/lisp/emacs-lisp/ert-x.el
Alternatively you may download the files within your browser.
That's it - you may now run the tests.
Emacs-24 includes ERT, so nothing to do.
To run the tests from the command line (without loading your personal configurations) use the following commands (Note, if you have more than one version of Emacs on your system, you may need to include the full path to the required Emacs executable).
cd /path/to/org-mode/testing Emacs -Q -batch -l org-test.el -eval "(setq org-confirm-babel-evaluate nil)" -f org-test-run-batch-tests'
To run the tests from within Emacs itself.
M-x load-file RET /path/to/org-mode/testing/org-test.el M-x org-test-run-all-tests
Thanks to the growing number of test-macros that are being made available, it is becoming easier to write regression tests with minimal lisp knowledge, and you are also likely to greatly improve your knowledge of Emacs-lisp as a result. And the more tests you write, the more confident you may be that your personal use of the many Org-mode features available has not been affected by regression.
And of course, any help toward improving the Org-mode test-suite is highly appreciated.
Coming soon...
Coming soon...
Several useful utilities exist to make the practice of writing tests even easier. These are available from the Org-mode repository as a seperate git sub-module. Assuming you have cloned Org-mode using git, you may install these extra utilities with the following git commands.
cd /path/to/org-mode git submodule init git submodule update
diff
(think export here).The tutorial I'd need to write a test is one which lays out code I could copy and paste to do the following:
TAB
key, or C-c C-c
(some folks might need to be reminded how toI implemented a toy test of org's html export facility using ert.el.
See for the implementation. To run the test
open up org-publish-test.el, and
M-x eval-buffer
. This should load
ert, and run the simple html export test.
LocalWords: ert el EmacsLispExpectations org's