Monday, 19 September 2011

Calling Lisp with Mathlink

My final example of using MathLink is a tutorial on calling Lisp code, from Mathematica.  The implementation I will show is probably quite crude to a computer scientist.  However to someone interested in producing a solution I think it is useful.  The method I put together is sufficiently general as to allow Mathematica to call any executable file.  This method is useful for calling code written in a language that is not easy to integrate in C.

One language that is not very easy to call in C is Lisp.  First of all let me be clear, it is possible to use a foreign function interface in C to call Lisp.  However it is very difficult.  Limiting myself to the open source Lisp compilers I found no easy option.  I did find the following quote from the Steel Bank Common Lisp documentation,

"Calling Lisp functions from C is sometimes possible, but is extremely hackish and poorly supported"

Thus I was searching for an easy way that gave me results.  I also felt that if one were to work out how to write some particular program in Lisp, that C could call, then the problem would not be completely solved.  More complicated data structures, or any modest change to the Lisp program, could require more hacking to make the C calls work okay.  My method, whilst less elegant and a little slower, does not suffer from these problems.

First of all I will run through an example program, written in Lisp, that can be called by C as a system call.  Then, using MathLink, I will show how to make Mathematica call the C code and return the output from the Lisp program to the Mathematica program.  I wish to emphasise that this method is not in any way particular to Lisp, it will work for any executable file.  After I have given an example I will briefly discuss the efficiency of the program.  I will attempt to argue the solution I propose is quick enough for many applications.

Below is a some Lisp code for a factorial function,

(defun factorial (x)
  (if (eql x 1) 1 (* x (factorial (1- x)))))

(defun main (argv)
  (with-open-file (s "$Path/output.dat" :direction :output :if-exists :supersede)
          (format s "~A" (factorial (read-from-string (first (rest argv)))))))

This code performs two basic tasks.  Firstly it takes the argument supplied at runtime, called argv, and, secondly, writes the factorial of argv to the file located at $Path/output.dat.  I will not explain the complete details of the Lisp code.  What is important is that the code computes functions of variables supplied at run time and writes the result(s) to a file.  If such an arrangement can be realised in any language then this tutorial shows how to run such code from Mathematica.  One important note is the location of the output file, here called output.dat.  When using Mathematica the default path will be changed.  Therefore it is a good idea to specify all paths in full in all files/code.  This will avoid any bugs occurring when trying to run code from Mathematica that causes files not to be found.

Now I will take a short detour to explain how to obtain executable files from Lisp.  If you are not interested in Lisp then please skip on to the C code.  My solution to creating Lisp executables was to use the program BuildApp.  You can obtain BuildApp from their website, here.  There are other options available to create executable files from Lisp.  After installing BuildApp you need to type in to a terminal,

terminal $ BuildApp --entry main --output factorial --load factorial.lisp

What this does is to compile the file factorial.lisp into an executable called factorial.  The flag --entry main is a technial point that tells the program how the executable will be ran, here it is as a top-level function.  The effect of doing this is that, using the Lisp code given, one can now type in to a terminal,

terminal $ ./factorial 5

and the output, of 120, will be written to a file.  This is the first goal for any language, Lisp or otherwise (that can not be embedded into C): producing an executable that can receive arguments from a terminal, as shown above.  How to do this depends on the compiler and language used.

Next some C code is required that MathLink can talk to.  From previous blogs we know that what is required is to write a C function that takes the arguments and returns the relevant output.  I wrote the code,

#include <stdio.h>
#include <stdlib.h>
#include "mathlink.h"

int f(int x) {
    FILE *fp;

    char output[100], fstr[100];
    int i;

    sprintf(fstr, "$Path/factorial %d", x);

    fp = fopen("$Path/output.dat", "r");
    fscanf(fp, "%d", &i);

    return i;

int main(int argc, char *argv[]) {
    return MLMain(argc, argv);

As before you should recognise the MathLink program code at the very bottom and the mathlink.h header file.  In the function f an integer x is converted in to a character.  This is concatenated to the path of the executable by sprintf(fstr, "$Path/factorial %d", x);.  In this case the executable is called factorial.  This is echoed to the terminal by the syntax system(fstr); and fstr is the string that the terminal will (try to) execute.  In short, the C code writes to the terminal,

terminal $ $Path/factorial x

where $Path is the complete path to the executable factorial and x is the variable whose factorial we are computing.  So now in the output file is the resulting factorial of x.  The C function, f, now reads the file output.dat and returns the number it finds.  I admit this is a crude implementation but its simplicity means that it does not matter how the executable was written.  It could be a terminal script, or anything.  This means that the solution I have written really is very powerful.

There is now the formality of writing a template and compiling the template and C code together.  Here we can use the script mcc.  The template for the function f is,

:Function:      f
:Pattern:       f[x_Integer]
:Arguments:     {x}
:ArgumentTypes: {Integer}
:ReturnType:    Integer

Saving this file as it is just a task of using mcc.  In to a terminal type,

terminal $ mcc f.c -o result

Installing this to Mathematica gives the ability to call the Lisp program.

I'd like to close with some general comments on efficiency and the power of this solution.  Firstly the solution is crude and slow.  However the speed is lost by calling the executable then writing and reading to file.  These tasks are only performed once.  The cost will only be a small fraction of a second.  So in a program that takes a minute to run, for example, one would not mind wasting a quarter of a second to be able to code in any language whatsoever.  Such freedom might well allow one to use a program that is many hours faster than an equivalent Mathematica program.  Or perhaps the ability to code in some particular language, as compared to C, might offer a solution to the problem in days rather than weeks.  As a result I do not consider the efficiency loss as an issue for most situations.  I also consider the code I have given to be extremely flexible and powerful to a range of users.

No comments:

Post a Comment