I've been playing around with CGI (Common Gateway Interface) scripts recently, and reading a bit about Lisp. Having written some Clojure, Racket, and Chicken Scheme, but never any Common Lisp, I thought it would be fun to make a little CGI script in Common Lisp - specifically Steel Bank Common Lisp (SBCL).

A CGI script is a program that a web server calls to handle a request. The program takes the request's body in its stdin, and some of the headers and the query string are available to it through environment variables. The stdout of the program is then sent back as the response to the client.

CGI was very popular in the 90s, but has gone out of fashion since. Its biggest disadvantage is that a separate process has to be started for every request, and more efficient solutions overtook it in popularity. Security can also pose an issue, as the scripts are typically not sandboxed or isolated in any way, and CGI scripts would often handle strings poorly and without escaping them correctly.

Over the last few years, we have seen the rise of a similar model of request handling, in the form of "serverless" and "lambda" functions. These use improved process isolation and containerisation techniques to largely overcome the security problems associated with CGI scripts, and the massive improvements in server hardware since the 90s have made the cost of creating a new process for every request much lower (additionally, the writers of serverless runtimes have come up with bags of tricks to make them more efficient).

CGI scripts are wonderfully simple, and something that most programmers of my generation seem to be unaware of.

To run CGI scripts, a web server that can do so is needed. There are many choices, but seeing as python comes with one, and I already have python installed, I used that one.

mkdir -p /www/cgi-bin
cd /www
python -m http.server --cgi

With the server running, any files under /www will be statically hosted, except for the executables under /www/cgi-bin, which will run as CGI scripts to generate results for requests made to their paths.

I added an executable Lisp script under this directory:

touch /www/cgi-bin/hello_world.cl
chmod +x /www/cgi-bin/hello_world.cl

I opened the Lisp file /www/cgi-bin/hello_world.cl, added a hashbang, and added a hello world message.

#!/usr/bin/sbcl --script
(format t "content-length:11~%~%hello world")

Let's break this down:

• #!/usr/bin/sbcl --script tells the system where to find the interpreter for this script
• format is a function that formats a string, and when given a first argument of t, it outputs this string to stdout.
• ~% is an escape sequence to add a newline in the formatted string, much like \n in most common programming languages.

Testing this with curl:

curl 'http://localhost:8000/cgi-bin/hello_world.cl'

The result should be:

hello world

I wanted to do something slightly more elaborate, dumping the request bodies into a file and echoing it back to the client. (Note that because the file I am logging to will be in the server root directory, it will be possible to download it from the server by going to http://localhost:8000/req_dump.log).

#!/usr/bin/sbcl --script

; I imported uiop, a library included with SBCL, so I can use the uiop:getenv function from it
(require "uiop")

; I get the "Content-Length" header from its environment variable, and
; parse it into an integer, `len`
(let ((len (parse-integer (uiop:getenv "CONTENT_LENGTH"))))
  ; I print this same value back to the response.
  ; `~D` marks a place in the formatting template where a decimal integer
  ; will be inserted, and the argument (`len`) after the formatting string
  ; is the value that will be inserted at this point.
  ; The header is ended with two newlines, to indicate that there will
  ; be no more headers.
  (format t "Content-Length:~D~%~%" len)
  ; I then open a file to append to, and loop `len` times, each time
  ; reading a character from the request body, and outputting this to
  ; both the response body, and the file.
  (with-open-file (log-file "req_dump.log"
      :direction :output
      :if-exists :append
      :if-does-not-exist :create)
    (loop repeat len
      for char = (read-char *standard-input* nil nil)
      while char
      do
        (format log-file "~A" char)
        (format t "~A" char))
    ; After the loop, I finish up by adding a newline to the file.
    (format log-file "~%")))

Now, when I send a request with a body, it saves it to the file and echoes back the body, as intended.

curl 'http://localhost:8000/cgi-bin/hello_world.cl' --data 'foo bar'