29 Sep 2015

feedPlanet Lisp

Zach Beane: From Baggers: Tamei categorizes CL forms by purity

Baggers wrote about Tamei on reddit today. It sounds neat to me.

29 Sep 2015 10:37pm GMT

Michael Malis: Zap

This post makes use of places. If you are unfamiliar with how places work, see my post Getting Places.

Many languages provide syntactic sugar for evaluating an expression involving a variable and assigning the result of that expression to the variable at the same time. In these languages you can do something such as the following:

x += 5

The above expression both adds five to the value of x and writes that new value back to x. In this post, I'm going to show you how you can write a macro zap that is a generalized version of this technique. With zap the above example would look like the following:

(zap #'+ x 5)

There are a couple things that make zap really cool. First of all, it can be used with any function. For example, if you wanted to cube the value in x, you could use the following:

(zap #'expt x 3)

The other thing that makes zap so awesome is that it can be used on any place. If you want to use zap on the value stored in a hash table with key 5, you can do that:

(zap #'+ (gethash 5 table) 5)

Now that you've seen how zap is used, here is how it can be implemented:

(defmacro zap (fn place &rest args)
        (temps exprs stores store-expr access-expr) 
      (get-setf-expansion place)
    `(let* (,@(mapcar #'list temps exprs)
            (,(car stores) 
              (funcall ,fn ,access-expr ,@args)))

You should be able to see that the code for zap is eerily similar to that of incf (from Getting Places). They are the exact same except instead of binding the gensym that will hold the new value to one plus the value already in the place:

(,(car stores) (+ 1 ,access-expr))

The gensym is bound to the result of calling the function with the value in the place and all of the other arguments passed to zap:

(,(car stores) (funcall ,fn ,access-expr ,@args))

Although zap is just a nice syntactic shortcut, it is a great example of the crazy things you can do with places.

The post Zap appeared first on Macrology.

29 Sep 2015 4:38pm GMT

25 Sep 2015

feedPlanet Lisp

Quicklisp news: September 2015 Quicklisp dist update now available

New projects:

Updated projects: access, array-utils, asdf-linguist, binfix, birch, bit-smasher, bknr-datastore, buffalo, carrier, caveman, ceramic, cffi, cl-6502, cl-ana, cl-async, cl-base64, cl-bson, cl-ca, cl-containers, cl-coveralls, cl-curlex, cl-emb, cl-ev, cl-fuse, cl-gearman, cl-geocode, cl-gists, cl-glfw3, cl-grace, cl-hash-util, cl-html5-parser, cl-influxdb, cl-intbytes, cl-irc, cl-jpeg, cl-ledger, cl-liballegro, cl-marklogic, cl-messagepack, cl-mlep, cl-modlisp, cl-mustache, cl-opengl, cl-opsresearch, cl-pdf, cl-photo, cl-ppcre, cl-project, cl-pslib, cl-rabbit, cl-read-macro-tokens, cl-rethinkdb, cl-rss, cl-shellwords, cl-string-match, clack, clavier, cletris, clfswm, climc, climon, clipper, clml, closer-mop, clsql, clss, coleslaw, com.informatimago, common-doc, common-doc-plump, common-html, commonqt, crane, croatoan, dartsclhashtree, defpackage-plus, delta-debug, dexador, djula, docparser, eazy-gnuplot, eazy-project, esrap-liquid, fast-http, fast-io, fiasco, fred, frpc, gbbopen, gendl, getopt, hu.dwim.web-server, hunchensocket, inferior-shell, integral-rest, irc-logger, jonathan, jsown, kenzo, kmrcl, lack, let-over-lambda, lml, lml2, lparallel, lquery, lucerne, mk-string-metrics, named-readtables, opticl, osc, parse-js, pgloader, pipes, plump, portableaserve, postmodern, pounds, prove, ptester, puri, purl, qlot, qt-libs, qtools, quri, racer, reversi, rlc, rutils, scalpl, scriptl, serapeum, spinneret, stumpwm, sxql, tagger, trivial-benchmark, trivial-features, uffi, umlisp, umlisp-orf, websocket-driver, woo, wookie, xlunit, xptest.

Removed projects: hinge, py-configvalidator, read-csv.

Hinge was removed because I can't check it out from git any more. py-configvalidator and read-csv no longer build with SBCL, and the authors have not responded to github issues.

To get this update, use: (ql:update-dist "quicklisp")

25 Sep 2015 1:49pm GMT

23 Sep 2015

feedPlanet Lisp

Lispjobs: Clojure developer, EMBL-EBI Hinxton, nr Cambridge, UK

The European Bioinformatics Institute is looking for a Clojure developer:

We are seeking an experienced software developer to develop a new system for the capture of complex experimental data from model organism genetics research. You will join the Non-vertebrate Genomics team at the European Bioinformatics Institute (EMBL-EBI), located on the Wellcome Genome Campus near Cambridge in the UK, working on the WormBase project.

Here is the complete job description: http://www.embl.de/jobs/searchjobs/index.php?newlang=1&ref=EBI_00611

Although the job description doesn't mention Clojure, I have been assured that Clojure will be the main development language.

23 Sep 2015 10:58am GMT

22 Sep 2015

feedPlanet Lisp

Michael Malis: Getting Places

This post will serve as an introduction to writing macros that work with places. I will refer back to it whenever I examine a macro which deals with places.

Places are an incredible part of Common Lisp. In short, a "place" is any location that can hold a value. The obvious example of a place is a variable. Less obvious examples include the elements of an array, or the slots of an object. What makes the concept of places special is that Common Lisp provides a standard interface for reading and writing to them. You can write macros on top of this interface that work for every kind of place. As an example, look at the macro incf. It takes a place as an argument, adds one to its value, and stores the new value back into the place. If you want to increment a variable x, you would use:

(incf x)

And if you wanted to increment the element at index x of a sequence, you would use:

(incf (elt seq x))

They use the exact same syntax even though a variable is very different from an element of a sequence. Because it takes advantage of the interface for places, incf will work on any place, be it a variable, the slot of an object, or a user defined place.

So at this point you are probably wondering how does incf work and more generally, how do you write macros that use places? To write such a macro, you need to use the function get-setf-expansion.1 Get-setf-expansion takes an expression representing a place and returns a total of five values (if you are unfamiliar with multiple values, see my post on multiple-value-bind). Altogether, these five values tell you everything you need to know about the place in order to read and write to it.

To show you how you are supposed to use get-setf-expansion, I'm first going to demonstrate how you could use it to write the expansion of incf by hand. After that, I will show code that will automate this, which winds up being an implementation of incf. Let's start by writing the expansion of the example above. The one where the element of a sequence is being incremented. To write the expansion of that by hand, you would first call get-setf-expansion to obtain all of the information:2

(get-setf-expansion '(elt seq x))

In SBCL this call will return the following values:

;; (1) temps
(#:seq1017 #:x1016)

;; (2) exprs
(seq x) 

;; (3) stores

;; (4) store-expr
(sb-kernel:%setelt #:seq1017 #:x1016 #:new1015) 

;; (5) access-expr
(elt #:seq1017 #:x1016))

From now on, I will refer to each value returned by get-setf-expansion by the name in the comment before it (e.g. temps refers to the first value).

In order to uniquely identify the element of a sequence (the place we are working with in this example), you need two things. You need the sequence itself and the index into the sequence. That is exactly what the two expressions in exprs evaluate to! Since incf needs to use these values multiple times, the two values have to be bound to gensyms in order to prevent multiple evaluation (see my post on once-only for why multiple evaluation is a problem). You are supposed to bind the values of the expressions to the gensyms in temps so that the other expressions returned by get-setf-expansion can use those gensyms to easily determine the place being worked with. The bindings need to be made with let* because it is possible for an expression in exprs to refer to the value of a previous expression in exprs. So the first part of the expansion will bind all of the symbols in temps to values of the expressions in exprs with let*:

(let* ((#:seq1017 seq) (#:x1016 x))

Now the gensyms in temps can be used to uniquely identify the place. As I mentioned previously, the other expressions can now easily determine the place through the gensyms. For example, access-expr can be used to retrieve the value currently in the place. Since the place we are dealing with is the element of a sequence, access-expr is just a call to elt using the gensyms in temps as the arguments. We are going to use access-expr in a moment, but first I have to talk about how to write to the place.

In order to write to the place, you need to use stores and store-expr. Stores is a list of gensyms that need to be bound to the values that are to be stored in the place (it is possible for a single place to hold multiple values). In this case we want to bind the gensym in stores to one plus the value already in the place. We can easily obtain the value in the place through access-expr. Once the gensyms have been bound, you can use store-expr to actually write the values in stores to the place. Notice how store-expr is a call to an internal SBCL function sb-kernel:setelt% that uses the gensyms in temps and stores as arguments. Presumably sb-kernel:setelt% sets the element of a sequence. After adding the binding for the gensym in stores and store-expr, we wind up with the final expansion which looks like:3

(let* ((#:seq1017 seq) 
       (#:x1016 x) 
       (#:new1015 (+ 1 (elt #:seq1017 #:x1016))))
  (sb-kernel:%setelt #:seq1017 #:x1016 #:new1015))

To review, the above code first binds the gensyms in temps to the values of the expressions in exprs. This allows access-expr and store-expr to use the gensyms in temps in order to determine the place being worked with. Then the code uses access-expr to retrieve the value, adds one to that, and binds that value to the gensym in stores. This is because the value of the gensym in stores is ultimately going to be the one written to the place. Finally the code evaluates store-expr in order to actually store the value in the gensym into the place.

Now here is one possible implementation of incf,4 which is code for everything we just did by hand. I called it incf% so that it doesn't have the same name as the builtin version.

(defmacro incf% (place)
        (temps exprs stores store-expr access-expr)
      (get-setf-expansion place)
    `(let* (,@(mapcar #'list temps exprs)
            (,(car stores) (+ 1 ,access-expr)))

The above code first binds the five values returned by get-setf-expansion to variables. It then generates a let* binding which binds the symbols in temps to the expressions in exprs and also binds the gensym in stores to one plus the result of evaluating access-expr. Finally the above code splices in store-expr to actually write the value. And that is everything there is to incf.

Incf is but a single example of what can be done with places. In the next couple of posts, I plan to cover some really cool macros that encapsulate a bunch of common patterns related to places.

The post Getting Places appeared first on Macrology.

22 Sep 2015 11:00pm GMT

19 Sep 2015

feedPlanet Lisp

Patrick Stein: Say What You Mean

There is a scene in the movie The Birdcage where the son tells his father that he (the son) has met a girl and is going to get married. The father begins gulping down the glass of wine that he has in hand. The son asks, Are you upset? The father finishes the glass of wine and says, Let me tell you why.

Here is a function that I wrote several years ago.

(sheeple:defreply mouse-move ((item =draggable=) xx yy)
(let ((dragging (dragging item)))
(when dragging
(let ((dx (- xx (car dragging)))
(dy (- yy (cdr dragging))))
(incf (offset-x item) dx)
(incf (offset-y item) dy))
(let ((pp (parent item)))
(when pp
(when (< (width pp) (+ (offset-x item) (width item)))
(setf (offset-x item) (- (width pp) (width item))))
(when (< (height pp) (+ (offset-y item) (height item)))
(setf (offset-y item) (- (height pp) (height item))))))
(when (< (offset-x item) 0) (setf (offset-x item) 0))
(when (< (offset-y item) 0) (setf (offset-y item) 0))

This is awful! Am I upset? Let me tell you why.

Is it the Single Responsibility Principle (SRP)? No.

Is it Don't Repeat Yourself (DRY)? No.

Is it Mixing Levels of Abstraction? Closer, but not quite.

Those are all clearly violated by this code. But, that's not really the problem. The problem is Why. Nothing about this code tells you why it is here or what is doing.

There is no way to glance at that function and have any idea what's going on. You have to read it carefully. You have to understand things that aren't even in this source file to make head nor tail of it. Once you understand the second LET block, you will have nine more lines of code without the least inkling of why there should be nine more lines of code. Anyone care to hazard a guess as to why this function returns T (only) when we're dragging?


Two years ago, a colleague and I were tasked with providing docstrings for every function in all of the code we'd written in the last year. We'd done well on providing docstrings to the outward-facing functions, but now we had to do the rest. He started at one end of the directory (in alphabetical order), and I started at the other end. This gave me a good opportunity to look closely at a boat-load of code he'd written that I'd never really delved into before.

He was absolutely religious about encapsulating containers. If he had a hash-table or a p-list or a flat list in a DEFVAR, there was one and only one function that retrieved items from it and at most one function that added items to it. Those functions were one or two lines each (two if they needed a mutex). Those functions were named after what the collection was storing not what mechanism was used to store them.

A lot of times when people talk about the value of encapsulating, they talk about shielding the rest of the code from the implementation details so that if you need to replace how it's actually implemented on the back end you can do it without breaking any existing code. You are protecting your precious implementation from how people will use it so that you can someday replace the implementation with an even more precious implementation next year (when your language finally gets first-class functions).

I've been coding for a good long time now. I'm going to let you in on a little secret. Code almost never gets replaced. When code does get replaced, it almost never continues to adhere to the old API (there was always a semantic leak). If there is a business justification strong enough to let you replace the code, it's because the old code has become an unmaintainable mess of people subverting the interface or the code as it is didn't scale and now synchronous things need to happen asynchronously or local things have to happen remotely and hiding that under your old API isn't going to relieve the bottlenecks.

Trying to insulate your code so that it's easy to replace is looking down the wrong end of the telescope. The real benefit of encapsulation is that the people who read your code later can be half-asleep and still get everything-your code will scream its meaning. The real benefit of encapsulation is that the person debugging your code can set a break-point in a place that means something-not in the seventeen places the state might have changed but in the only place it could change.

Making It Better

Any ideas what the body in this function does?

(sheeple:defreply mouse-move ((item =draggable=) xx yy)
(if (being-dragged-p item)
(handle-event ()
(let ((dx (- xx (drag-starting-x item)))
(dy (- yy (drag-starting-y item))))
(translate-widget item dx dy)
(keep-widget-inside-parent item)))
(ignore-event ())))

The new functions BEING-DRAGGED-P, DRAG-STARTING-X, and DRAG-STARTING-Y are just wrappers around what had been explicitly treated as an (OR NULL (CONS INTEGER INTEGER)).

(defun being-dragged-p (item)
(dragging item))

(defun drag-starting-x (item)
(car (dragging item)))
(defun drag-starting-y (item)
(cdr (dragging item)))

It is still an (OR NULL (CONS INTEGER INTEGER)) but nobody ever has to care. Nobody ever has to try to remember what the integers mean. Sure, you could replace it with a structure or a complex number, but why would you ever bother? Why would you ever look at it again?

The new macros HANDLE-EVENT and IGNORE-EVENT encapsulate the return value of this function into something with meaning.

(defmacro handle-event (() &body body)
(values t)

(defmacro ignore-event (() &body body)
(values nil)

It might still be too easy to write an event-handler with a path which doesn't end in one of these two macros, but it is way better than that dangling T was. It looks like it's really supposed to be there, and it looks like what it means rather than what it is.

The TRANSLATE-WIDGET and KEEP-WIDGET-INSIDE-PARENT functions can benefit greatly with some further helper functions (and analogous functions for top and bottom):

(defun left (item)
(offset-x item))
(defun (setf left) (x item)
(setf (offset-x item) x))
(defun right (item)
(+ (left item) (width item)))
(defun (setf right) (x item)
(setf (offset-x item) (- x (width item))))

Some Rules of Thumb

If you find that when you want to check (PRED1 ...) you instead have to check:

(and (PRED0 ...)
(PRED1 ...))

Then you should consider making a function that does them both. Consider the difference between these two blocks of code:

(when (and (connectedp (player1 g))
(connectedp (player2 g))
(not (pausedp g)))

(when (game-active-p g)

If you find that you are depending on the NULL-ness or positiveness or some other property of some number of state variables to decide which course of action to take, then you should consider making predicates named after your state. In many OO scenarios, you may even want to explicitly track (or calculate) which state you are in at all times.

(defmacro state-case (g &body clauses)
`(ecase (calculate-or-fetch-state-of g)

(state-case g

In more imperative languages, it may even be beneficial to keep a STATE member variable in your class. When doing that, make sure that there is one and only one function which actually mutates the value of that STATE member. This will let you:

  1. Log all state transitions without having to hunt for all of them.
  2. Quickly hunt for all of them if you want to do that
  3. Set a break point on all state changes.
  4. Enforce the validity of transitions (or at least scream loudly when something transitions from STOPPED to PAUSED without having passed through PLAYING first).

If you have to check whether some resource is being used by some instance, don't ask it which resource it is using, ask it whether it is using the one you want.

;;; Common: Reader is forced to know each player has one socket and
;;; that sockets are comparable with #'=
(loop :for player :in all-networked-players
:until (= socket-with-something-happening
(player-socket player))
:finally (return player))

;;; Better: All I wanted to know is, "Is this yours?"
(loop :for player :in all-networked-players
:until (player-using-socket-p player
:finally (return player))

Encapsulation is about protecting the person who has to read your code. It's not about protecting your code.

19 Sep 2015 2:55pm GMT

18 Sep 2015

feedPlanet Lisp

ABCL Dev: abcl-1.3.3 released

In anticipation of the coming season, we have released abcl-1.3.3

Version 1.3.3


* [r14802,r14813] Add character name for non-breaking space

Use a human readable name for character 160, #\No-break_space,
following sbcl, ccl and clisp. This permits the Quicklisp system
spinneret to load. The #\No-break_space name is a valid
CHAR-NAME/NAME-CHAR pair, but is emitted directly as a glyph wrt. the
current output encoding under the CL:FORMAT "~:c" directive as
these implementations do by default.

Thanks to Javier Olaechea.


* [r14808] CL:FILE-WRITE-DATE fixed for logical pathnames


** Update references to new git repository https://gitlab.common-lisp.net/ansi-test/ansi-test

** ABCL now runs the git master consolidated ANSI-TEST suite which
features subdirectories and distinquished value for

** ABCL.TEST.ANSI:CLEAN-TESTS now acts recursively via appropiate
Pathname wildcards to match new directory structure.


Fix COMPILE-SYSTEM to offer full ANSI environment for ASDF and


** Use of Maven has been robustified.

*** [r14803] Fix usage with all known versions through maven-3.3.3

Addresses http://abcl.org/trac/ticket/390

*** [r14806] Fix usage with specifying local Maven repository

** More complete attempt at re-initialization via

(ABCL-ASDF:INIT :force t)

18 Sep 2015 9:39pm GMT

Victor Anyakin: Introducing scanf in Common Lisp

There are many ways to extract data from strings or files. The scanf family of function offers one of them.

These functions scan input according to a provided format string. The format string might contain conversion specifiers or conversion directives to extract integers, floating-point numbers, characters, strings, etc. from the input and store it in the arguments.

For example, a format string for parsing components of an IP address might look like: "%3d.%3d.%3d.%3d" - the scanf function will parse 4 integers (maximum 3 digits each) that are delimited with dots and return them to the caller.

There are basically two ways to implement scanf:

  1. as an interpreter, that scans format string and executes commands as they are retrieved.
  2. as a translator to an intermediate language that, in turn, is compiled into machine code.

The trivial-scanf package (that comes as the part of the CL-STRING-MATCH library) takes the first approach. The trivial-scanf implementation reads one character at a time, and depending on the read character performs the designated operation. Underneath, it uses PROC-PARSE library to deal with the input. Outline of the function's main loop looks as follows:

 (while (< fmt-pos fmt-len))
 (for c = (char fmt fmt-pos))
 (case c
    ;; process conversion directive
   ((#\Space #\Tab #\Return #\Newline #\Page)
    ;; process white space characters
    ;; process ordinary characters

Conversion directives might have optional flags and parameters that must be taken into account. Simple directives, like %d, are handled in a straightforward way: input matching to the designated data type (digits) are bound to a string that is then parsed using corresponding function (parse-integer in this case).

However, the standard scanf also specifies a directive to match a set of designated characters. For example, directive '%[a-z0-9-]' would scan input and return a string composed of letters, digits, and a dash from the current position, until first mismatch. In case, if we dealt with an octet-string (a string where every character is guaranteed to be a single byte in size), it would be feasible to interpret this directive using a table to mark characters that belong to the set. The trivial-scanf takes another approach: characters set directive is converted into a list of closures that serve as predicates for the input string binding operation. In our example, the list of closures would contain predicates for: (range #\a...#\z), (range #\0...#\9) (character #\).

trivial-scanf will be accessible through Quicklisp after the next packages update. At the moment you can clone the repository and install it locally.

Some usage examples:

(ql:quickload :trivial-scanf)
(snf:scanf "%3d.%3d.%3d.%3d" "") => (127 0 0 1)
(snf:scanf "%d %[A-C] %d" "1  ABBA  2") => (1 "ABBA" 2)

This the first (almost alpha) release of the code, so some bugs are expected. Feel free to comment or submit them.

trivial-scanf is the part of the CL-STRING-MATCH library.

18 Sep 2015 12:14pm GMT

16 Sep 2015

feedPlanet Lisp

Zach Beane: My new favorite slime/sbcl/ccl trick

Thanks to _death on #lisp, I just updated ~/.swank.lisp with this:

(push (lambda (&rest args)
        (apply #'swank:ed-in-emacs args)

Update: Matt Emerson clued me in to this:

(setq ccl:*resident-editor-hook* #'swank:ed-in-emacs)

Now I can (ed "foo.lisp") in the repl and it pops open in a new buffer. Fantastic!

16 Sep 2015 2:37pm GMT

15 Sep 2015

feedPlanet Lisp

Joe Marshall: Integers and rationals

In an earlier post, I asked readers to implement the bijection rational->integer and integer->rational. John Cowan suggested the Calkin-Wilf tree as a starting point. The Calkin-Wilf tree is a rooted binary tree where the nodes (or vertices, if you like) are labeled with positive rational numbers. It is infinite and complete: every node has two children. The Calkin-Wilf tree is constructed so that every rational number is assigned a unique node. Every positive rational number appears once and exactly once in the tree. The path from the root node to any selected rational is unique and can be encoded (in binary) as an integer.

1 ]=> (rational->integer 355/113)

;Value: 67107847

1 ]=> (integer->rational 67107847)

;Value: 355/113

1 ]=> (cwt/value *the-calkin-wilf-tree*)

;Value: 1

1 ]=> (cwt/value (cwt/left *the-calkin-wilf-tree*))

;Value: 1/2

1 ]=> (cwt/value (cwt/right *the-calkin-wilf-tree*))

;Value: 2

1 ]=> (cwt/value (cwt/left (cwt/right (cwt/left (cwt/left *the-calkin-wilf-tree*)))))

;Value: 4/7

Ho hum. We've all seen this sort of thing before.

Here's the unusual part:

1 ]=> cwt/left

;Value 1236: #[linear-fractional-transform 1236 x/(x + 1)]

1 ]=> cwt/right

;Value 1237: #[linear-fractional-transform 1237 (x + 1)]

So I can write

1 ]=> (cwt/value ((compose cwt/left cwt/right cwt/left cwt/left) *the-calkin-wilf-tree*))

;Value: 4/7

1 ]=> (lft/compose cwt/left cwt/right cwt/left cwt/left)

;Value 1260: #[linear-fractional-transform 1260 (3x + 1)/(5x + 2)]

(See "Playing with Linear Fractional Transforms")

The dyadic fractions are those rational numbers whose denominator is a power of 2. Numbers like 1/4, 3/8, or 11/32. These are the divisions you'd find on a ruler (in the US). Floating point numbers are usually implemented as dyadic fractions.
You can put the dyadic fractions into a binary tree as follows:

(define *the-dyadic-fraction-tree* 1)

(define (dft/left node)
  (/ (- (* (numerator node) 2) 1)
     (* (denominator node) 2)))

(define (dft/right node)
  (/ (+ (* (numerator node) 2) 1)
     (* (denominator node) 2)))

1 ]=> *the-dyadic-fraction-tree*

;Value: 1

1 ]=> (dft/left *the-dyadic-fraction-tree*)

;Value: 1/2

1 ]=> (dft/right *the-dyadic-fraction-tree*)

;Value: 3/2

1 ]=> (dft/left (dft/left (dft/right (dft/left *the-dyadic-fraction-tree*))))

;Value: 9/16

The next question is, what happens if I use a path derived from the Calkin-Wilf tree and use it on the dyadic fraction tree? Yes, this is a fairly random thing to try, but the trees are the same (that is, have the same structure) even if the values at the nodes are not. Either set of fractions is in a one-to-one mapping with the tree, so there is a one-to-one mapping between rational numbers and dyadic fractions.

This is Minkowski's ? (question mark) function. It maps the rational numbers on the X axis to the dyadic fraction on the Y axis. It has a number of weird properties. For example, it is strictly increasing and continuous, but it is not absolutely continuous. The function does not have a derivative in the usual sense.

15 Sep 2015 6:19pm GMT