23 Jan 2023

feedPlanet Lisp

Nicolas Martyanoff: Custom Common Lisp indentation in Emacs

While SLIME is most of the time able to indent Common Lisp correctly, it will sometimes trip on custom forms. Let us see how we can customize indentation.

In the process of writing my PostgreSQL client in Common Lisp, I wrote a READ-MESSAGE-CASE macro which reads a message from a stream and execute code depending on the type of the message:

(defmacro read-message-case ((message stream) &rest forms)
  `(let ((,message (read-message ,stream)))
     (case (car ,message)
       (:error-response
        (backend-error (cdr ,message)))
       (:notice-response
        nil)
       ,@forms
       (t
        (error 'unexpected-message :message ,message)))))

This macro is quite useful: all message loops can use it to automatically handle error responses, notices, and signal unexpected messages.

But SLIME does not know how to indent READ-MESSAGE-CASE, so by default it will align all message forms on the first argument:

(read-message-case (message stream)
                   (:authentication-ok
                     (return))
                   (:authentication-cleartext-password
                     (unless password
                       (error 'missing-password))
                     (write-password-message password stream)))

While we want it aligned the same way as HANDLER-CASE:

(read-message-case (message stream)
  (:authentication-ok
    (return))
  (:authentication-cleartext-password
    (unless password
      (error 'missing-password))
    (write-password-message password stream)))

Good news, SLIME indentation is defined as a list of rules. Each rule associates an indentation specification (a S-expression describing how to indent the form) to a symbol and store it as the common-lisp-indent-function property of the symbol.

You can obtain the indentation rule of a Common Lisp symbol easily. For example, executing (get 'defun 'common-lisp-indent-function) (e.g. in IELM or with eval-expression) yields (4 &lambda &body). This indicates that DEFUN forms are to be indented as follows:

You can refer to the documentation of the common-lisp-indent-function Emacs function (defined in SLIME of course) for a complete description of the format.

We want READ-MESSAGE-CASE to be indented the same way as HANDLER-CASE, whose indentation specification is (4 &rest (&whole 2 &lambda &body)) (in short, an argument and a list of lambda lists). Fortunately there is a way to specify that a form must be indented the same way as another form, using (as <symbol>).

Let us first define a function to set the indentation specification of a symbol:

(defun g-common-lisp-indent (symbol indent)
  "Set the indentation of SYMBOL to INDENT."
  (put symbol 'common-lisp-indent-function indent))

Then use it for READ-MESSAGE-CASE:

(g-common-lisp-indent 'read-message-case '(as handler-case))

While it is in general best to avoid custom indentation, exceptions are sometimes necessary for readability. And SLIME makes it easy.

23 Jan 2023 6:00pm GMT

18 Jan 2023

feedPlanet Lisp

TurtleWare: Method Combinations

Table of Contents

  1. Introduction
  2. Defining method combinations - the short form
  3. Defining method combinations - the long form
    1. The Hooker
    2. The Memoizer
  4. Conclusions

Update [2023-01-23]

Christophe Rhodes pointed out that "The Hooker" method combination is not conforming because there are multiple methods with the same "role" that can't be ordered and that have different qualifiers:

Note that two methods with identical specializers, but with different qualifiers, are not ordered by the algorithm described in Step 2 of the method selection and combination process described in Section 7.6.6 (Method Selection and Combination). Normally the two methods play different roles in the effective method because they have different qualifiers, and no matter how they are ordered in the result of Step 2, the effective method is the same. If the two methods play the same role and their order matters, an error is signaled. This happens as part of the qualifier pattern matching in define-method-combination.

http://www.lispworks.com/documentation/HyperSpec/Body/m_defi_4.htm

So instead of using qualifier patterns we should use qualifier predicates. They are not a subject of the above paragraph because of its last sentence (there is also an example in the spec that has multiple methods with a predicate). So instead of

(define-method-combination hooker ()
  (... (hook-before (:before*)) ...) ...)

the method combination should use:

(defun hook-before-p (method-qualifier)
  (typep method-qualifier '(cons (eql :before) (cons t null))))

(define-method-combination hooker ()
  (... (hook-before hook-before-p) ...) ...)

and other "hook" groups should also use predicates.

Another thing worth mentioning is that both ECL and SBCL addressed issues with the qualifier pattern matching and :arguments since the publication of this blog post.

Introduction

Method combinations are used to compute the effective method for a generic function. An effective method is a body of the generic function that combines a set of applicable methods computed based on the invocation arguments.

For example we may have a function responsible for reporting the object status and each method focuses on a different aspect of the object. In that case we may want to append all results into a list:

(defgeneric status (object)
  (:method-combination append))

(defclass base-car ()
  ((engine-status :initarg :engine :accessor engine-status)
   (wheels-status :initarg :wheels :accessor wheels-status)
   (fuel-level :initarg :fuel :accessor fuel-level))
  (:default-initargs :engine 'ok :wheels 'ok :fuel 'full))

(defmethod status append ((object base-car))
  (list :engine (engine-status object)
        :wheels (wheels-status object)
        :fuel (fuel-level object)))

(defclass premium-car (base-car)
  ((gps-status :initarg :gps :accessor gps-status)
   (nitro-level :initarg :nitro :accessor nitro-level))
  (:default-initargs :gps 'no-signal :nitro 'low))

(defmethod status append ((object premium-car))
  (list :gps (gps-status object)
        :nitro (nitro-level object)))

CL-USER> (status (make-instance 'premium-car))
(:GPS NO-SIGNAL :NITRO LOW :ENGINE OK :WHEELS OK :FUEL FULL)

CL-USER> (status (make-instance 'base-car))
(:ENGINE OK :WHEELS OK :FUEL FULL)

The effective method may look like this:

(append (call-method #<method status-for-premium-car>)
        (call-method #<method status-for-base-car>   ))

Note that append is a function so all methods are called. It is possible to use other operators (for example a macro and) and then the invocation of particular methods may be conditional:

(and (call-method #<method can-repair-p-for-premium-car>)
     (call-method #<method can-repair-p-for-base-car>   ))

Defining method combinations - the short form

The short form allows us to define a method combination in the spirit of the previous example:

(OPERATOR (call-method #<m1>)
          (call-method #<m2>)
          ...)

For example we may want to return as the second value the count of odd numbers:

(defun sum-and-count-odd (&rest args)
  (values (reduce #'+ args)
          (count-if #'oddp args)))

(define-method-combination sum-and-count-odd)

(defclass a () ())
(defclass b (a) ())
(defclass c (b) ())

(defgeneric num (o)
  (:method-combination sum-and-count-odd)
  (:method sum-and-count-odd ((o a)) 1)
  (:method sum-and-count-odd ((o b)) 2)
  (:method sum-and-count-odd ((o c)) 3)
  (:method :around ((o c))
    (print "haa!")
    (call-next-method)))

(num (make-instance 'b)) ;; (values 3 1)
(num (make-instance 'c)) ;; (values 6 2)

Note that the short form supports also around methods. It is also important to note that effective methods are cached, that is unless the generic function or the method combination changes, the computation of the effective method may be called only once per the set of effective methods.

Admittedly these examples are not very useful. Usually we operate on data stored in instances and this is not a good abstraction to achieve that. Method combinations are useful to control method invocations and their results. Here is another example:

(defmacro majority-vote (&rest method-calls)
  (let* ((num-methods (length method-calls))
         (tie-methods (/ num-methods 2)))
    `(prog ((yes 0) (no 0))
        ,@(loop for invocation in method-calls
                append `((if ,invocation
                             (incf yes)
                             (incf no))
                         (cond
                           ((> yes ,tie-methods)
                            (return (values t yes no)))
                           ((> no ,tie-methods)
                            (return (values nil yes no))))))
        (error "we have a tie! ~d ~d" yes no))))

(define-method-combination majority-vote)

(defclass a () ())
(defclass b (a) ())
(defclass c (b) ())
(defclass d (c) ())

(defgeneric foo (object param)
  (:method-combination majority-vote)
  (:method majority-vote ((o a) param) nil)
  (:method majority-vote ((o b) param) t)
  (:method majority-vote ((o c) param) t)
  (:method majority-vote ((o d) param) nil))

(foo (make-instance 'a) :whatever) ; (values nil 0 1)
(foo (make-instance 'b) :whatever) ; #<error tie 1 1>
(foo (make-instance 'c) :whatever) ; (values t 2 0)
(foo (make-instance 'd) :whatever) ; #<error tie 2 2>

Defining method combinations - the long form

The long form is much more interesting. It allows us to specify numerous qualifiers and handle methods without any qualifiers at all.

The Hooker

Here we will define a method combination that allows us to define named hooks that are invoked before or after the method. It is possible to have any number of hooks for the same set of arguments (something we can't achieve with the standard :before and :after auxiliary methods):

(defun combine-auxiliary-methods (primary around before after)
  (labels ((call-primary ()
             `(call-method ,(first primary) ,(rest primary)))
           (call-methods (methods)
             (mapcar (lambda (method)
                       `(call-method ,method))
                     methods))
           (wrap-after (the-form)
             (if after
                 `(multiple-value-prog1 ,the-form
                    ,@(call-methods after))
                 the-form))
           (wrap-before (the-form)
             (if before
                 `(progn
                    ,@(call-methods before)
                    ,the-form)
                 the-form))
           (wrap-around (the-form)
             (if around
                 `(call-method ,(first around)
                               (,@(rest around)
                                (make-method ,the-form)))
                 the-form)))
    (wrap-around (wrap-after (wrap-before (call-primary))))))

(define-method-combination hooker ()
  ((normal-before (:before))
   (normal-after  (:after)
                  :order :most-specific-last)
   (normal-around (:around))
   (hook-before   (:before *))
   (hook-after    (:after  *)
                  :order :most-specific-last)
   (hook-around   (:around *))
   (primary () :required t))
  (let ((around (append hook-around normal-around))
        (before (append hook-before normal-before))
        (after  (append normal-after hook-after)))
    (combine-auxiliary-methods primary around before after)))

With this we may define a generic function and associated methods similar to other functions with an extra feature - we may provide named :before, :after and :around methods. Named auxiliary methods take a precedence over unnamed ones. Only after that the specialization is considered. There is one caveat - PCL-derived CLOS implementations (clasp, cmucl, ecl, sbcl) currently ([2023-01-18 Ε›ro]) have a bug preventing wildcard qualifier pattern symbol * from working. So better download ccl or wait for fixes. Here's an example for using it:

;;; The protocol.
(defgeneric note-buffer-dimensions-changed (buffer w h)
  (:method (b w h)
    (declare (ignore b w h))
    nil))

(defgeneric change-dimensions (buffer w h)
  (:method-combination hooker))

;;; The implementation of unspecialized methods.
(defmethod change-dimensions :after (buffer w h)
  (note-buffer-dimensions-changed buffer w h))

;;; The stanard class.
(defclass buffer ()
  ((w :initform 0 :accessor w)
   (h :initform 0 :accessor h)))

;;; The implementation for the standard class.
(defmethod change-dimensions ((buffer buffer) w h)
  (print "... Changing the buffer size ...")
  (setf (values (w buffer) (h buffer))
        (values w h)))

(defmethod note-buffer-dimensions-changed ((buffer buffer) w h)
  (declare (ignore buffer w h))
  (print "... Resizing the viewport ..."))

;;; Some dubious-quality third-party code that doesn't want to interfere with
;;; methods defined by the implementation.
(defmethod change-dimensions :after system (buffer w h)
  (print `(log :something-changed ,buffer ,w ,h)))

(defmethod change-dimensions :after my-hook ((buffer buffer) w h)
  (print `(send-email! :me ,buffer ,w ,h)))

CL-USER> (defvar *buffer* (make-instance 'buffer))
*BUFFER*
CL-USER> (change-dimensions *buffer* 10 30)

"... Changing the buffer size ..." 
"... Resizing the viewport ..." 
(LOG :SOMETHING-CHANGED #<BUFFER #x30200088220D> 10 30) 
(SEND-EMAIL! :ME #<BUFFER #x30200088220D> 10 30) 
10
30

The Memoizer

Another example (this time it will work on all implementations) is optional memoization of the function invocation. If we define a method with the qualifier :memoize then the result will be cached depending on arguments. The method combination allows also "normal" auxiliary functions by reusing the function combine-auxiliary-methods from the previous section.

The function ensure-memoized-result accepts the following arguments:

When the current generation is nil that means that caching is disabled and we remove the result from the cache. Otherwise we use the test to compare the generation of a cached value and the current one - if they are the same, then the cached value is returned. Otherwise it is returned.

(defparameter *memo* (make-hash-table :test #'equal))
(defun ensure-memoized-result (test memo cache-key form)
  `(let ((new-generation ,memo))
     (if (null new-generation)
         (progn
           (remhash ,cache-key *memo*)
           ,form)
         (destructuring-bind (old-generation . cached-result)
             (gethash ,cache-key *memo* '(nil))
           (apply #'values
                  (if (,test old-generation new-generation)
                      cached-result
                      (rest
                       (setf (gethash ,cache-key *memo*)
                             (list* new-generation (multiple-value-list ,form))))))))))

The method with the qualifier :memoize is used to compute the current generation key. When there is no such method then the function behaves as if the standard method combination is used. The method combination accepts a single argument test, so it is possible to define different predicates for deciding whether the cache is up-to-date or not.

(define-method-combination memoizer (test)
  ((before (:before))
   (after  (:after) :order :most-specific-last)
   (around (:around))
   (memoize (:memoize))
   (primary () :required t))
  (:arguments &whole args)
  (:generic-function function)
  (let ((form (combine-auxiliary-methods primary around before after))
        (memo `(call-method ,(first memoize) ,(rest memoize)))
        (ckey `(list* ,function ,args)))
    (if memoize
        (ensure-memoized-result test memo ckey form)
        form)))

Now let's define a function with "our" method combination. We will use a counter to verify that values are indeed cached.

(defparameter *counter* 0)

(defgeneric test-function (arg &optional opt)
  (:method-combination memoizer eql))

(defmethod test-function ((arg integer) &optional opt)
  (list* `(:counter ,(incf *counter*)) arg opt))

CL-USER> (test-function 42)
((:COUNTER 1) 42)
CL-USER> (test-function 42)
((:COUNTER 2) 42)
CL-USER> (defmethod test-function :memoize ((arg integer) &optional (cache t))
           (and cache :gen-z))
#<STANDARD-METHOD TEST-FUNCTION :MEMOIZE (INTEGER)>
CL-USER> (test-function 42)
((:COUNTER 3) 42)
CL-USER> (test-function 42)
((:COUNTER 3) 42)
CL-USER> (test-function 42 nil)
((:COUNTER 4) 42)
CL-USER> (test-function 42)
((:COUNTER 3) 42)
CL-USER> (test-function 43)
((:COUNTER 5) 43)
CL-USER> (test-function 43)
((:COUNTER 5) 43)
CL-USER> (defmethod test-function :memoize ((arg (eql 43)) &optional (cache t))
           (and cache :gen-x))
#<STANDARD-METHOD TEST-FUNCTION :MEMOIZE ((EQL 43))>
CL-USER> (test-function 43)
((:COUNTER 6) 43)
CL-USER> (test-function 43)
((:COUNTER 6) 43)
CL-USER> (test-function 42)
((:COUNTER 3) 42)

Conclusions

Method combinations are a feature that is often overlooked but give a great deal of control over the generic function invocation. The fact that ccl is the only implementation from a few that I've tried which got method combinations "right" doesn't surprise me - I've always had an impression that it shines in many unexpected places.

18 Jan 2023 12:00am GMT

16 Jan 2023

feedPlanet Lisp

Nicolas Martyanoff: ANSI color rendering in SLIME

I was working on the terminal output for a Common Lisp logger, and I realized that SLIME does not interpret ANSI escape sequences.

This is not the end of the world, but having at least colors would be nice. Fortunately there is a library to do just that.

First let us install the package, here using use-package and straight.el.

(use-package slime-repl-ansi-color
  :straight t)

While in theory we are supposed to just add slime-repl-ansi-color to slime-contribs, it did not work for me, and I add to enable the minor mode manually.

If you already have a SLIME REPL hook, simply add (slime-repl-ansi-color-mode 1). If not, write an initialization function, and add it to the SLIME REPL initialization hook:

(defun g-init-slime-repl-mode ()
  (slime-repl-ansi-color-mode 1))
  
(add-hook 'slime-repl-mode-hook 'g-init-slime-repl-mode)

To test that it works as intended, fire up SLIME and print a simple message using ANSI escape sequences:

(let ((escape (code-char 27)))
  (format t "~C[1;33mHello world!~C[0m~%" escape escape))

While it is tempting to use the #\Esc character, it is part of the Common Lisp standard; therefore we use CODE-CHAR to obtain it from its ASCII numeric value. We use two escape sequences, the first one to set the bold flag and foreground color, and the second one to reset display status.

If everything works well, should you see a nice bold yellow message:

ANSI escape sequence rendering

16 Jan 2023 6:00pm GMT

13 Jan 2023

feedPlanet Lisp

Lispjobs: DevOps Engineer | HRL Laboratories | Malibu, CA

Job posting: https://jobs.lever.co/dodmg/85221f38-1def-4b3c-b627-6ad26d4f5df7?lever-via=CxJdiOp5C6

HRL has been on the leading edge of technology, conducting pioneering research and advancing the state of the art. This position is integrated with a growing team of scientists and engineers on HRL's quantum computing research program.

GENERAL DESCRIPTION:

As a DevOps/DevSecOps engineer, you'll be focused on maintaining reliable systems for testing and delivery of HRL's quantum software. (You will not be directly responsible for developing the software or its tests.)

Specifically, you will be responsible for:

* Monitoring the status of CI/CD infrastructure on open and air-gapped networks.

* Building and maintaining infrastructure for synchronizing software between open and air-gapped networks.

* Working closely with developers and IT staff to ensure continued reliability of integration and deployment infrastructure.

* Tracking and vetting software dependencies.

* Looking for and implementing improvements to DevSecOps practices.

Among other candidate requirements, we highly value expertise in Lisp, Python, and C++.

CONFIDENTIALITY NOTICE: The information transmitted in this email, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential, proprietary and/or privileged material exempt from disclosure under applicable law. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this message in error, please contact the sender immediately and destroy any copies of this information in their entirety.

13 Jan 2023 7:36pm GMT

12 Jan 2023

feedPlanet Lisp

Nicolas Martyanoff: Switching between implementations with SLIME

While I mostly use SBCL for Common Lisp development, I regularly switch to CCL or even ECL to run tests.

This is how I do it with SLIME.

Starting implementations

SLIME lets you configure multiple implementations using the slime-lisp-implementations setting. In my case:

(setq slime-lisp-implementations
   '((sbcl ("/usr/bin/sbcl" "--dynamic-space-size" "2048"))
     (ccl ("/usr/bin/ccl"))
     (ecl ("/usr/bin/ecl"))))

Doing so means that running M-x slime will execute the first implementation, i.e. SBCL. There are two ways to run other implementations.

First you can run C-u M-x slime which lets you type the path and arguments of the implementation to execute. This is a bit annoying because the prompt starts with the content of the inferior-lisp-program variable, i.e. "lisp" by default, meaning it has to be deleted manually each time. Therefore I set inferior-lisp-program to the empty string:

(setq inferior-lisp-program "")

Then you can run C-- M-x slime (or M-- M-x slime which is easier to type) to instruct SLIME to use interactive completion (via completing-read) to let you select the implementations among those configured in slime-lisp-implementations.

To make my life easier, I bind C-c C-s s to a function which always prompt for the implementation to start:

(defun g-slime-start ()
  (interactive)
  (let ((current-prefix-arg '-))
    (call-interactively 'slime)))

Using C-c C-s as prefix for all my global SLIME key bindings helps me remember them.

Switching between multiple implementations

Running the slime function several times will create multiple connections as expected. Commands executed in Common Lisp buffers are applied to the current connection, which is by default the most recent one.

There are two ways to change the current implementation:

  1. Run M-x slime-next-connection.
  2. Run M-x slime-list-connections, which opens a buffer listing connections, and lets you choose the current one with the d key.

I find both impractical: the first one does not let me choose the implementation, forcing me to run potentially several times before getting the one I want. The second one opens a buffer but does not switch to it.

All I want is a prompt with completion. So I wrote one.

First we define a function to select a connection among existing one:

(defun g-slime-select-connection (prompt)
  (interactive)
  (let* ((connections-data
          (mapcar (lambda (process)
                    (cons (slime-connection-name process) process))
                  slime-net-processes))
         (completion-extra-properties
          '(:annotation-function
            (lambda (string)
              (let* ((process (alist-get string minibuffer-completion-table
                                         nil nil #'string=))
                     (contact (process-contact process)))
                (if (consp contact)
                    (format "  %s:%s" (car contact) (cadr contact))
                  (format "  %S" contact))))))
         (connection-name (completing-read prompt connections-data)))
    (let ((connection (cl-find connection-name slime-net-processes
                               :key #'slime-connection-name
                               :test #'string=)))
      (or connection
          (error "Unknown SLIME connection %S" connection-name)))))

Then use it to select a connection as the current one:

(defun g-slime-switch-connection ()
  (interactive)
  (let ((connection (g-slime-select-connection "Switch to connection: ")))
    (slime-select-connection connection)
    (message "Using connection %s" (slime-connection-name connection))))

I bind this function to C-c C-s c.

In a perfect world, we could format nice columns in the prompt and highlight the current connection, but the completing-read interface is really limited, and I did not want to use an external package such as Helm.

Stopping implementations

Sometimes it is necessary to stop an implementations and kill all associated buffers. It is not something I use a lot; but when I need it, it is frustrating to have to switch to the REPL buffer, run slime-quit-lisp, then kill the REPL buffer manually.

Adding this feature is trivial with the g-slime-select-connection defined earlier:

(defun g-slime-kill-connection ()
  (interactive)
  (let* ((connection (g-slime-select-connection "Kill connection: "))
         (repl-buffer (slime-repl-buffer nil connection)))
    (when repl-buffer
      (kill-buffer repl-buffer))
    (slime-quit-lisp-internal connection 'slime-quit-sentinel t)))

Finally I bind this function to C-c C-s k.

It is now much more comfortable to manage multiple implementations.

12 Jan 2023 6:00pm GMT

11 Jan 2023

feedPlanet Lisp

Tim Bradshaw: A case-like macro for regular expressions

I often find myself wanting a simple case-like macro where the keys are regular expressions. regex-case is an attempt at this.

I use CL-PPCRE for the usual things regular expressions are useful for, and probably for some of the things they should not really be used for as well. I often find myself wanting a case like macro, where the keys are regular expressions. There is a contributed package for Trivia which will do this, but Trivia is pretty overwhelming. So I gave in and wrote regex-case which does what I want.

regex-case is a case-like macro. It looks like

(regex-case <thing>
  (<pattern> (...)
   <form> ...)
  ...
  (otherwise ()
   <form> ...))

Here <pattern> is a literal regular expression, either a string or in CL-PPCRE's s-expression parse-tree syntax for them. Unlike case there can only be a single pattern per clause: allowing the parse-tree syntax makes it hard to do anything else. otherwise (which can also be t) is optional but must be last.

The second form in a clause specifies what, if any, variables to bind on a match. As an example

(regex-case line
  ("fog\\s+(.*)\\s$" (:match m :registers (v))
    ...)
  ...)

will bind m to the whole match and v to the substring corresponding to the first register. You can also bind match and register positions. A nice (perhaps) thing is that you can not bind some register variables:

(regex-case line
  (... (:registers (_ _ v))
   ...)
  ...)

will bind v to the substring corresponding to the third register. You can use nil instead of _.

The current state of regex-case is a bit preliminary: in particular I don't like the syntax for binding thngs very much, although I can't think of a better one. Currently therefore it's in my collection of toys: it will probably migrate from there at some point.

Currently documentation is here and source code is here.

11 Jan 2023 6:17pm GMT

Nicolas Hafner: Kandria is now out!

https://filebox.tymoon.eu//file/TWpZME1RPT0=

Kandria is now finally available for purchase and play!

I recommend buying it on Steam, as the algorithm there will help us bring the game in front of more people, as well. However, if that isn't a possibility for you, there's also options on Itch.io and through Xsolla on our webpage:

I am also live on Steam, Twitch, and YouTube right now, to celebrate the launch! Come on and hang out in the chat: https://stream.shinmera.com

I hope you all enjoy the game, and thank you very much for sticking with us for all this time!

11 Jan 2023 2:02pm GMT

10 Jan 2023

feedPlanet Lisp

Nicolas Hafner: Kandria launches tomorrow!

https://filebox.tymoon.eu//file/TWpZek9BPT0=

​Kandria launches tomorrow, on Wednesday the 11th, at 15:00 CET / 9:00 EST!

There'll be a launch stream for the occasion as well. It'll be live on Twitch! I'll be happy to answer any questions you may have about the game, and hope to see you there!​

Last opportunity to wishlist the game, too: https://kandria.com/steam

10 Jan 2023 1:08pm GMT

09 Jan 2023

feedPlanet Lisp

vindarel: These Years in Common Lisp: 2022 in review

And 2022 is over. The Common Lisp language and environment are solid and stable, yet evolve. Implementations, go-to libraries, best practices, communities evolve. We don't need a "State of the Ecosystem" every two weeks but still, what happened and what did you miss in 2022?

This is my pick of the most exciting, fascinating, interesting or just cool projects, tools, libraries and articles that popped-up during that time (with a few exceptions that appeared in late 2021).

This overview is not a "State of the CL ecosystem" (HN comments (133)) that I did in 2020, for which you can find complementary comments on HN.

I think this article (of sorts) is definitely helpful for onlookers to Common Lisp, but doesn't provide the full "story" or "feel" of Common Lisp, and I want to offer to HN my own perspective.

And, suffice to say, I tried to talk about the most important things, but this article (of sorts) is by no means a compilation of all new CL projects or all the articles published on the internet. Look on Reddit, Quicklisp releases, Github, and my favourite resources:

If I had to pick 3 achievements they would be:

Let's go for more.

Thanks to @k1d77a, @Hexstream, @digikar and @stylewarning for their feedback.

Table of Contents

Documentation

A newcomer to Lisp came, asked a question, and suddenly he created a super useful rendering of the specification. Check it out!

But that's not all, he also started work on a new Common Lisp editor, built in Rust and Tauri, see below.

We continue to enrich the Common Lisp Cookbook. You are welcome to join, since documention is best built by newcomers for newcomers.

A resurrected project:

Also:

Implementations

We saw achievements in at least 7 8 implementations.

New implementation! It's 2022 and people start new CL implementations.

See also:

They are doing great work to revive a Lisp machine:

Medley Interlisp is a project aiming to restore the Interlisp-D software environment of the Lisp Machines Xerox produced since the early 1980s, and rehost it on modern operating systems and computers. It's unique in the retrocomputing space in that many of the original designers and implementors of major parts of the system are participating in the effort.

Paolo Amoroso blog post: my encounter with Medley Interlisp.

Jobs

I won't list expired job announces, but this year Lispers could apply for jobs in: web development(WebCheckout, freelance announces), cloud service providers (Keepit), big-data analysis (Ravenpack, and chances are they are still hiring)), quantum computing (HLR Laboratories), AI (Mind AI, SRI International), real-time data aggregration and alerting engines for energy systems (3E); for a startup building autism tech (and using CLOG already); there have been a job seeking to rewrite a Python backend to Common Lisp (RIFFIT); there have been some bounties; etc.

Prior Lisp experience was not 100% necessary. There were openings for junior and senior levels, remote and not remote (Australia for "a big corp", U.S., Spain, Ukraine...).

Comes a question:

I remind the reader that most Lisp jobs do not have a public job posting, instead candidates are often found organically on the community channels: IRC, Twitter, Discord, Reddit... or teams simply train their new developer.

In 2022 we added a few companies to the ongoing, non-official list on awesome-lisp-companies. If your company uses Common Lisp, feel free to tell us on an issue or in the comments!

For example, Feetr.io "is entirely Lisp".

Lisp was a conscious decision because it allows a small team to be incredibly productive, plus the fact that it's a live image allows you to connect to it over the internet and poke and prod the current state, which has really allowed a much clearer understanding of the data.

They post SLY screenshots on their Twitter^^

Evacsound (HN):

We're using CL in prod for an embedded system for some years now, fairly smooth sailing. It started out as an MVP/prototype so implementation was of no concern, then gained enough velocity and market interest that a rewrite was infeasible. We re-train talent on the job instead.

Pandorabots, or barefootnetworks, designing the Intel Tofino programmable switches, and more.

Projects

Language libraries

Editors, online editors, REPLs, plugins

New releases:

Concurrency

See also lisp-actors, which also does networking. It looks like more of a research project, as it doesn't have unit-tests nor documentation, but it was used for the (stopped) Emotiq blockchain.

Discussions:

Databases

More choices: awesome-cl#databases.

Delivery tools

There has been outstanding work done there. It is also great to see the different entities working on this. That includes SBCL developers, Doug Katzman of Google, and people at HRL Laboratories (also responsible of Coalton, Haskell-like on top of CL).

Have you ever wanted to call into your Lisp library from C? Have you ever written your nice scientific application in Lisp, only to be requested by people to rewrite it in Python, so they can use its functionality? Or, maybe you've written an RPC or pipes library to coordinate different programming languages, running things in different processes and passing messages around to simulate foreign function calls.

[...] If you prefer using SBCL, you can now join in on the cross-language programming frenzy too.

Games

Kandria launches on Steam on the 11th of January, in two days!

πŸŽ₯ Kandria trailer.

Graphics, GUIs

We saw the release of fresh bindings for Gtk4.

We had bindings for Qt5... but they are still very rough, hard to install so far.

Also:

History:

But an awesome novelty of 2022 is Kons-9.

Kons-9, a new 3D graphics project

πŸš€ A new 3D graphics project: Kons-9.

The idea would be to develop a system along the lines of Blender/Maya/Houdini, but oriented towards the strengths of Common Lisp.

I'm an old-time 3D developer who has worked in CL on and off for many years.

I don't consider myself an expert [...] A little about me: Β· wrote 3D animation software used in "Jurassic Park" Β· software R&D lead on "Final Fantasy: The Spirits Within" movie Β· senior software developer on "The Hobbit" films.

Interfaces with other languages

For more, see awesome-cl.

Numerical and scientific

New releases:

Call to action:

Web

Screenshotbot (Github) was released. It is "a screenshot testing service to tie with your existing Android, iOS and Web screenshot tests".

It is straightforward to install with a Docker command. They offer more features and support with their paid service.

LicensePrompt was released. It is "a single place to track all recurring software and IT expenses and send relevant reminders to all interested people". It's built in CL, interface with HTMX.

Libraries:

Skeletons:

Bindings:

Apps:

I don't have lots of open-source apps to show. Mines are running in production and all is going well. I share everything on my blog posts. I also have an open-source one in development, but that's for the 2023 showcase :D

CLOG

πŸš€ The awesome novelty of 2022 I spoke of in the introduction is CLOG, the Common Lisp Omnificent GUI:

The CLOG system browser

I know of one open-source consequent CLOG app: mold-desktop, in development.

I'm developing a programmable desktop and a bookmarks manager application with CLOG. I think I know one of the things that make CLOG user interfaces so easy to develop. It is that they are effortlessly composable. That's it for now :)

@mmontone

New releases

There are lots of awesome projects in music composition, including OpusModus and OpenMusic which saw new releases. I also like to cite ScoreCloud, a mobile app built with LispWorks, where you whistle, sing or play your instrument, and the app writes the music score O_o

See awesome-cl and Cliki for more.

(re) discoveries

Articles

Graphics

Tooling

Scripting

Around the language

History:

Web related

Call for action:

Other articles

Screencasts and podcasts

New videos by me:

by Gavin Freeborn:

KONS-9 series:

CLOG series:

CL study group:

Others:

and of course, find 3h48+ of condensed Lisp content on my Udemy video course! (I'm still working on new content, as a student you get updates).

Aside screncasts, some podcasts:

Other discussions

Community

Learning Lisp

Common Lisp VS ...

Thanks everyone, happy lisping and see you around!

09 Jan 2023 6:54pm GMT

08 Jan 2023

feedPlanet Lisp

Nicolas Martyanoff: Improving Git diffs for Lisp

All my code is stored in various Git repositories. When Git formats a diff between two objects, it generates a list of hunks, or groups of changes.

Each hunk can be displayed with a title which is automatically extracted. Git ships with support for multiple languages, but Lisp dialects are not part of it. Fortunately Git lets users configure their own extraction.

The first step is to identify the language using a pattern applied to the filename. Edit your Git attribute file at $HOME/.gitattributes and add entries for both Emacs Lisp and Common Lisp:

*.lisp diff=common-lisp
*.el diff=elisp

Then edit your Git configuration file at $HOME/.gitconfig and configure the path of the Git attribute file:

[core]
    attributesfile = ~/.gitattributes

Finally, set the regular expression used to match a top-level function name:

[diff "common-lisp"]
    xfuncname="^\\((def\\S+\\s+\\S+)"
    
[diff "elisp"]
    xfuncname="^\\((((def\\S+)|use-package)\\s+\\S+)"

For Lisp dialects, we do not just identify function names: it is convenient to identify hunks for all sorts of top-level definitions. We use a regular expression which captures the first symbol of the form and the name that follows.

Of course you can modifiy these expressions to identify more complex top-level forms. For example, for Emacs Lisp, I also want to identify use-package expressions.

You can see the result in all tools displaying Git diffs, for example in Magit with Common Lisp code:

Common Lisp diff

Or for my Emacs configuration file:

Emacs Lisp diff

Hunk titles, highlighted in blue, now contain the type and name of the top-level construction the changes are associated with.

A simple change, but one which really helps reading diffs.

08 Jan 2023 6:00pm GMT