fidzu : software

We are pleased to announce that Infomaniak has committed to sponsor DebConf25 as a Platinum Sponsor.

Infomaniak is Switzerland's leading developer of Web technologies. With operations all over Europe and based exclusively in Switzerland, the company designs and manages its own data centers powered by 100% renewable energy, and develops all its solutions locally, without outsourcing. With millions of users and the trust of public and private organizations across Europe - such as RTBF, the United Nations, central banks, over 3,000 radio and TV stations, as well as numerous cities and security bodies - Infomaniak stands for sovereign, sustainable and independent digital technology. The company offers a complete suite of collaborative tools, cloud hosting, streaming, marketing and events solutions, while being owned by its employees and self-financed exclusively by its customers.

With this commitment as Platinum Sponsor, Infomaniak is contributing to the Debian annual Developers' conference, directly supporting the progress of Debian and Free Software. Infomaniak contributes to strengthen the community that collaborates on Debian projects from all around the world throughout all of the year.

Thank you very much, Infomaniak, for your support of DebConf25!

Become a sponsor too!

DebConf25 will take place from 14th to July 20th 2025 in Brest, France, and will be preceded by DebCamp, from 7th to 13th July 2025.

DebConf25 is accepting sponsors! Interested companies and organizations should contact the DebConf team through sponsors@debconf.org, or visit the DebConf25 website at https://debconf25.debconf.org/sponsors/become-a-sponsor/.

25 Jan 2025 10:22am GMT

Fixed a major crash bug in our apps that use webengine, I also went ahead and updated these to core24 https://bugs.launchpad.net/snapd/+bug/2095418 andhttps://bugs.kde.org/show_bug.cgi?id=498663

Fixed okular
Can't import certificates to digitally sign in Okular https://bugs.kde.org/show_bug.cgi?id=498558 Can't open files https://bugs.kde.org/show_bug.cgi?id=421987 and https://bugs.kde.org/show_bug.cgi?id=415711

Skanpage won't launch https://bugs.kde.org/show_bug.cgi?id=493847 in -edge please help test.

Ghostwriter https://bugs.kde.org/show_bug.cgi?id=481258

New KDE Snaps!

Kalm - Breathing techniques

Telly-skout - Display TV guides

Kubuntu: Plasma 5.27.12 has been uploaded to archive -proposed and should make the .2 release!

I hate asking but I am unemployable with this broken arm fiasco. If you could spare anything it would be appreciated! https://gofund.me/573cc38e

24 Jan 2025 8:00pm GMT

I'm going to FOSDEM 2025!

As usual, I'll be in the Java Devroom for most of that day, which this time around is Saturday.

Please recommend me any talks!

This is my shortlist so far:

• no more boot loader: boot using the Linux kernel
• aerc, an email client for the discerning hacker
• Supersonic retro development with Docker
• Raiders of the lost hard drive
• Rediscovering the fun of programming with the Game Boy
• Fixing CVEs on Debian: almost everything you should know about it
• Building the Future: Understanding and Contributing to Immutable Linux Distributions
• Generating immutable, A/B updatable, securely booting Debian images
• a tale of several distros joining forces for a common goal: reproducible builds
• Finding Anomalies in the Debian Packaging System to Detect Supply Chain Attacks
• The State of OpenJDK
• Project Lilliput - Looking Back and Ahead
• (Almost) everything I knew about Java performance was wrong
• Reduce the size of your Java run-time image
• Project Leyden - Past and the Future
• DMARCaroni: where do DMARC reports go after they are sent?

24 Jan 2025 9:41am GMT

Our compay proides us with GitHub copilot, which is yet another example of an "AI" engine. I've reviewed it before and I wasn't particulary impressed. It writes mediocre code.

However, I found a use case where it really shines. I was setting up the blank project repositories for our upcoming hackathon. This involved creating a bunch of otherwise empty repositories with a customize README.md file. The simple solution was to automate this with an ad hoc shell script.

I don't make a practice of writing shell scripts. I write a few every now and then, but I don't live and breathe bash. It takes me a while because I forget the flags. I have to look up the exact syntax for things like the find command. Then I have to fight with the shell to get the quoting and escaping right. It's a tedious chore. No doubt I would become adept at it if it were something I did daily, but it's not.

So I decided to ask GitHub copilot to write some scripts for me. I gave it my problem statement: start with a file with a list of repository and description pairs. For each pair, look for the subdirectory with that name and descend into it. Use the sed command to modify the README.md file in the subdirectory to add the repository discription. Then use git to commit and push the change.

Copilot gave this a couple of seconds thought and came up with this script:

while IFS= read -r line; do
  dir_name=$(echo "$line" | cut -d' ' -f1)
  description=$(echo "$line" | cut -d' ' -f2-)
  if [ -d "$dir_name" ]; then
    sed -i "s/\[\[github_repo_description\]\]/$description/g" "$dir_name/README.md"
  else
    echo "Directory $dir_name does not exist."
  fi
done < descriptions.txt

find . -type d -name "hack*" -exec sh -c '
  cd "$0"
  if [ -f README.md ]; then
    git add README.md
    git commit -m "Update description."
    git push
  fi
' {} \;       

It isn't a script that'd I'd write home about, but it got the job done. I ran the script and had the repositories set up in couple of minutes. I could have written the script myself, but it would have take me longer. I had forgotten the options to the find command. I had forgotten the arguments to the cut command. I would have had to look them up. Copilot saved me that time.

A co-worker of mine questioned the engineering tradeoff of using a resource hog like generative AI to write crappy, throwaway shell scripts. From the standpoint of an indiviual developer, though, this is the first use case for copilot that I've where it actualy saved me time and effort.

18 Jan 2025 5:38am GMT

As in previous years, some small rooms will be available for Birds of a Feather sessions. The concept is simple: Any project or community can reserve a timeslot (30 minutes or 1 hour) during which they have the room just to themselves. These rooms are intended for ad-hoc discussions, meet-ups or brainstorming sessions. They are not a replacement for a developer room and they are certainly not intended for talks. Schedules: BOF Track A, BOF Track B, BOF Track C. To apply for a BOF session, enter your proposal at https://fosdem.org/submit. Select any of the BOF tracks and mention in舰

17 Jan 2025 11:00pm GMT

Iteration is simply that special case of recursion that doesn't accumulate storage in the long term. It's a notable special case because computer storage is finite, and you want to be able to write agorithms that are bound by constant space.

There are two common strategies that computer languages use to approach iteration. Functional languages like Scheme and Haskell make sure that normal function calls do not accumulate storage per se. Function calls can be used to direct the control flow, and if they direct the control flow in a loop, you will iterate. Most other languages achieve iteration via special iteration constructs that you must use if you want to iterate. Each of these approaches has its own advantages and disadvantages.

The advantage of using special iteration constructs are these:

• It is clear that you are iterating.
• Special constructs are usually optimized for iteration and have particular compiler support to make them efficient.
• Special constructs are constrained so that you cannot accidentally write non-iterative code.

The disadvantage of using special iteration constructs are these:

• Special constructs are drawn from a fixed set of constructs that are built in to the language. If you want to iterate differently, you are out of luck.
• Special constructs usually do not cross function boundaries. Iteration must reside in a single function.
• You have to decide beforehand that you want to iterate and choose an iteration construct.
• Special constructs are usually imperative in nature and operate via side effects.

The alternative approach used by functional languages is to make the language implementation tail recursive. This has these advantages:

• Iteration is automatic. You don't have to decide that you want to iterate, it just happens when it can.
• Iteration can cross function boundaries.
• You can write your own iteration constructs and build them out of ordinary functions.
• Iteration can be done purely functionally, without side effects.

The disadvantages of using tail recursion for iteration are these:

• It is not obvious that you are iterating or intended to.
• You have to be careful to place all the iteration in tail position or you will blow the stack. Beginner programmers often have difficulty recognizing which calls are tail calls and can find it hard to avoid blowing the stack.
• Small, innocent looking changes in the code can change its behavior to be non tail recursive, again blowing the stack.
• The stack no longer contains a complete call history. If you rely on the stack as a call history buffer for debugging, you may find debugging more difficult.

The code in an iteration can be classified as being part of the machinery of iteration - the part that sets up the itertion, tests the ending conditional, and advances to the next iteration - or part of the logic of the iteration - the specific part that you are repeating. The machinery of the iteration is usually the same across many iterations, while the logic of the iteration is idiomatic to the specific instance of iteration. For example, all iterations over a list will have a null test, a call to CDR to walk down the list, and a call to CAR to fetch the current element, but each specific iteration over a list will do something different to the current element.

There are several goals in writing iterative code. One is to have efficient code that performs well. Another is to have clear code that is easy to understand, debug, and maintain. You choose how to iterate based on these considerations. For the highest performing code, you will want detailed control over what the code is doing. You may wish to resort to using individual assignments and GOTO statements to squeeze the last clock cycles out of an inner loop. For the clearest code, you will want to use a high degree of abstraction. A clever compiler can generate efficient code from highly abstracted code, and experienced programmers know how to write abstract code that can be compiled to efficient code.

Here are some examples of iteration strategies Lisp. To make these examples easy to compare I chose a simple problem to solve: given a list of numbers, return both a list of the squares of the numbers and the sum of the squares. This is a simple problem that can be solved in many ways.

Tagbody and Go

A tagbody is a block of code that is labeled with tags. You can jump to a tag with a go statement. This is a very low level form of iteration that is not used much in modern Lisp programming. Here is an example of a tagbody:

(defun iteration-example-with-tagbody (numbers)
  (let ((squares '())
        (total 0)
        (nums numbers))
    (tagbody
     start
       (if (null nums)
           (go end))
       (let ((square (* (car nums) (car nums))))
         (setq squares (cons square squares))
         (incf total square))
       (setq nums (cdr nums))
       (go start)
     end
       (values (nreverse squares) total))))

This is like programming in assembly code. The go instructions turn into jumps. This code is very efficient, but it is not particularly clear. The machinery of the iteration is mixed in with the logic of the iteration, making it hard to see what is going on. The code is not very abstract.

State Machine via Mutual Tail Recursion

Here we use tail recursion to iterate. The compiler will turn the tail recursive call into a jump and the variable rebinding into assignments, so this code will be about as efficient as the tagbody code above.

(defun iteration-example-tail-recursive (numbers &optional (squares '()) (total 0))
  (if (null numbers)
      (values (nreverse squares) total)
      (let ((square (* (car numbers) (car numbers))))
        (iteration-example-tail-recursive
         (cdr numbers)
         (cons square squares)
         (+ total square)))))

This state machine only has one state, so it is not a very interesting state machine. The ultimate in iteration control is to write an iterative state machine using mutually tail recursive functions. The compiler will generate very efficient code for this, and you can write the code in a very abstract way. Here is an example of a state machine that simulates the action of a turnstile:

(defun turnstile (actions)
  "State machine to simulate a turnstile with actions 'push', 'coin', and 'slug'."
  (locked-state actions '() '()))

(defun locked-state (actions collected return-bucket)
  (cond ((null actions) (list collected return-bucket))
        ((eql (car actions) 'coin)
         (unlocked-state (cdr actions) collected return-bucket))
        ((eql (car actions) 'push)
         (locked-state (cdr actions) collected return-bucket))  ;; Ignore push in locked state
        ((eql (car actions) 'slug)
         (locked-state (cdr actions) collected (append return-bucket '(slug)))) ;; Return slug
        (t (locked-state (cdr actions) collected return-bucket))))

(defun unlocked-state (actions collected return-bucket)
  (cond ((null actions) (list collected return-bucket))
        ((eql (car actions) 'push)
         (locked-state (cdr actions) (append collected '(coin)) return-bucket))
        ((eql (car actions) 'coin)
         (unlocked-state (cdr actions) collected (append return-bucket '(coin)))) ;; Return coin
        ((eql (car actions) 'slug)
         (unlocked-state (cdr actions) collected (append return-bucket '(slug)))) ;; Return slug
        (t (unlocked-state (cdr actions) collected return-bucket))))

;; Example usage:
(turnstile '(coin push coin push))  ;; => ((coin coin) ())
(turnstile '(push coin push))       ;; => ((coin) ())
(turnstile '(coin coin push push))  ;; => ((coin) (coin))
(turnstile '(push))                 ;; => (NIL NIL)
(turnstile '(coin push push))       ;; => ((coin) ())
(turnstile '(coin coin coin push))  ;; => ((coin) (coin coin))
(turnstile '(slug coin push))       ;; => ((coin) (slug))
(turnstile '(coin slug push))       ;; => ((coin) (slug))
(turnstile '(slug slug push coin push)) ;; => ((coin) (slug slug))

The iteration machinery is still interwoven with the logic of the code. We're still finding calls to null and cdr sprinkled around the code. Nonetheless, structuring iterative code this way is a big step up from using a tagbody and go. This is my go-to method for compex iterations that cannot easily be expressed as a map or reduce.

Loop Macro

Common Lisp's loop macro is a very powerful iteration construct that can be used to express a wide variety of iteration patterns.

defun loop-iteration-example (numbers)
  (loop for num in numbers
        for square = (* num num)
        collect square into squares
        sum square into total
        finally (return (values squares total))))

Call me a knee-jerk anti-loopist, but this doesn't look like Lisp to me. It has some major problems:

• It is highly imperative. To understand what is going on, you have to follow the code in the order it is written. You need to have a mental model of the state of the loop at each point in the iteration. Running into a loop when reading functional code takes you out of the zen of functional programming.
• The bound variables are not lexical, they are scattered around the code. You have to carefully examine each clause to figure out what variables are being bound.
• You need a parser to walk the code. There is nothing that delimits the clauses of the loop; it is a flat list of random symbols and forms. You couldn't easily write a program that takes a loop form and transforms it in some way.

Do and Friends

The do macro, and its friends dolist, dotimes, and do*, etc., are the most common iteration constructs in Common Lisp.

(defun iteration-example-with-do (numbers)
  (let ((squares '())
        (total 0))
    (do ((nums numbers (cdr nums)))
        ((null nums) (values (nreverse squares) total))
      (let ((square (* (car nums) (car nums))))
        (setq squares (cons square squares))
        (incf total square)))))

The do macros have some drawbacks:

• They are imperative. The body of a do loop ultimately must have some sort of side effect or non-local exit to "get a value out". Notice how we bind accumulator variables in an outer scope and assign them in the inner one. This is a common pattern in a do loop.
• They do not compose. You can nest a dotimes inside a dolist, e.g., but you cannot run a dotimes in parallel with a dolist.
• They are incomplete. There is no do-array or do-string, for example.

But at least you can parse them and transform them. They are structured, and you can write a program that walks the clauses of a do loop and does something with them.

Map and Reduce

Map and reduce abstract the machinery of iteration away from the logic of the iteration through use of a monoid (a higher order function). The resulting code is clear and concise:

(defun iteration-example-with-map-reduce (numbers)
  (let* ((squares (map 'list (lambda (num) (* num num)) numbers))
         (total (reduce #'+ squares)))
    (values squares total)))

The looping is implicit in the mapcar and reduce functions. You can usually make the assumption that the language implemetors have optimized these functions to be reasonably efficient.

I often see programmers writing looping code when a perfectly good library function exists that does the same thing. For example, it is common to want to count the number of items in a sequence, and Commmon Lisp supplies the count function just for this purpose. There is no need to write a loop.

Common Lisp provides a filter function, but it is called remove-if-not.

The drawback of using these functions is that large intermediate sequences can be created. In our example code, the entire list of squares is constructed prior to reducing it with #'+. Of course the entire list is one of the return values, so you need it anyway, but if you only needed the sum of the squares, you would prefer to sum it incrementally as you go along rather than constructing a list of squares and then summing it. For small sequences, it doesn't make a difference.

Series

The series macro suite attempt to bring you best of both worlds. You write series expressions that look like sequence functions, but the macro recognizes that you are iterating and generates efficient incremental code.

(defun iteration-example-with-series (numbers)
  (let ((squares (map-fn 'integer (lambda (n) (* n n)) (scan 'list numbers)))
    (values (collect 'list squares)
            (collect-sum squares))))

This code is very similar to the sequence case, but the series macro will generate code that does not construct the entire list of squares before summing them. It will sum them incrementally as it goes along.

Series will expand into a tagboy. For example, the above code will expand into something like this:

(COMMON-LISP:LET* ((#:OUT-1015 NUMBERS))
  (COMMON-LISP:LET (#:ELEMENTS-1012
                    (#:LISTPTR-1013 #:OUT-1015)
                    (SQUARES 0)
                    #:SEQ-1018
                    (#:LIMIT-1019
                     (COMMON-LISP:MULTIPLE-VALUE-BIND (SERIES::X SERIES::Y)
                         (SERIES::DECODE-SEQ-TYPE (LIST 'QUOTE 'LISTS))
                       (DECLARE (IGNORE SERIES::X))
                       SERIES::Y))
                    (#:LST-1020 NIL)
                    (#:SUM-1023 0))
    (DECLARE (TYPE LIST #:LISTPTR-1013)
             (TYPE INTEGER SQUARES)
             (TYPE (SERIES::NULL-OR SERIES::NONNEGATIVE-INTEGER) #:LIMIT-1019)
             (TYPE LIST #:LST-1020)
             (TYPE NUMBER #:SUM-1023))
    (TAGBODY
     #:LL-1026
      (IF (ENDP #:LISTPTR-1013)
          (GO SERIES::END))
      (SETQ #:ELEMENTS-1012 (CAR #:LISTPTR-1013))
      (SETQ #:LISTPTR-1013 (CDR #:LISTPTR-1013))
      (SETQ SQUARES ((LAMBDA (N) (* N N)) #:ELEMENTS-1012))
      (SETQ #:LST-1020 (CONS SQUARES #:LST-1020))
      (SETQ #:SUM-1023 (+ #:SUM-1023 SQUARES))
      (GO #:LL-1026)
     SERIES::END)
    (COMMON-LISP:LET ((SERIES::NUM (LENGTH #:LST-1020)))
      (DECLARE (TYPE SERIES::NONNEGATIVE-INTEGER SERIES::NUM))
      (SETQ #:SEQ-1018 (MAKE-SEQUENCE 'LISTS (OR #:LIMIT-1019 SERIES::NUM)))
      (DO ((SERIES::I (1- SERIES::NUM) (1- SERIES::I)))
          ((MINUSP SERIES::I))
        (SETF (ELT #:SEQ-1018 SERIES::I) (POP #:LST-1020))))
    (VALUES #:SEQ-1018 #:SUM-1023)))

90% of the time, the series macro will produce very efficient code, but 10% of the time the macro loses its lunch. It takes a little practice to get use to when the series macro will work and to write code that the series macro can handle.

Conclusion

There are many ways to iterate in Lisp, some are more efficient than others, some are more abstrac than others. You choose the way that suits your needs. I like the abstraction of the series macro, but I will also use a library function like count when it is appropriate. When I need tight control, I'll write a state machine.

17 Jan 2025 8:36pm GMT

If your non-geek partner and/or kids are joining you to FOSDEM, they may be interested in spending some time exploring Brussels while you attend the conference. Like previous years, FOSDEM is organising sightseeing tours.

16 Jan 2025 11:00pm GMT

With FOSDEM just a few days away, it is time for us to enlist your help. Every year, an enthusiastic band of volunteers make FOSDEM happen and make it a fun and safe place for all our attendees. We could not do this without you. This year we again need as many hands as possible, especially for heralding during the conference, during the buildup (starting Friday at noon) and teardown (Sunday evening). No need to worry about missing lunch at the weekend, food will be provided. Would you like to be part of the team that makes FOSDEM tick?舰

16 Jan 2025 11:00pm GMT

I just released a new documentation website specialized on web development in Common Lisp:

• 🚀 https://web-apps-in-lisp.github.io/

I'd be embarrassed to tell how long it took me to grasp all the building blocks and to assemble a resource that makes sense. I hope it serves you well, now don't hesitate to share what you are building, it creates emulation!

In the first tutorial we build a simple app that shows a web form that searches and displays a list of products.

We see many necessary building blocks to write web apps in Lisp:

• how to start a server
• how to create routes
• how to define and use path and URL parameters
• how to define HTML templates
• how to run and build the app, from our editor and from the terminal.

In doing so, we'll experience the interactive nature of Common Lisp.

In the user log-in section, we build a form that checks a user name and a password:

We also introduce databases, and more topics.

The sources are here: https://github.com/web-apps-in-lisp/web-apps-in-lisp.github.io/ and the GitHub Discussions are open.

15 Jan 2025 9:39am GMT