20 Sep 2020

feedPlanet Debian

Dirk Eddelbuettel: RcppSpdlog 0.0.2: New upstream, awesome new stopwatch

Following up on the initial RcppSpdlog 0.0.1 release earlier this week, we are pumped to announce release 0.0.2. It contains upstream version 1.8.0 for spdlog which utilizes (among other things) a new feature in the embedded fmt library, namely completely automated formatting of high resolution time stamps which allows for gems like this (taken from this file in the package and edited down for brevity):

    spdlog::stopwatch sw;                                   // instantiate a stop watch

    // some other code

    sp->info("Elapsed time: {}", sw);

What we see is all there is: One instantiates a stopwatch object, and simply references it. The rest, as they say, is magic. And we get tic / toc alike behaviour in modern C++ at essentially no cost us (as code authors). So nice. Output from the (included in the package) function exampleRsink() (again edited down just a little):

R> RcppSpdlog::exampleRsink()
[13:03:23.845114] [fromR] [I] [thread 793264] Welcome to spdlog!
[13:03:23.845205] [fromR] [I] [thread 793264] Elapsed time: 0.000103611
[13:03:23.845281] [fromR] [I] [thread 793264] Elapsed time: 0.00018203

We see that the two simple logging instances come 10 and 18 microseconds into the call.

RcppSpdlog bundles spdlog, a wonderful header-only C++ logging library with all the bells and whistles you would want that was written by Gabi Melman, and also includes fmt by Victor Zverovic.

The NEWS entry for this release follows.

Changes in RcppSpdlog version 0.0.2 (2020-09-17)

  • Upgraded to upstream release 1.8.0

  • Switched Travis CI to using BSPM, also test on macOS

  • Added 'stopwatch' use to main R sink example

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppSpdlog page.

The only sour grapes, again, are once more over the CRAN processing. And just how 0.0.1 was delayed for no good reason for three weeks, 0.0.2 was delayed by three days just because … well that is how CRAN rules sometimes. I'd be even more mad if I had an alternative but I don't. We remain grateful for all they do but they really could have let this one through even at one-day update delta. Ah well, now we're three days wiser and of course nothing changed in the package.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

20 Sep 2020 6:21pm GMT

feedPlanet Python

John Cook: Descartes and Toolz

I was looking recently at the Python module toolz, a collection of convenience functions. A lot of these functions don't do that much. They don't save you much code, but they do make your code more readable by making it more declarative. You may not realize need them until you see them.

For example, there is a function partitionby that breaks up a sequence at the points where a given function's value changes. I'm pretty sure that function would have improved some code I've written recently, making it more declarative than procedural, but I can't remember what that was.

Although I can't think of my previous example, I can think of a new one, and that is Descartes' rule of signs.

Given a polynomial p(x), read the non-zero coefficients in order and keep note of how many times they change sign, either from positive to negative or vice versa. Call that number n. Then the number of positive roots of p(x) either equals n or n minus a positive even number.

For example, suppose

p(x) = 4x4 + 3.1x3 - x2 - 2x + 6.

The coefficients are 4, 3.1, -1, -2, and 6. The list of coefficients changes signs twice: from positive to negative, and from negative to positive. Here's a first pass at how you might have Python split the coefficients to look sign changes.

    from toolz import partitionby

    coefficients = [4, 3.1, -1, -2, 6]
    parts = partitionby(lambda x: x > 0, coefficients)
    print([p for p in parts])

This prints

    [(4, 3.1), (-1, -2), (6,)]

The first argument to partitionby an anonymous function that tests whether its argument is positive. When this function changes value, we have a sign alteration. There are three groups of consecutive coefficients that have the same sign, so there are two times the signs change. So our polynomial either has two positive roots or no positive roots. (It turns out there are no positive roots.)

The code above isn't quite right though, because Descartes said to only look at non-zero coefficients. If we change our anonymous function to

    lambda x: x >= 0

that will work for zeros in the middle of positive coefficients, but it will give a false positive for zeros in the middle of negative coefficients. We can fix the code with a list comprehension. The following example works correctly.

    coefficients = [4, 0, 3.1, -1, 0, -2, 6]
    nonzero = [c for c in coefficients if c != 0]
    parts = partitionby(lambda x: x > 0, nonzero)
    print([p for p in parts])

If our coefficients were in a NumPy array rather than a list, we could remove the zeros more succinctly.

    from numpy import array

    c = array(coefficients)
    parts = partitionby(lambda x: x > 0, c[c != 0])

The function partitionby returns an iterator rather than a list. That's why we don't just print parts above. Instead we print [p for p in parts] which makes a list. In applications, it's often more efficient to have an iterator than a list, generating items if and when they are needed. If you don't need all the items, you don't have to generate them all. And even if you do need all the items, you could save memory by not keeping them all in memory at once. I'll ignore such efficiencies here.

We don't need the partitions per se, we just need to know how many there are. The example that escapes my mind would have been a better illustration if it needed to do more with each portion than just count it. We could count the number of sign alternations for Descartes rule as follows.

   len([p for p in parts]) - 1

Related posts

The post Descartes and Toolz first appeared on John D. Cook.

20 Sep 2020 3:22pm GMT

feedPlanet Debian

Daniel Lange: Getting rid of the Google cookie consent popup

If you clear your browser cookies regularly (as you should do), Google will annoy you with a full screen cookie consent overlay these days. And - of course - there is no "no tracking consent, technically required cookies only" button. You may log in to Google to set your preference. Yeah, I'm sure this is totally following the intent of the EU Directive 2009/136/EC (the "cookie law").

Google cookie consent pop-up

Unfortunately none of the big "anti-annoyances" filter lists seem to have picked that one up yet but the friendly folks from the Computerbase Forum [German] to the rescue. User "Sepp Depp" has created the following filter set that WFM:

Add this to your uBlock Origin "My filters" tab:

! Google - remove cookie-consent-popup and restore scoll functionality
google.*##html:style(overflow: visible !important;)

20 Sep 2020 2:11pm GMT

feedPlanet Python

Kodnito: Tweet from Django application using Tweepy

In this tutorial, we will learn how to post a tweet from Django application using Tweepy.

20 Sep 2020 12:25pm GMT

Evennia: Creating Evscaperoom, part 1

Over the last month (April-May 2019) I have taken part in the Mud Coder's Guild Game Jam "Enter the (Multi-User) Dungeon". This year the theme for the jam was One Room.

The result was Evscaperoom, an text-based multi-player "escape-room" written in Python using the Evennia MU* creation system. You can play it from that link in your browser or MU*-client of choice. If you are so inclined, you can also vote for it here in the jam (well, no more).

This little series of (likely two) dev-blog entries will try to recount the planning and technical aspects of the Evscaperoom. This is also for myself - I'd better write stuff down now while it's still fresh in my mind!

Update: The next part of this blog is here.

Update 2: The Evscaperoom can these days be played by connecting to the Evennia game demo at https://demo.evennia.com.


When I first heard about the upcoming game-jam's theme of One Room, an 'escape room' was the first thing that came to mind, not the least because I just recently got to solve my my own first real-world escape-room as a gift on my birthday.

If you are not familiar with escape-rooms, the premise is simple - you are locked into a room and have to figure out a way to get out of it by solving practical puzzles and finding hidden clues in the room.

While you could create such a thing in your own bedroom (and there are also some one-use board game variants), most escape-rooms are managed by companies selling this as an experience for small groups. You usually have one hour to escape and if you get stuck you can press a button (or similar) to get a hint.

I thought making a computer escape-room. Not only can you do things in the computer that you cannot do in the real world, restricting the game to a single room limits so that it's conceivable to actually finish the damned thing in a month.

A concern I had was that everyone else in the jam surely must have went for the same obvious idea. In the end that was not an issue at all though.

Basic premises

I was pretty confident that I would technically be able to create the game in time (not only is Python and Evennia perfect for this kind of fast experimentation and prototyping, I know the engine very well). But that's not enough; I had to first decide on how the thing should actually play. Here are the questions I had to consider:

Room State

An escape room can be seen as going through multiple states as puzzles are solved. For example, you may open a cabinet and that may open up new puzzles to solve. This is fine in a single-player game, but how to handle it in a multi-player environment?

My first thought was that each object may have multiple states and that players could co-exist in the same room, seeing different states at the same time. I really started planning for this. It would certainly be possible to implement.

But in the end I considered how a real-world escape-room works - people in the same room solves it together. For there to be any meaning with multi-player, they must share the room state.

So what I went with was a solution where players can create their own room or join an existing one. Each such room is generated on the fly (and filled with objects etc) and will change as players solve it. Once complete and/or everyone leaves, the room is deleted along with all objects in it. Clean and tidy.

So how to describe these states? I pictured that these would be described as normal Python modules with a start- and end function that initialized each state and cleaned it up when a new state was started. In the beginning I pictured these states as being pretty small (like one state to change one thing in the room). In the end though, the entire Evscaperoom fits in 12 state modules. I'll describe them in more detail in the second part of this post.

Accessibility and "pixel-hunting" in text

When I first started writing descriptions I didn't always note which objects where interactive. It's a very simple and tempting puzzle to add - mention an object as part of a larger description and let the player figure out that it's something they can interact with. This practice is sort-of equivalent to pixel-hunting in graphical games - sweeping with the mouse across the screen until you find that little spot on the screen that you can do something with.

Problem is, pixel-hunting's not really fun. You easily get stuck and when you eventually find out what was blocking you, you don't really feel clever but only frustrated. So I decided that I should clearly mark every object that people could interact with and focus puzzles on better things.

In fact, in the end I made it an option:

Option menu ('quit' to return) 1: ( ) No item markings (hard mode) 2: ( ) Items marked as item (with color) 3: (*) Items are marked as [item] (screenreader friendly) 4: ( ) Screenreader mode

As part of this I had to remind myself never to use colors only when marking important information: Visually impaired people with screen readers will simply miss that. Not to mention that some just disable colors in their clients.

So while I personally think option 2 above is the most visually pleasing, Evscaperoom defaults to the third option. It should should start everyone off on equal footing. Evennia has a screen-reader mode out of the box, but I moved it into the menu here for easy access.

Inventory and collaboration

In a puzzle-game, you often find objects and combine them with other things. Again, this is simple to do in a single-player game: Players just pick things up and use them later.

But in a multi-player game this offers a huge risk: players that pick up something important and then log off. The remaining players in that room would then be stuck in an unsolvable room - and it would be very hard for them to know this.

In principle you could try to 'clean' player inventories when they leave, but not only does it add complexity, there is another issue with players picking things up: It means that the person first to find/pick up the item is the only one that can use it and look at it. Others won't have access until the first player gives it up. Trusting that to anonymous players online is not a good idea.

So in the end I arrived at the following conclusions:

As a side-effect of this I also set a limit to the kind of puzzles I would allow:

Focusing on objects

So without inventory system, how do you interact with objects? A trademark of any puzzle is using one object with another and also to explore things closer to find clues. I turned to graphical adventure games for inspiration:

Hovering with mouse over lens object offers action
Secret of Monkey Island ©1990 LucasArts. Image from old-games.com

A common way to operate on an object in traditional adventure games is to hover the mouse over it and then select the action you want to apply to it. In later (3D) games you might even zoom in of the object and rotate it around with your mouse to see if there are some clues to be had.

While Evennia and modern UI clients may allow you to use the mouse to select objects, I wanted this to work the traditional MUD-way, by inserting commands. So I decided that you as a player would be in one of two states:

A small stone fireplace sits in the middle of the wall opposite the [door]. On the chimney hangs a small oil [painting] of a man. Hanging over the hearth is a black [cauldron]. The piles of [ashes] below are cold. (It looks like fireplace may be suitable to [climb].)

In the example above, the fireplace points out other objects you could also focus on, whereas the last parenthesis includes one or more "actions" that you can perform on the fireplace only when you have it focused.

This ends up pretty different from most traditional MUD-style inputs. When I first released this to the public, I found people logged off after their first examine. It turned out that they couldn't figure out how to leave the focus mode. So they just assumed the thing was buggy and quit instead. Of course it's mentioned if you care to write help, but this is clearly one step too many for such an important UI concept.

So I ended up adding the header above that always reminds you. And since then I've not seen any confusion over how the focus mode works.

For making it easy to focus on things, I also decided that each room would only ever have one object named a particular thing. So there is for example only one single object in the game named "key" that you can focus on.


I wanted players to co-exist in the same room so that they could collaborate on solving it. This meant communication must be possible. I pictured people would want to point things out and talk to each other.

In my first round of revisions I had a truckload of individual emotes; you could

point at target

for example. In the end I just limited it to

say/shout/whisper <message>


emote <whatever>

And seeing what people actually use, this is more than enough (say alone is probably 99% of what people need, really). I had a notion that the shout/whisper could be used in a puzzle later but in the end I decided that communication commands should be strictly between players and not have anything to do with the puzzles.

I removed all other interaction: There is no fighting and without an inventory or requirement to collaborate on puzzles, there is no need for other interactions than to communicate.

First version you didn't even see what the others did, but eventually I added so that you at least saw what other players were focusing on at the moment (and of course if some major thing was solved/found).

In the end I don't even list characters as objects in the room (you have to use the who command to see who's in there with you).

Listing of commands available in the Evscaperoom (output of the help-command in game)
The main help command output.


It's very common for this type of game to have a dangerous or scary theme. Things like "get out before the bomb explodes", "save the space ship before the engines overheat", "flee the axe murderer before he comes back" etc). I'm no stranger to dark themes, but for this I wanted something friendlier and brighter, maybe with a some dark undercurrents here and there.

My Jester character is someone I've not only depicted in art, but she's also an old RP character and literary protagonist of mine. Who else would find it funny to lock someone into a room only to provide crazy puzzles and hints for them to get out again? So my flimsy 'premise' was this:

The village Jester wants to win the pie eating contest. You are one of her most dangerous opponents. She tricked you to her cabin and now you are locked in! If you don't get out in time, she'll get to eat all those pies on her own and surely win!

That's it - this became the premise from which the entire game flowed. I quickly decided that it to be a very "small-scale" story: no life-or-death situation, no saving of the world. The drama takes place in a small village with an "adversary" that doesn't really want to hurt you, but only to eat more pies than you.

From this, the way to offer hints came naturally - just eat a slice of "hintberry pie" the jester made (she even encourage you to eat it). It gives you a hint but is also very filling. So if you eat too much, how will you beat her in the contest later, even if you do get out?

To further the rustic and friendly tone I made sure the story took place on a warm summer day. Many descriptions describe sunshine, chirping birds and the smell of pie. I aimed at letting the text point out quirky and slightly comedic tone of the puzzles the Jester left behind. The player also sometimes gets teased by the game when doing things that does not make sense.

I won't go into the story further here - it's best if you experience it yourself. Let's just say that the village has some old secrets. And and the Jester has her own ways of doing things and of telling a story. The game has multiple endings and so far people have drawn very different conclusions in the end.


Most often in escape rooms, final score is determined by the time and the number of hints used. I do keep the latter - for every pie you eat, you get a penalty on your final score.

As for time - this background story would fit very well with a time limit (get out in X time, after which the pie-eating contest will start!). But from experience with other online text-based games I decided against this. Not only should a player be able to take a break, they may also want to wait for a friend to leave and come back etc.

But more importantly, I want players to explore and read all my carefully crafted descriptions! So I'd much rather prefer they take their time and reward them for being thorough.

So in the end I give specific scores for actions throughout the game instead. Most points are for doing things that drive the story forward, such as using something or solving a puzzle. But a significant portion of the score comes from turning every stone and trying everything out. The nice side-effect of this is that even if you know exactly how to solve everything and rush through the game you will still not end up with a perfect score.

The final score, adjusted by hints is then used to determine if you make it in time to the contest and how you fare. This means that if you explore carefully you have a "buffer" of points so eating a few pies may still land you a good result in the end.

First sketch

I really entered the game 'building' aspect with no real notion of how the Jester's cabin should look nor which puzzles should be in it. I tried to write things down beforehand but it didn't really work for me.

So in the end I decided "let's just put a lot of interesting stuff in the room and then I'll figure out how they interact with each other". I'm sure this is different from game-maker to game-maker. But for me, this process worked perfectly.

Scribbles on my notebook, sketching up the room's main items
My first, very rough, sketch of the Jester's cabin

The above, first sketch ended up being what I used, although many of the objects mentioned never ended up in the final game and some things switched places. I did some other sketches too, but they'd be spoilers so I won't show them here ...

The actual game logic

The Evscaperoom principles outlined above deviate quite a bit from the traditional MU* style of game play.

While Evennia provides everything for database management, in-game objects, commands, networking and other resources, the specifics of your game is something you need to make yourself - and you have the full power of Python to do it!

So for the first three days of the jam I used Evennia to build the custom game logic needed to provide the evscaperoom style of game play. I also made the tools I needed to quickly create the game content (which then took me the rest of the jam to make).

In part 2 of this blog post I will cover the technical details of the Evscaperoom I built. I'll also go through some issues I ran into and conclusions I drew. I'll link to that from here when it's available!

Continue to part 2.

20 Sep 2020 10:39am GMT

feedPlanet Debian

Andy Simpkins: Using an IP camera in conference calls

My webcam has broken, something that I have been using a lot during the last few months for some reason.

A friend of mine suggested that I use the mic and camera on my mobile phone instead. There is a simple app 'droidcam' that makes the phone behave as a simple webcam, it also has a client application to run on your PC to capture the web-stream and present it as a video device. All well and good but I would like to keep propitiatory software off my PCs (I have a hard enough time accepting it on a phone but I have to draw a line somewhere).

I decided that there had to be a simple way to ingest that stream and present it as a local video device on a Linux box. It turns out that it is a lot simpler than I thought it would be. I had it working within 10 minutes!

Packages needed: ffmpeg, v4l2loopback-utils

sudo apt-get install ffmpeg v4l2loopback-utils

Start the loop-back device:

sudo modprobe v4l2loopback

Ingest video and present on loop-back device:

ffmpeg -re -i <URL_VideoStream> -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video0


Read input at native frame rate. Mainly used to simulate a grab
device, or live input stream (e.g. when reading from a file). Should
not be used with actual grab devices or live input streams (where it
can cause packet loss). By default ffmpeg attempts to read the
input(s) as fast as possible. This option will slow down the reading
of the input(s) to the native frame rate of the input(s). It is
useful for real-time output (e.g. live streaming).
-i <source>
In this case my phone running droidcam
-vcodec rawvideo
Select the output video codec to raw video (as expected for /dev/video#)
-pix_fmt yuv420p
Set pixel format. in this example tell ffmpeg that the video will
be yuv colour space at 420p resolution
-f v4l2 /dev/video0
Force the output to be in video for Linux 2 (v4l2) format bound to
device /dev/video0

20 Sep 2020 9:50am GMT

feedPlanet Lisp

Leo Zovic: The Prisoners Part 2

Dawn of the second day.

According to the internet, the thing I intend to build is called a Roguelikelike, teetering on the very edge of being a Roguelike. So it goes; we'll see if I end up taking the title or not.

Last time, we laid out the basics of prisoners, their interactions and their strategies. This time, lets get some different scenarios and some player interaction going.


Payoff matrices involve deciding who gets what bonus or penalty as a result of an interaction. Given a pair of defect/cooperate choices, a payoff-matrix will return the scores to be delivered to each player in turn.

(defun payoff-matrix (cc-a cc-b cd-a cd-b dc-a dc-b dd-a dd-b)
  (let ((tbl {(cons :cooperate :cooperate) (list cc-a cc-b)
              (cons :defect :cooperate) (list dc-a dc-b)
              (cons :cooperate :defect) (list cd-a cd-b)
              (cons :defect :defect) (list dd-a dd-b)}))
    (lambda (a b) (lookup tbl (cons a b)))))

Now we can define some basic scenarios. A dilemma is the name I'll pick for the situation where co-operating is better for the group, and both defecting is the worst thing for everyone, but a single defector will end out better off by defecting.

(defparameter dilemma
   3 3  1 5
   5 1  0 0))

A stag-hunt is a situation where a pair of players can pool their resources for a greater prize, and ignore each other for the lesser. If either player attempts to hunt the stag alone, they get nothing, while their defecting partner still gets a rabbit.

(defparameter stag-hunt
   3 3  0 1
   1 0  1 1))

A trade is one in which both parties benefit, but to which both parties must agree.

(defparameter trade
   3 3  0 0
   0 0  0 0))

A theft is one where a player takes from the other. But if both players cooperate, or both try to rob each other, they come to an impasse.

(defparameter theft
   0 0   -3 3
   3 -3   0 0))

A trap is a situation where cooperating leads to disaster, ignoring the situation leads to no gain, and defecting to make it clear to your partner that you don't intend to follow ends up benefiting both players.

(defparameter trap
   -3 -3  2 2
    2  2  0 0))

The last scenario I'll concern myself with is the mutual-prediction. Where guessing what your partner/opponent will choose benefits you, and failing to do so does nothing.

(defparameter mutual-prediction
   3 3  0 0
   0 0  3 3))


In order to move through the world, our prisoners need a world to move through. Let us begin at the ending.

(defparameter ending
  {:description "You have come to the end of your long, perilous journey."})

There is nothing to do at the end other than display this fact.

(defun repl! (adventure)
  (format t "~%~%~a~%~%" (lookup adventure :description)))

THE-PRISONERS> (repl! ending)

You have come to the end of your long, perilous journey.


But what led us here was a choice. An adventure is more than a description, it's also the options, a prisoner, the scenario, and a way to continue the action. continueing means making a choice and effectively playing the opposing/cooperating prisoner and abiding by the results.

(defun mk-adventure ()
  (let ((prisoner (polo)))
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner prisoner :scenario trade
     :continue (lambda (choice)
                 (let ((their-choice (play prisoner)))
                   (update! prisoner choice)
                   (funcall trade choice their-choice)

This sort of adventure also takes a bit more machinery to run from the repl. We need to present the description, but also get an appropriate choice from the user. Getting that choice is a bit more complicated than you might think at first.

(defun get-by-prefix (lst prefix)
  (let ((l (length prefix)))
    (loop for elem in lst
       when (and (>= (length elem) l)
                 (== (subseq elem 0 l) prefix))
       do (return elem))))

(defun get-repl-choice (adventure)
  (let* ((responses (mapcar #'string-downcase (list (lookup adventure :cooperate) (lookup adventure :defect))))
         (r-map {(string-downcase (lookup adventure :cooperate)) :cooperate
                 (string-downcase (lookup adventure :defect)) :defect})
         (by-pref nil)
         (resp ""))
    (loop until (and (symbolp resp)
                     (setf by-pref
                            (string-downcase (symbol-name resp)))))
       do (format
           t "~a/~a:"
           (lookup adventure :cooperate)
           (lookup adventure :defect))
       do (setf resp (read)))
    (lookup r-map by-pref)))

Well behaved players are easy to deal with, true...

THE-PRISONERS> (get-repl-choice (mk-adventure))

THE-PRISONERS> (get-repl-choice (mk-adventure))

THE-PRISONERS> (get-repl-choice (mk-adventure))


... but we want to be a bit more general than that.

THE-PRISONERS> (get-repl-choice (mk-adventure))
Accept/Refuse:fuck you
Accept/Refuse: (error 'error)
Accept/Refuse: (quit)


That's the only hard par though. Interacting with the game once we're sure we have valid input from our player is relatively simple.

(defun repl! (adventure)
  (format t "~%~%~a~%~%" (lookup adventure :description))
  (when (contains? adventure :continue)
    (let ((choice (get-repl-choice adventure)))
      (repl! (funcall (lookup adventure :continue) choice)))))

THE-PRISONERS> (repl! (mk-adventure))

A stranger approaches. "I see you have baubles. Would you like to trade, that we both may enrich ourselves?"


You have come to the end of your long, perilous journey.


This is obviously not the perilous journey being spoken of. At least, not all of it. The simplest way to extend it into one is to wrap scenarios around our existing adventure.

(defun mk-adventure ()
  (let ((def (defector)))
    {:description "A muscled street thug approachs, knife drawn."
     :cooperate "surrender" :defect "run" :prisoner def :scenario theft
     :continue (lambda (choice)
                 (let ((their-choice (play def)))
                   (update! def choice)
                   (funcall theft choice their-choice))
                 (let ((prisoner (polo)))
                    "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
                    :cooperate "accept" :defect "refuse" :prisoner prisoner :scenario trade
                    :continue (lambda (choice)
                                (let ((their-choice (play prisoner)))
                                  (update! prisoner choice)
                                  (funcall trade choice their-choice)

THE-PRISONERS> (repl! (mk-adventure))

A muscled street thug approachs, knife drawn.


A stranger approaches. "I see you have baubles. Would you like to trade, that we both may enrich ourselves?"


You have come to the end of your long, perilous journey.


Of course, since we want it to be much longer and more perilous, we'll want that process automated to at least some degree.

(defun wrap-scenario (adventure scenario)
    (lambda (choice)
      (let* ((them (lookup scenario :prisoner))
             (their-choice (play them)))
        (update! them choice)
        (funcall (lookup scenario :scenario) choice their-choice)

(defun mk-adventure ()
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner (polo) :scenario trade})
    "A muscled street thug approachs, knife drawn. \"Yer money or yer life, fop!\""
    :cooperate "surrender" :defect "run" :prisoner (defector) :scenario theft}))

This isn't enough for the Roguelikelike title, and I don't think I'll get there today, but I do want the ability to make an arbitrarily long adventure. The dumbest way of doing this is to make a list of scenarios, and pick from them when the need arises.

(defun random-scenario ()
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner (polo) :scenario trade}
     "A muscled street thug approachs, knife drawn. \"Yer money or yer life, fop!\""
     :cooperate "surrender" :defect "run" :prisoner (defector) :scenario theft})))

(defun mk-adventure (&key (scenarios 5))
  (let ((adventure ending))
    (loop repeat scenarios
       do (setf adventure (wrap-scenario adventure (random-scenario))))

An adventure of even 5 scenarios will end up being repetitive since we currently only have a grand total of two. But we can do something about that...

(defun random-scenario ()
     "A stranger approaches. \"I see you have baubles. Would you like to trade, that we both may enrich ourselves?\""
     :cooperate "accept" :defect "refuse" :prisoner (polo) :scenario trade}
     "A muscled street thug approachs, knife drawn. \"Yer money or yer life, fop!\""
     :cooperate "surrender" :defect "run" :prisoner (defector) :scenario theft}
     "As you walk through an expansive market square, a gambler motions you over. \"Fancy your chances at evens or odds?"
     :cooperate "Evens!" :defect "Odds!" :prisoner (gambler) :scenario mutual-prediction}
     "A hunter approaches you in a forest clearing. \"Hallo there, young one. Would you help me hunt a deer? I've had enough hares for now, but I promise we'll eat well if we work together!\""
     :cooperate "<Nocks bow>" :defect "Rather go my own way" :prisoner (dantes) :scenario stag-hunt}
     "\"Hey follow me into this bear trap!\""
     :cooperate "Sure; I've grown tired of living" :defect "No. No, I'd rather not."
     :prisoner (robin) :scenario trap}
     "You see a merchant ahead of you, paying little attention to his overfull coin purse. You could cut it and run."
     :cooperate "It's too tempting" :defect "No; I hold strong"
     :prisoner (dantes) :scenario theft}
     "At the end of your travails with your co-conspirator, you get to the treasure first and can pocket some if you want."
     :cooperate "Take it" :defect "No, we split fairly"
     :prisoner (gambler :defect 5) :scenario dilemma})))

This gives me some ideas about how to go about generating scenarios a lot more programmatically, but I'll leave that for later, when I'm in the right frame of mind to do cosmetic improvements.

Wanna play a game?

THE-PRISONERS> (repl! (mk-adventure))

At the end of your travails with your co-conspirator, you get to the treasure first and can pocket some if you want.

Take it/Split fairly:split

You see a merchant ahead of you, paying little attention to his overfull coin purse. You could cut it and run.

It's too tempting/No:it's

"Hey follow me into this bear trap!"

Sure; I've grown tired of living/No. No, I'd rather not.:no

You see a merchant ahead of you, paying little attention to his overfull coin purse. You could cut it and run.

It's too tempting/No:it's

A stranger approaches. "I see you have baubles. Would you like to trade, that we both may enrich ourselves?"


You have come to the end of your long, perilous journey.


This is about as far as I'm going today, and I'm not entirely sure how far I'm going during my next session.

As always, I'll let you know.

20 Sep 2020 4:25am GMT

19 Sep 2020

feedPlanet Lisp

Leo Zovic: The Prisoners Part 1

Ok, so I guess I'm doing this.

In hopes of participating in the Autumn Lisp 2020 Game Jam, I'm going to write a multiplayer game. It's going to deal with players in several ways, implement 1FA, and probably end up being asymmetric and heavily infulenced by some readings that The Cabal have been doing lately.

But don't worry about that for the moment.

Piece by piece

The basics

(in-package #:the-prisoners)
(named-readtables:in-readtable clj:syntax)

I'm using clj. You can find it on my github, and it'll be included as part of the asd file.


Prisoners can do two things. They can cooperate or they can defect.

(defun coop? (res) (eq :cooperate res))
(defun defe? (res) (eq :defect res))

In order to play a game, you take the game function and apply it to the ordered list of prisoners that will be playing.

(defun play! (game &rest players)
  (apply game players))

A two-player, one-time game looks like this:

  1. We take two prisoners
  2. We ask them to either cooperate or defect
  3. We tell each of them what the other did
  4. We score them

To start with, we're going with a payoff matrix that looks like

          | Cooperate | Defect
Cooperate | 3, 3      | 1, 5
   Defect | 5, 1      | 0, 0

We might play with this later, but lets pretend we won't have the time.

(defun one-time (player-a player-b)
  (let ((a (funcall (lookup player-a :strategy)))
        (b (funcall (lookup player-b :strategy))))
    (if-let (update (lookup player-a :update))
      (funcall update b))
    (if-let (update (lookup player-b :update))
      (funcall update a))
    (cond ((and (coop? a) (coop? b))
           (list 3 3))
          ((and (coop? a) (defe? b))
           (list 1 5))
          ((and (defe? a) (coop? b))
           (list 5 1))
           (list 0 0)))))

The two simplest possible prisoners we can have are one who always :cooperates, and one who always :defects. A prisoner needs to be able to take into account what their opponent did last time, and separately, do something.

(defun defector ()
  {:name :defector :strategy (lambda () :defect)})

(defun cooperator ()
  {:name :cooperator :strategy (lambda () :cooperate)})

We can now play. Would you like to play a game?

The Simplest Game

Would you like to play a game?

THE-PRISONERS> (play! #'one-time (defector) (cooperator))
(5 1)
THE-PRISONERS> (play! #'one-time (cooperator) (defector))
(1 5)
THE-PRISONERS> (play! #'one-time (cooperator) (cooperator))
(3 3)
THE-PRISONERS> (play! #'one-time (defector) (defector))
(0 0)

There are other, simple kinds of prisoners. One is the prisoner who tosses a coin and does what it tells them to.

(defun gambler ()
  {:name :gambler :strategy (lambda () (nth (random 2) (list :cooperate :defect)))})

The more general case doesn't necessarily flip a coin, but can weigh either :cooperate or :defect more strongly.

(defun gambler (&key (cooperate 1) (defect 1))
  (let ((total (+ cooperate defect))
        (moves (concatenate
                (loop repeat cooperate collect :cooperate)
                (loop repeat defect collect :defect))))
    {:name (intern (format nil "GAMBLER~a/~a" cooperate defect) :keyword)
           :strategy (lambda () (nth (random total) moves))}))

This way, we can get a true coin-flipper.

THE-PRISONERS> (gambler)

Or someone who mostly cooperates/defects, but sometimes defects/cooperates.

THE-PRISONERS> (gambler :cooperate 5)
THE-PRISONERS> (gambler :defect 5)

How do they play against each of the others? Lets find out.

The Second Simplest Game

(defun matches (elems &key (mirror? t))
  (loop for (a . rest) on elems while rest
      if mirror? collect (cons a a)
      append (loop for b in rest collect (cons a b))))

(defun all-against-all! (game matches)
   (lambda (memo res)
     (merge-by #'+ memo res))
   (loop for (a . b) in matches
      collect (let ((res (play! game a b)))
                {(lookup a :name) (first res) (lookup b :name) (second res)}))))

This lets us see who does better against everyone.

THE-PRISONERS> (all-against-all! #'one-time (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5))))
THE-PRISONERS> (all-against-all! #'one-time (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5))))
THE-PRISONERS> (all-against-all! #'one-time (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5))))
THE-PRISONERS> (all-against-all! #'one-time (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5))))

The defector comes out on top here. And the mostly-defecting gambler doesn't do bad either. Of course, this is what we would expect from the one-time game.

An iterated game is like a series of one-time games, and it keeps a running total of the score.

(defun iterated (&key (iterations 10))
  (lambda (player-a player-b)
    (loop repeat iterations
       for (a b) = (one-time player-a player-b)
       sum a into a-sum sum b into b-sum
       finally (return (list a-sum b-sum)))))

It plays about how you'd expect

THE-PRISONERS> (play! (iterated) (defector) (cooperator))
(50 10)
THE-PRISONERS> (play! (iterated) (cooperator) (cooperator))
(30 30)
THE-PRISONERS> (play! (iterated) (defector) (defector))
(0 0)

And setting the world at its' own throat works the way you'd expect of this process so far.

THE-PRISONERS> (all-against-all! (iterated) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5))))
{:GAMBLER1/5 119 :GAMBLER1/1 117 :GAMBLER5/1 105 :DEFECTOR 135 :COOPERATOR 100}
THE-PRISONERS> (all-against-all! (iterated) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5))))
{:GAMBLER1/5 132 :GAMBLER1/1 109 :GAMBLER5/1 103 :DEFECTOR 120 :COOPERATOR 100}
THE-PRISONERS> (all-against-all! (iterated) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5))))

There are more elaborate strategies we can call upon. I won't implement them all here, but these have been thought of.

Thoughtful Players

Robin alternates between cooperating and defecting.

(defun robin ()
  (let ((prev :cooperate))
    {:name :robin
           :strategy (lambda ()
                       (if (coop? prev)
                           (setf prev :defect)
                           (setf prev :cooperate)))}))

And then, there are the simplest strategies that consider their opponent.

(defun polo ()
  (let ((prev nil))
    {:name :polo
           :update (lambda (opponent-action) (setf prev opponent-action))
           :strategy (lambda () (or prev :cooperate))}))

(defun dantes ()
  (let ((plan :cooperate))
    {:name :dantes
           :update (lambda (action) (when (defe? action) (setf plan :defect)))
           :strategy (lambda () plan)}))

With the addition of these, it's no longer obviously a defectors game.

THE-PRISONERS> (all-against-all! (iterated) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 164 :DANTES 131 :GAMBLER1/1 150 :GAMBLER5/1 169 :DEFECTOR 150 :COOPERATOR 184 :POLO 120 :ROBIN 147}
THE-PRISONERS> (all-against-all! (iterated) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 168 :DANTES 126 :GAMBLER1/1 176 :GAMBLER5/1 159 :DEFECTOR 165 :COOPERATOR 184 :POLO 129 :ROBIN 136}
THE-PRISONERS> (all-against-all! (iterated) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 158 :DANTES 121 :GAMBLER1/1 154 :GAMBLER5/1 156 :DEFECTOR 150 :COOPERATOR 184 :POLO 123 :ROBIN 154}
THE-PRISONERS> (all-against-all! (iterated) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 163 :DANTES 131 :GAMBLER1/1 163 :GAMBLER5/1 161 :DEFECTOR 175 :COOPERATOR 184 :POLO 117 :ROBIN 146}
THE-PRISONERS> (all-against-all! (iterated :iterations 50) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 789 :DANTES 656 :GAMBLER1/1 940 :GAMBLER5/1 964 :DEFECTOR 720 :COOPERATOR 1056 :POLO 585 :ROBIN 752}
THE-PRISONERS> (all-against-all! (iterated :iterations 50) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 845 :DANTES 651 :GAMBLER1/1 892 :GAMBLER5/1 959 :DEFECTOR 775 :COOPERATOR 1054 :POLO 609 :ROBIN 719}
THE-PRISONERS> (all-against-all! (iterated :iterations 50) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 788 :DANTES 651 :GAMBLER1/1 929 :GAMBLER5/1 946 :DEFECTOR 775 :COOPERATOR 1044 :POLO 609 :ROBIN 744}
THE-PRISONERS> (all-against-all! (iterated :iterations 50) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 859 :DANTES 651 :GAMBLER1/1 867 :GAMBLER5/1 952 :DEFECTOR 765 :COOPERATOR 1048 :POLO 609 :ROBIN 729}
THE-PRISONERS> (all-against-all! (iterated :iterations 50) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 833 :DANTES 666 :GAMBLER1/1 920 :GAMBLER5/1 953 :DEFECTOR 775 :COOPERATOR 1046 :POLO 603 :ROBIN 720}
THE-PRISONERS> (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 8325 :DANTES 6436 :GAMBLER1/1 9255 :GAMBLER5/1 9544 :DEFECTOR 7565 :COOPERATOR 10508 :POLO 8976 :ROBIN 7383}
THE-PRISONERS> (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 8365 :DANTES 6531 :GAMBLER1/1 9289 :GAMBLER5/1 9531 :DEFECTOR 7645 :COOPERATOR 10486 :POLO 6018 :ROBIN 7379}
THE-PRISONERS> (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 8407 :DANTES 6546 :GAMBLER1/1 9139 :GAMBLER5/1 9574 :DEFECTOR 7590 :COOPERATOR 10554 :POLO 6117 :ROBIN 7389}
THE-PRISONERS> (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 8063 :DANTES 6371 :GAMBLER1/1 9231 :GAMBLER5/1 9492 :DEFECTOR 7555 :COOPERATOR 10508 :POLO 6084 :ROBIN 7412}
THE-PRISONERS> (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 8068 :DANTES 6456 :GAMBLER1/1 9165 :GAMBLER5/1 9614 :DEFECTOR 7395 :COOPERATOR 10516 :POLO 6003 :ROBIN 7451}
THE-PRISONERS> (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin))))
{:GAMBLER1/5 8241 :DANTES 6356 :GAMBLER1/1 9150 :GAMBLER5/1 9579 :DEFECTOR 7545 :COOPERATOR 10480 :POLO 9021 :ROBIN 7392}

When it's a prisoner against the world, the makeup of the world makes a difference in which prisoner ultimately wins.

(defun winner (results)
  (let ((max nil)
        (score nil))
    (loop for (k . v) in (as-list results)
       do (if (or (not score) (> v score))
              (setf score v
                    max (cons k v))))

Currently, with mirror matches happening, the world is tilted towards cooperators.

THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10554)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10532)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10486)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10536)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10478)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10502)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10540)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10516)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)))))
(:COOPERATOR . 10476)

Without mirror matches, it's still mostly a cooperators' game, but not quite so strongly.

THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))

(:DEFECTOR . 7665)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:ROBIN . 7497)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:COOPERATOR . 7512)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:COOPERATOR . 7580)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:COOPERATOR . 7516)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:COOPERATOR . 7528)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:DEFECTOR . 7615)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:DEFECTOR . 7610)
THE-PRISONERS> (winner (all-against-all! (iterated :iterations 500) (matches (list (cooperator) (defector) (gambler) (gambler :cooperate 5) (gambler :defect 5) (polo) (dantes) (robin)) :mirror? nil)))
(:COOPERATOR . 7550)

This wasn't the end. It was step one.

19 Sep 2020 12:19am GMT

18 Sep 2020

feedPlanet Lisp

Alexander Artemenko: eco

This template engine is interesting because it allows mixing lisp code blocks and HTML in a way simple enough to be used by non-lisp developers and designers.

It's interesting feature is that each template definition includes the arguments list.

Here is how we can define templates for user list from the previous post about cl-emb:

POFTHEDAY> (eco:compile-string
<% deftemplate user (nickname name) () %>
<a href=\"/users/<%= nickname %>\"><%= name %></a>
<% end %>

POFTHEDAY> (eco:compile-string "
<% deftemplate user-list (users) () %>
  <% loop for (nickname name) in users do %>
    <li><%- user nickname name %><% end %></li>
  <% end %>
<% end %>

POFTHEDAY> (eco-template:user-list
            '(("bob" "Bob Hopkins")
              ("alice" "Alice Cooker")))
<a href=\"/users/bob\">Bob Hopkins</a>
<a href=\"/users/alice\">Alice Cooker</a>

Also, there is a way to load templates from the files with .eco extensions. There is an ASDF extension which allows defining these templates as components of your ASDF system.

Documentation does not cover this, but the template components should be defined like this:

(defsystem mysite
  :defsystem-depends-on (eco)
  :components ((:module "src"
                :depends-on "templates"
                :components ((:file "backend-code")
                             (:file "utils")))
               (:module "templates"
                :components ((:eco-template "index-page")
                             (:eco-template "users")))))

Well, let's measure Eco's performance!

POFTHEDAY> (eco:compile-string "
<% deftemplate perform (title items) () %>
<title><%= title %></title>
  <% loop for item in items do %>
    <li><%= item %></li>
  <% end %>
<% end %>

            (loop repeat 1000000
                  do (eco-template:perform "Foo Bar"
                       '("One" "Two" "Three"))))
Evaluation took:
  2.135 seconds of real time
  2.144360 seconds of total run time (2.121050 user, 0.023310 system)
  [ Run times consist of 0.141 seconds GC time, and 2.004 seconds non-GC time. ]
  100.42% CPU
  4,713,480,570 processor cycles
  1,008,017,904 bytes consed

This is slower than half of the tested template engines. It took place between cl-who and print-html. I've expected it will be faster :(

18 Sep 2020 8:52pm GMT

feedPlanet Grep

Koen Vervloesem: Exploring the PinePhone with the multi-distro demo image

The PinePhone, powered by Linux

A few days ago my PinePhone arrived. It came with postmarketOS, which is based on Alpine Linux. By default it has a fairly limited set of apps. And if you look in the application manager, there are not many apps available to install either. With apk, Alpine's package manager, you can install a lot of other software. I installed Firefox this way, it seems to work, but you often need to zoom out to 50% to get the whole website on the screen.

But what I find most impressive is how this low-cost Linux phone has garnered extensive support from all major mobile Linux distributions. Who would have thought a few years ago that 'distro hopping' on a phone would be a thing?

Trying out some Linux distributions on the PinePhone

The easiest way to explore a lot of the supported distributions is the PinePhone multi-distro demo image, which is a 5 GB image with 13 distributions and a neat bootloader, p-boot.

The bootloader p-boot, here with 13 Linux distributions for the PinePhone

So installed the multi-distro demo image on a spare microSD card with USBImager, resized the partition to the microSD card's full size and then I have been trying all 13 distributions for the last few days. These are my notes for the 2020-09-14 version of the image, not in any way systematic or complete: 1

Arch Linux

Boots into a command line. I don't even see a virtual keyboard appearing to log in. This is not what I expect from a phone, but it's very understandable coming from the Arch Linux philosophy :-)


Has a bit more apps installed by default: maps, email, a PDF viewer, ... The maps app uses Google Maps data, but it doesn't work. I can't find general settings. It seems somewhat limited and basic things such as rebooting or shutting down don't work. LuneOS is based on webOS.

Maemo Leste

Boots into landscape mode, and I can't find how to change it to portrait mode. It has Vim and Htop installed by default, but they crash. There's no browser installed by default. It has an app manager, but it doesn't have a lot of apps available to install. The settings are fairly limited. The virtual keyboard is barely usable. Maemo Leste is the successor of Nokia's Maemo and it's currently an ARM64 port of Devuan (Debian without systemd).


Has a good set of default apps. It has both Web and Firefox installed as web browsers. There's a To Do app, Processor usage, Telegram (the Desktop version), Maps (using OpenStreetMap), Geary for email. There's also an Authenticator app, but it doesn't show anything. There's no terminal installed by default. Software shows a lot of apps, and you get access to a lot of GNOME settings, including accounts, for instance to set up your email account (Geary uses the accounts you have set up here). It definitely feels Linux-like, but at the same time feels like a real mobile operating system. It's essentially just Debian for ARM64 with the Phosh user interface developed by Purism for the Librem 5, and it works surprisingly well.

KDE Neon

Looks nice, there are but not many apps installed. I'm more familiar with the GNOME world, so I feel a bit lost here. There's Spacebar (to chat), Koko (music), Okular (document viewer), Konsole. Contrary to all other distributions I tried, there's a screenshot app in the drawer. But how does it work? I don't think I will get used to the Plasma Mobile interface. So while it looks like a perfectly usable system, it's not for me.

postmarketOS with fbkeyboard

Boots into a command line. The virtual keyboard needed to log in is quite usable, but this is not what I expect from a phone.

postmarketOS with GNOME

I hear a high-pitched sound when the PinePhone boots into the graphical environment. The virtual keyboard has awkwardly small buttons. There's a terminal installed. But the apps are not easy to access: if you open the application drawer, the icons suddenly fly off the screen, so the only way to start the apps is by typing an app's name in the search box. There are a lot of GNOME settings. However, the system doesn't seem to be adapted to a mobile screen: windows are shown only partially, even the GNOME initial setup window. All in all, this is not very usable in this state.

postmarketOS with Phosh

Also gives the high-pitched sound, but less long. There are not many apps preinstalled. There are also not many apps to install from Software. There are a lot of GNOME settings. Not all windows are adapted to the mobile screen size.

postmarketOS with Plasma Mobile

I hear a high-pitched sound when the PinePhone boots into the graphical environment. A limited set of preinstalled apps. The system doesn't seem to be very responsive. I couldn't even enter my Wi-Fi settings because the interface hangs when I want to open the settings window. Unusable in this state.

postmarketOS with sxmo

A nice lightweight alternative using suckless programs. Sxmo (Simple X Mobile) is actually a collection of simple programs and scripts to create a fully functional mobile user interface adhering to the Unix philosophy. It's a bit too minimalistic for me, but it works nicely to my surprise, controlling the user interface largely through the PinePhone's volume and power buttons and swipe gestures.


Has a limited set of apps. It seems to be better adapted to the mobile screen size than some of the other distributions. The default web browser is GNOME Web. There's no Firefox to install from the Software app. There seems to be an Iceweasel in the app, but it hasn't been updated since the version equivalent to Firefox 55! The available GNOME settings are rather limited.

Sailfish OS

Looks very nice. It has a very user-friendly user onboarding experience. There's a fairly limited set of installed apps. The user interface is closed source, so this is no option for me.

Ubuntu Touch

Looks very nice, it also a nice user onboarding experience. The default web browser is Morph. It also has a media player, weather app, music, terminal, navigation (uNav, but it didn't work for me, as it keeps "waiting for GPS"). It seems to be a bit slower to boot and less responsive than some of the other distributions. I also experienced some weird behaviour with the window placement.

Powered by Linux

Mobian is a surprisingly useful Debian distribution on the PinePhone.

The PinePhone supports even more Linux distributions: Fedora, Manjaro, Nemo Mobile (an open source build of Sailfish OS), NixOS, openSUSE and AVMultiPhone (basically postmarketOS with MATE). These are not (yet) available from the multi-distro demo image, and I haven't tried them yet.

As you see in my notes above, there are still a lot of issues if you want to use the PinePhone on a daily basis. 2 Most of them will probably be solved eventually: things seem to move very fast in this domain. Also, the fact that these are all just Linux distributions is extremely helpful. If you encounter a problem, you can have a look at dmesg, run systemctl list-units --state=failed and so on and you have the same powerful toolbox at your disposal as on the Linux desktop.

All in all, Mobian impressed me most, with PureOS as my second choice. So after distro hopping for the last few days, I'm going to do a clean install with Mobian and explore it further. The feeling of having freedom and control on my phone in a familiar environment is amazing. Powered by Linux!


I haven't tested phone calls or SMS messages, as I haven't put my SIM card yet in the PinePhone.


For instance, on none of the distributions I tested I was able to get the camera working. I also couldn't take a screenshot on any system, even with scrot.

18 Sep 2020 6:49pm GMT

Lionel Dricot: Le mensonge des réseaux sociaux

La promesse que les réseaux sociaux ont fait aux créateurs de contenus n'a pas été tenue.

Pour les créateurs comme moi, la promesse des réseaux sociaux semble claire. Nous permettre de créer une communauté facilement joignable tout en donnant à nos créations l'opportunité d'atteindre de nouvelles personnes.

En échange, nous, créateurs, faisons de la publicité permanente pour lesdits réseaux sociaux, nous les animons, nous créons du contenu souvent exclusif voire même nous acceptons de payer pour diffuser nos créations. Sans oublier que nous vendons notre temps, notre vie privée ainsi que celle des membres de notre communauté.

Un pacte pour le moins faustien. Mais est-il rempli ?

Avec plus de 2.600 abonnés à ma page Facebook, près de 6.500 sur Twitter, 2.500 sur Mastodon, 1.500 sur Linkedin, 1.100 sur Medium et 1.100 amis Facebook, on pourrait dire que « ma communauté » est de 15.000 comptes. En admettant que certains me suivent sur plusieurs réseaux, 10.000 semble plus réaliste. Un joli chiffre.

Vraiment ?

De ces 10.000 comptes, il faut retirer les bots qui suivent automatiquement ce qui passe à leur portée, les comptes temporaires, les comptes qui ne sont plus utilisés, les comptes commerciaux qui ne vous suivent qu'en espérant un « follow back », les comptes des personnes décédées.

Ajoutons que les personnes qui vous suivent, mais qui suivent également 2000 ou 3000 autres personnes n'ont statistiquement aucune chance de voir vos messages.

Enfin, parmi les véritables comptes actifs qui ne suivent pas beaucoup de personnes, encore faut-il que l'algorithme du réseau social décide d'afficher votre message ou avoir la chance de poster dans les minutes qui précèdent le moment où vos abonnés vont consulter leur flux.

Au final, un message posté sur les réseaux sociaux est « affiché » (même pas lu) une ou deux centaines de fois. Pour une grosse dizaine de visites sur mon billet de blog (d'après les statistiques des réseaux sociaux eux-mêmes).

Les réseaux sociaux me font croire que j'ai une communauté de plus de 10.000 membres. C'est un mensonge. Les réseaux sociaux me font croire que je peux atteindre ma communauté. C'est un second mensonge.

Peut-être faut-il payer ? Pour l'expérience, j'ai décidé de dépenser 50€ en publicités chez Facebook et 50€ en publicités chez Twitter. L'objectif ? Promouvoir mon roman Printeurs. Dans les deux cas, j'ai obtenu, d'après les statistiques des réseaux sociaux eux-mêmes, beaucoup de visites et de clics. En regardant de plus près, je constate que tous les retweets obtenus sont issus de faux comptes Twitter. Que les clics n'ont eu aucun effet. Aucun livre n'a été vendu suite à ces promotions.

Tentant de prévenir à tout prix ma « communauté » que je publiais un roman, que c'était important pour moi et que c'était le moment de me soutenir pour ceux qui en avaient l'envie et les moyens, je me suis retrouvé à spammer sans relâche les réseaux sociaux. Certains, à raison, se sont sentis harcelés. D'autres, qui me suivent sans lire chacun de mes billets de blog, m'ont demandé de quoi je parlais. Ils n'avaient vu aucun des précédents messages, ils ne savaient pas ce qu'était ce « Printeurs » qui apparaissait pour la première fois dans leur flux. Certains ne sont toujours pas au courant aujourd'hui ! (alors que les préventes s'achèvent ce mercredi !)

Pour toucher ma « communauté », les réseaux sociaux font pire que mal. Ils me font croire qu'il y'a une communication là où elle se réduit à quelques interactions entre les mêmes personnes.

Mais, au moins, on pourrait espérer que les réseaux sociaux me permettent de toucher de nouvelles personnes.

Cela arrive, parfois. Cela s'appelle un « buzz ». C'est imprévisible, aléatoire. Cela m'est arrivé plus d'une fois, souvent pour des billets qui, selon moi, ne le méritaient pas. C'est de plus en plus difficile, car tout le monde cherche à faire le buzz. L'objectif n'est plus d'informer ou de partager, juste de faire le buzz. Ce n'est pas mon métier.

Mais ai-je touché l'utilisateur qui a choisi de ne pas avoir de compte de réseau social pour raison philosophique ? Celle qui a supprimé ses comptes pour raisons professionnelles ou suite à un harcèlement. Celui qui a un compte, mais qui n'en a jamais vu l'intérêt, qui ne l'utilise pas vraiment. Celle qui s'y intéresse, mais qui n'a pas acquis le réflexe, qui l'oublie pendant plusieurs mois, qui n'a pas le temps.

Ces personnes existent. Elles sont nombreuses. Peut-être plus nombreuses que la population des addicts aux réseaux sociaux. J'en croise tous les jours. Une longue conversation téléphonique avec un lecteur de mon blog qui n'a jamais eu de compte m'a ouvert les yeux.

Toutes ces personnes ne peuvent être atteintes par les réseaux sociaux. Pire : en optimisant ma communication pour les réseaux sociaux, je les exclus.

Là encore, pour un créateur, les réseaux sociaux sont un mensonge.

J'ai rempli ma part du contrat. J'ai produit du contenu pour les réseaux sociaux. J'ai fait leur promotion partout, jusque sur mon blog et dans mes conférences en demandant à être suivi. J'ai même payé pour être diffusé. J'ai, indirectement, vendu la vie privée de mes lecteurs.

Qu'ai-je obtenu en échange ? Une impression d'importance, une impression d'audience, impression qui se dissipe vite lorsqu'il s'agit de vendre un vrai livre.

Les réseaux sociaux sont un mensonge pour les créateurs, pour les utilisateurs, pour tous ceux qui pensent communiquer. Ils m'avaient déjà rendu injoignables.

C'est la raison pour laquelle je pense me concentrer sur l'écriture de livres et sur mon blog. Si vous voulez me soutenir, achetez mes livres, abonnez-vous à ma mailing-liste ci-dessous (ou par RSS), parlez de mes écrits à vos amis, mais ne me suivez plus sur les réseaux sociaux ! Ils sont un mensonge.

Certains pensent que j'y perdrais grandement en visibilité. Je prends le risque. Je ne supporte pas le mensonge.

Photo by Siora Photography on Unsplash

Je suis @ploum, écrivain électronique. Printeurs, mon dernier roman de science-fiction, est disponible en précommande jusqu'au 24 septembre. Si vous avez apprécié ce texte, n'hésitez pas à me soutenir sur Paypal ou en millibitcoins 34pp7LupBF7rkz797ovgBTbqcLevuze7LF. Vos soutiens réguliers, même symboliques, sont une réelle motivation et reconnaissance. Merci !

Vérifiez votre boite de réception ou votre répertoire d'indésirables pour confirmer votre abonnement.

Ce texte est publié sous la licence CC-By BE.

18 Sep 2020 12:20pm GMT

Dries Buytaert: How governments can help sustain Open Source

Yesterday I wrote about why software funded with tax dollars should be Open Source. Based on the feedback in email and social media, lots of people seem to agree.

Today, I want to highlight how this could be a game changer for Open Source sustainability.

Using Drupal as an example, let's do some quick math. Imagine if:

Even if these numbers are conservative, it would lead to $15 million in annual contributions to Drupal: 1,000 x $300,000 x 0.05 = $15,000,000. That could be 150 full-time contributors each year.

In other words, requiring public code in government could be Open Source's best funding mechanism.

18 Sep 2020 11:34am GMT

27 Apr 2020

feedPlanet Sun

The Hubble Space Telescope celebrates its 30th birthday

For 30 years, the space telescope Hubble has provided the most impressive images from the vast wide space. The telescope was developed by the US space agency NASA and its European counterpart ESA. Hubble started its journey into space on 24 April 1990. With the help of the space shuttle "Discovery" it was lifted into ... Read more

27 Apr 2020 3:47pm GMT

26 Apr 2019

feedPlanet Sun

First Image of a Black Hole – Event Horizon

The Event Horizon Telescope (EHT) - a planet-scale array of 8 ground-based radio telescopes and part of an international collaboration - captured the first image of a black hole. On April 10th 2019, EHT researchers disclosed the first direct visual evidence of a supermassive black hole in the heart of the Galaxy Messier 87.

26 Apr 2019 2:32am GMT

04 Nov 2018

feedPlanet Sun

5 Budget-Friendly Telescopes You Can Choose For Viewing Planets

Socrates couldn't have been more right when he said: "I know one thing, that I know nothing." Even with all of the advancements we, as a species, have made in this world, it's still nothing compared to countless of wonders waiting to be discovered in the vast universe. If you've recently developed an interest in ... Read more

04 Nov 2018 1:27pm GMT

10 Nov 2011


OSDir.com - Java: Oracle Introduces New Java Specification Requests to Evolve Java Community Process

From the Yet Another dept.:

To further its commitment to the Java Community Process (JCP), Oracle has submitted the first of two Java Specification Requests (JSRs) to update and revitalize the JCP.

10 Nov 2011 6:01am GMT

OSDir.com - Java: No copied Java code or weapons of mass destruction found in Android

From the Fact Checking dept.:

ZDNET: Sometimes the sheer wrongness of what is posted on the web leaves us speechless. Especially when it's picked up and repeated as gospel by otherwise reputable sites like Engadget. "Google copied Oracle's Java code, pasted in a new license, and shipped it," they reported this morning.

Sorry, but that just isn't true.

10 Nov 2011 6:01am GMT

OSDir.com - Java: Java SE 7 Released

From the Grande dept.:

Oracle today announced the availability of Java Platform, Standard Edition 7 (Java SE 7), the first release of the Java platform under Oracle stewardship.

10 Nov 2011 6:01am GMT

08 Nov 2011

feedfosdem - Google Blog Search

papupapu39 (papupapu39)'s status on Tuesday, 08-Nov-11 00:28 ...

papupapu39 · http://identi.ca/url/56409795 #fosdem #freeknowledge #usamabinladen · about a day ago from web. Help · About · FAQ · TOS · Privacy · Source · Version · Contact. Identi.ca is a microblogging service brought to you by Status.net. ...

08 Nov 2011 12:28am GMT

05 Nov 2011

feedfosdem - Google Blog Search

Write and Submit your first Linux kernel Patch | HowLinux.Tk ...

FOSDEM (Free and Open Source Development European Meeting) is a European event centered around Free and Open Source software development. It is aimed at developers and all interested in the Free and Open Source news in the world. ...

05 Nov 2011 1:19am GMT

03 Nov 2011

feedfosdem - Google Blog Search

Silicon Valley Linux Users Group – Kernel Walkthrough | Digital Tux

FOSDEM (Free and Open Source Development European Meeting) is a European event centered around Free and Open Source software development. It is aimed at developers and all interested in the Free and Open Source news in the ...

03 Nov 2011 3:45pm GMT

28 Oct 2011

feedPlanet Ruby

O'Reilly Ruby: MacRuby: The Definitive Guide

Ruby and Cocoa on OS X, the iPhone, and the Device That Shall Not Be Named

28 Oct 2011 8:00pm GMT

14 Oct 2011

feedPlanet Ruby

Charles Oliver Nutter: Why Clojure Doesn't Need Invokedynamic (Unless You Want It to be More Awesome)

This was originally posted as a comment on @fogus's blog post "Why Clojure doesn't need invokedynamic, but it might be nice". I figured it's worth a top-level post here.

Ok, there's some good points here and a few misguided/misinformed positions. I'll try to cover everything.

First, I need to point out a key detail of invokedynamic that may have escaped notice: any case where you must bounce through a generic piece of code to do dispatch -- regardless of how fast that bounce may be -- prevents a whole slew of optimizations from happening. This might affect Java dispatch, if there's any argument-twiddling logic shared between call sites. It would definitely affect multimethods, which are using a hand-implemented PIC. Any case where there's intervening code between the call site and the target would benefit from invokedynamic, since invokedynamic could be used to plumb that logic and let it inline straight through. This is, indeed, the primary benefit of using invokedynamic: arbitrarily complex dispatch logic folds away allowing the dispatch to optimize as if it were direct.

Your point about inference in Java dispatch is a fair one...if Clojure is able to infer all cases, then there's no need to use invokedynamic at all. But unless Clojure is able to infer all cases, then you've got this little performance time bomb just waiting to happen. Tweak some code path and obscure the inference, and kablam, you're back on a slow reflective impl. Invokedynamic would provide a measure of consistency; the only unforeseen perf impact would be when the dispatch turns out to *actually* be polymorphic, in which case even a direct call wouldn't do much better.

For multimethods, the benefit should be clear: the MM selection logic would be mostly implemented using method handles and "leaf" logic, allowing hotspot to inline it everywhere it is used. That means for small-morphic MM call sites, all targets could potentially inline too. That's impossible without invokedynamic unless you generate every MM path immediately around the eventual call.

Now, on to defs and Var lookup. Depending on the cost of Var lookup, using a SwitchPoint-based invalidation plus invokedynamic could be a big win. In Java 7u2, SwitchPoint-based invalidation is essentially free until invalidated, and as you point out that's a rare case. There would essentially be *no* cost in indirecting through a var until that var changes...and then it would settle back into no cost until it changes again. Frequently-changing vars could gracefully degrade to a PIC.

It's also dangerous to understate the impact code size has on JVM optimization. The usual recommendation on the JVM is to move code into many small methods, possibly using call-through logic as in multimethods to reuse the same logic in many places. As I've mentioned, that defeats many optimizations, so the next approach is often to hand-inline logic everywhere it's used, to let the JVM have a more optimizable view of the system. But now we're stepping on our own feet...by adding more bytecode, we're almost certainly impacting the JVM's optimization and inlining budgets.

OpenJDK (and probably the other VMs too) has various limits on how far it will go to optimize code. A large number of these limits are based on the bytecoded size of the target methods. Methods that get too big won't inline, and sometimes won't compile. Methods that inline a lot of code might not get inlined into other methods. Methods that inline one path and eat up too much budget might push out more important calls later on. The only way around this is to reduce bytecode size, which is where invokedynamic comes in.

As of OpenJDK 7u2, MethodHandle logic is not included when calculating inlining budgets. In other words, if you push all the Java dispatch logic or multimethod dispatch logic or var lookup into mostly MethodHandles, you're getting that logic *for free*. That has had a tremendous impact on JRuby performance; I had previous versions of our compiler that did indeed infer static target methods from the interpreter, but they were often *slower* than call site caching solely because the code was considerably larger. With invokedynamic, a call is a call is a call, and the intervening plumbing is not counted against you.

Now, what about negative impacts to Clojure itself...

#0 is a red herring. JRuby supports Java 5, 6, and 7 with only a few hundred lines of changes in the compiler. Basically, the compiler has abstract interfaces for doing things like constant lookup, literal loading, and dispatch that we simply reimplement to use invokedynamic (extending the old non-indy logic for non-indified paths). In order to compile our uses of invokedynamic, we use Rémi Forax's JSR-292 backport, which includes a "mock" jar with all the invokedynamic APIs stubbed out. In our release, we just leave that library out, reflectively load the invokedynamic-based compiler impls, and we're off to the races.

#1 would be fair if the Oracle Java 7u2 early-access drops did not already include the optimizations that gave JRuby those awesome numbers. The biggest of those optimizations was making SwitchPoint free, but also important are the inlining discounting and MutableCallSite improvements. The perf you see for JRuby there can apply to any indirected behavior in Clojure, with the same perf benefits as of 7u2.

For #2, to address the apparent vagueness in my blog post...the big perf gain was largely from using SwitchPoint to invalidate constants rather than pinging a global serial number. Again, indirection folds away if you can shove it into MethodHandles. And it's pretty easy to do it.

#3 is just plain FUD. Oracle has committed to making invokedynamic work well for Java too. The current thinking is that "lambda", the support for closures in Java 7, will use invokedynamic under the covers to implement "function-like" constructs. Oracle has also committed to Nashorn, a fully invokedynamic-based JavaScript implementation, which has many of the same challenges as languages like Ruby or Python. I talked with Adam Messinger at Oracle, who explained to me that Oracle chose JavaScript in part because it's so far away from Java...as I put it (and he agreed) it's going to "keep Oracle honest" about optimizing for non-Java languages. Invokedynamic is driving the future of the JVM, and Oracle knows it all too well.

As for #4...well, all good things take a little effort :) I think the effort required is far lower than you suspect, though.

14 Oct 2011 2:40pm GMT

07 Oct 2011

feedPlanet Ruby

Ruby on Rails: Rails 3.1.1 has been released!

Hi everyone,

Rails 3.1.1 has been released. This release requires at least sass-rails 3.1.4










You can find an exhaustive list of changes on github. Along with the closed issues marked for v3.1.1.

Thanks to everyone!

07 Oct 2011 5:26pm GMT

26 Jul 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Update your RSS link

If you see this message in your RSS reader, please correct your RSS link to the following URL: http://fosdem.org/rss.xml.

26 Jul 2008 5:55am GMT

25 Jul 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Archive of FOSDEM 2008

These pages have been archived.
For information about the latest FOSDEM edition please check this url: http://fosdem.org

25 Jul 2008 4:43pm GMT

09 Mar 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Slides and videos online

Two weeks after FOSDEM and we are proud to publish most of the slides and videos from this year's edition.

All of the material from the Lightning Talks has been put online. We are still missing some slides and videos from the Main Tracks but we are working hard on getting those completed too.

We would like to thank our mirrors: HEAnet (IE) and Unixheads (US) for hosting our videos, and NamurLUG for quick recording and encoding.

The videos from the Janson room were live-streamed during the event and are also online on the Linux Magazin site.

We are having some synchronisation issues with Belnet (BE) at the moment. We're working to sort these out.

09 Mar 2008 3:12pm GMT