23 Nov 2017

feedPlanet Lisp

Lispjobs: Lisp Engineer, MIND.AI, Seoul, Korea & L.A, USA

MIND.AI (http://www.mind.ai) is working on a brand new paradigm in artificial intelligence. We are looking to innovate REAL AI. Not just natural language processing but Natural Language Reasoning. We need versatile talents who are interested in changing the world. This is not deep learning but something totally new. And we need people with open and creative minds.

Artificial intelligence in a brand new paradigm. NOT deep learning/neural networks, but knowledge of same a plus. Symbolic/logical AI experience will be practiced. Developing core logic based on a new theory of information. Duties will include optimization of existing code and developing new features of the AI. We require significant experience in development activities on large projects and advanced software architecture and development in a large Common Lisp codebase. To build new components and extend existing tooling to meet project needs, and implement high-quality library components and products.

Specifically, the Lisp engineer will:

BASIC QUALIFICATIONS

PREFERRED QUALIFICATIONS

Please send your CV to reeyan@mind.ai


23 Nov 2017 2:50am GMT

15 Nov 2017

feedPlanet Lisp

Nicolas Hafner: Harmony - Confession 77

header
This is a blog entry about Shirakumo's sound system Harmony. While Harmony was released some months back, I spent the last few weeks rewriting large parts of it from the ground up, and I think it's worth it to write a short article describing how it's built and what you can do with it. So, if you're interested in doing sound processing and playback in Lisp, this is for you.

The need for Harmony arose out of me not finding any suitably powerful sound solution in Lisp. I tried doing a pure Lisp solution at first, but was not able to figure out how to make things go fast without sacrificing design. So, in the interest of performance, I first set out to write a C library that does the essential sound computations for me. This library is called libmixed.

I wanted to keep libmixed as simple and straight-forward as possible. As such, it does not do any sound file reading or writing, synthesising, or sound output to devices. Instead, it only concerns itself with the processing of sound samples. Another important factor for its design was that it should be easy to extend it with further capabilities, and to allow combining processing steps. This led to the segment pipeline model.

In libmixed, you assemble a set of segments - audio producers, consumers, or transforms - that perform the computations you need. You connect them together through buffers so that they can transparently exchange audio data. This produces a directed, acyclic graph where each vertex is a segment, and each edge is a buffer. This graph can then be serialised into a simple sequence that dictates the order in which the segments should be run.

Unfortunately, audio data comes in a variety of formats. Samples are frequently encoded as signed or unsigned integers of 8, 16, 24, or 32 bits, or in floats or doubles. The data might have multiple audio channels, and the samples can be either interleaved (LRLRLR..) or sequential (LLL..RRR..). The sample rate might be different as well. All of this can make it quite difficult to deal with the data between different audio components. Libmixed's solution to this problem is to force all the buffers to use the same sample rate, to encode samples in floats, and to only represent a single channel. Almost all segments present in a pipeline thus don't have to worry about any of these discrepancies anymore, reducing complexity tremendously. In order to allow interacting with foreign components easily, it does also include an unpacker and a packer segment.

The packer takes a set of buffers and information about the sample representation, and packs the buffer's data into a single C array, ensuring proper sample format, layout, and sample rate. The unpacker does the opposite. Thus, if for example you have a library that decodes an audio file, you most likely need to add an unpacker segment to the pipeline that decodes the audio data from the library into the proper internal format.

So, for a very simple example of taking two audio files, mixing them together, applying a reverb effect, and then playing them back, the pipeline would need to look something like this:

simple pipeline

We can get away with assigning the same two buffers for both of the unpackers here by using a special property of the basic-mixer segment. Instead of manually processing the two unpackers in our pipeline, we can set the segment property on the basic-mixer's inputs, which tells the basic-mixer to cause the processing on the segment on its own. This way, the mixer can process the segment that produces the input as it mixes it together, reducing the need to allocate individual buffers for each input to the mixer. This is one of the design decisions that still bother me a bit, but I found it necessary after discovering that I would otherwise need to allocate a huge amount of buffers if I wanted to allow playback of a lot of sources simultaneously.

As it currently stands, libmixed includes segments to mix audio by either just adding samples, or through 3D spatial positioning of the source. It also includes segments to change volume and pan, to fade in and out, to generate simple sawtooth, square, triangle, or sine waves, and to include LADSPA plugins in the pipeline. I'd like to add a bunch more effects segments to it to make it more useful for real-time sound processing, but I haven't felt the motivation to get into that yet. If you're interested in sound processing and would be willing to do this, let me know!

Basically the idea of libmixed boils down to this: there's segments that have properties, inputs, and outputs. You can write and read properties, and connect buffers to the inputs and outputs. You can then tell the segment to process a number of samples, and it will read its input buffers, and write to its output buffers. This all works over a struct that contains a bunch of function pointers to perform these actions. It is thus very easy to add further segments to libmixed, even as an outside library: simple produce a struct that holds the appropriate function pointers to the functions that do what you want. This is also how cl-mixed allows you to write segments from Lisp out.

Ascending from the C world to the C+L world then leads us to cl-mixed, which is the bindings and wrapper library for libmixed. It takes care of all the hairy low-level stuff of interacting with the C library, tracking and allocating foreign memory, and so forth. As mentioned, it also gives you a simple interface to write your own segments from Lisp. This can be really nice in order to prototype an effect.

While libmixed is a neat framework to base your sound processing around, it doesn't exactly make most of the common tasks very convenient. Usually you have some audio files that you would like to play back, and maybe apply some effects to them. This is where Harmony comes in.

Harmony takes libmixed's generic view of segments and extends it to include sources, drains, and mixers. Sources are segments with no inputs, drains are segments without outputs, and mixers take a run-time variable number of inputs. It also greatly simplifies the pipeline construction by handling the buffer allocation for you. It does this with the help of a graph library called Flow. More on that later. Harmony also gives you a sound server object that handles the mixing in the background, allowing you to focus on just adding, removing, and changing sources in your program. Finally, Harmony includes a number of pre-made sources and drains that either connect to other libraries, or present native implementations. Currently, it supports playing back MP3, WAV, FLAC, and raw buffers, and supports outputting to out123, OpenAL, WASAPI, CoreAudio, ALSA, PulseAudio, and to raw buffers.

The easiest way to get started is to use the harmony-simple system, which assembles a default pipeline for you, allowing you to just directly play some stuff.

 (ql:quickload :harmony-simple)
 (harmony-simple:initialize)
 (harmony-simple:play #p"my-cool-music.mp3" :music)
 (harmony-simple:play #p"kablammo.wav" :sfx)

Assembling your own pipeline isn't very difficult, though. It comes down to just telling Harmony how to connect the inputs and outputs between segments. An optimal buffer layout is automatically computed based on the segment's properties and the graph that you describe through the connections. To do this, Harmony uses the Flow library to describe segments in a graph. Unlike most graph libraries, Flow assigns distinct input and output ports to each vertex. These ports have properties like their arity, direction, and so on. For instance, the basic-mixer from the previously illustrated pipeline would be a vertex with an input port of arbitrary arity, and two output ports of single arity. Flow then employs a simple algorithm to assign colours to the edges in such a way that no input is connected to the same two colours, and no input and output on a vertex have the same colour unless the vertex is marked as in-place. This kind of allocation computation has cropped up in a couple of places, so I've been able to use Flow for it in other projects as well. I don't think it's important to know how the algorithm works, but in case you're interested, the source is commented and pretty short.

How to write new sources and segments, or how to assemble your own pipeline is already illustrated pretty succinctly in the documentation, so I suggest you check it out if you're interested in working on that kind of thing. Harmony is primarily geared towards use in games, where simple pipelines and immediate playback of a variety of audio sources is necessary. However, I can also see it being used in some kind of digital audio workstation, where a graphical user interface could allow you to put together segments and configure them, mapping to a libmixed pipeline underneath.

I feel like I've been rambling about tangents for a bit here, but I suppose the reason is that Harmony doesn't really do all that much. At best it just smooths over the last remaining corners that come from the libmixed C heritage and adds some useful segments to the library. All the concepts used and all the sound technology behind it lies more in libmixed's hands though, and I think I've already explained that all earlier.

So, to close off: if you're thinking of doing some kind of digital audio processing in Lisp, keep Harmony and cl-mixed in mind. I'm also more than open to feedback and suggestions, so if you have any ideas or things you'd like to add to the projects, head on over to GitHub's issues, or talk to us on Freenode/#shirakumo.

15 Nov 2017 3:56pm GMT

06 Nov 2017

feedPlanet Lisp

Quicklisp news: October 2017 download stats

Here are the top 100 projects from Quicklisp for October, by "raw" download count.

16626  alexandria
15213 closer-mop
13436 anaphora
13420 split-sequence
12954 babel
12879 trivial-features
12719 iterate
12464 cl-ppcre
11798 bordeaux-threads
11724 let-plus
11693 trivial-gray-streams
11389 cffi
11281 trivial-garbage
10622 puri
9884 nibbles
9732 more-conditions
9611 flexi-streams
9026 usocket
8602 cl+ssl
8544 utilities.print-items
8153 cl-base64
8130 chunga
7889 chipz
7791 drakma
7691 esrap
7635 trivial-backtrace
6308 ironclad
5910 cl-yacc
5691 cl-fad
5302 parse-number
4996 named-readtables
4984 fiveam
4959 asdf-flv
4886 log4cl
4756 bt-semaphore
4736 local-time
4701 lparallel
4647 closure-common
4638 cxml
4594 architecture.hooks
4552 lift
3784 plexippus-xpath
3586 cl-json
3569 trivial-utf-8
3322 optima
3157 parser.common-rules
3144 cl-clon
2837 uuid
2819 cxml-stp
2705 xml.location
2700 metabang-bind
2624 cl-dot
2475 utilities.print-tree
2473 slime
2458 cl-unicode
2456 cl-interpol
2273 md5
2267 cl-store
2232 fare-utils
2204 fare-quasiquote
2108 inferior-shell
2105 fare-mop
1769 cl-utilities
1706 quri
1671 ieee-floats
1625 static-vectors
1605 fast-io
1547 trivial-types
1545 cl-annot
1536 cl-syntax
1437 utilities.binary-dump
1431 trivial-indent
1364 trivial-mimes
1335 asdf-system-connections
1334 array-utils
1329 symbol-munger
1320 cl-containers
1318 metatilities-base
1318 plump
1302 cl-slice
1296 hunchentoot
1280 access
1267 arnesi
1266 collectors
1258 gettext
1236 djula
1226 cl-parser-combinators
1221 cl-locale
1187 postmodern
1164 rfc2388
1159 yason
1121 simple-date-time
1050 command-line-arguments
956 cl-sqlite
951 cl-log
947 osicat
943 salza2
913 py-configparser
903 cl-markdown
903 asdf-finalizers

06 Nov 2017 1:05pm GMT

27 Oct 2017

feedPlanet Lisp

Didier Verna: Standard IO syntax and the Robustness Principle

Here is a flagrant illustration of the robustness principle, or rather, of a failure to honor it.

I was investigating a bug in Declt where some floating point numbers were printed with exponent markers (e.g. 0.5f0 instead of just 0.5) in the Texinfo file, which broke the parsing of the file by Perl.

Eventually, I found out a double infringement of the robustness principle. First of all, Declt failed to comply with part 1 of the robustness principle: be lenient with the others. The Texinfo file generation routine should have been wrapped into a call to WITH-STANDARD-IO-SYNTAX and it wasn't. Always do that to be on the safe side. Lesson learnt.

This failure on my part, however, had the interesting consequence of exhibiting what I consider a serious infringement of part 2 of the robustness principle: be strict with yourself. It would have remained unnocited otherwise. The culprit here is not Declt. This time, it's the common-lisp-stat library. Problem: the simple fact of loading this library globally changes the value of *READ-DEFAULT-FLOAT-FORMAT* from SINGLE-FLOAT (the default) to DOUBLE-FLOAT. This is bad, and it can break your code in all sorts of nasty ways.

Explanation

*READ-DEFAULT-FLOAT-FORMAT* tells the reader how to read floats when no exponent marker is provided. By default, 0.5 will be read as a SINGLE-FLOAT. But this variable also influences the printer (out of a concern for printing readably I guess): when printing a float of a different format than the current default, then the appropriate exponent marker is also printed. So here is precisely what happened. Declt had been compiled with some floats (e.g. 0.5) read as SINGLE-FLOATs. Later on, those floats were supposed to be printed aesthetically as such. But right after loading common-lisp-stat, the default format changed to DOUBLE-FLOAT and all of a sudden 0.5 started to be printed as 0.5f0.

Consequences

This is bad enough already, but consider that messing with the standard IO syntax globally like this can break your code in all other sorts of even nastier ways. Imagine for instance that common-lisp-stat had been loaded before Declt, and Declt needed to be recompiled. All of a sudden, Declt would be using double floats and the bug would be gone. That is, until the next start of the REPL, after which all floats would be printed like 0.5d0!

So granted, my code wasn't robust enough. But please, don't mess with standard IO syntax globally!

27 Oct 2017 12:00am GMT

25 Oct 2017

feedPlanet Lisp

Zach Beane: UIOP, sly, and qlt

uiop

The problem I had with UIOP is due to using stock SBCL ASDF (3.1.5) with UIOP 3.3.0. UIOP changed a public function's behavior in a way that affected older ASDFs (and seemingly only older ASDFs). This is considered a bug and will be fixed in a future UIOP release.

sly

September of Sly has turned into Season of Sly. I haven't been hacking as much with Sly as I wanted in September, so I'm going to keep going with it for October and beyond, and write up a summary Very Soon. My current hangup is C-c M-q, which is slime-reindent-defun in slime, but does nothing in Sly, and there's no sly-reindent-defun to try to bind instead. I try to use C-c M-q a thousand times a day.

qlt

You might know that Quicklisp dists are constructed after building every project that Quicklisp tracks, and projects that don't build (due to compile-time problems) aren't included. This is better than nothing, but it does nothing to catch runtime problems.

Last week Quicklisp got hit with a runtime problem that broke a lot of stuff, so it prompted me to create qlt.

qlt is for collecting small files of Common Lisp code to run before a Quicklisp dist is created. If any of the files signal an error, Quicklisp generates a report with the console output and the dist is no-go until the problem is tracked down.

The project is sparse right now, but it does include a test file that catches the runtime problem from last week. I hope to include many more things to test as time goes on. If there is something you want to check, patches welcome!

25 Oct 2017 12:35pm GMT

23 Oct 2017

feedPlanet Lisp

Quicklisp news: October 2017 Quicklisp dist update now available

New projects:

Updated projects: 3bgl-shader, 3bmd, architecture.service-provider, asd-generator, asdf-viz, aws-sign4, cambl, caveman, cepl, chirp, cl+ssl, cl-ana, cl-ascii-art, cl-autowrap, cl-cache-tables, cl-cognito, cl-conllu, cl-enumeration, cl-fluent-logger, cl-forest, cl-forms, cl-glfw3, cl-hamcrest, cl-html5-parser, cl-i18n, cl-ledger, cl-moss, cl-mysql, cl-ntp-client, cl-oclapi, cl-one-time-passwords, cl-opengl, cl-pcg, cl-project, cl-pslib, cl-quickcheck, cl-rabbit, cl-sdl2, cl-sdl2-ttf, cl-smtp, cl-str, cl-yesql, classimp, clods-export, closer-mop, clsql-helper, clss, clunit, clx, codata-recommended-values, configuration.options, croatoan, declt, deeds, dendrite, deploy, dexador, dirt, doubly-linked-list, easy-routes, eazy-project, esrap, fare-scripts, fast-io, fiasco, flac-parser, fn, fs-utils, fset, fxml, gamebox-dgen, gamebox-ecs, gamebox-frame-manager, gamebox-grids, gamebox-math, gamebox-sprite-packer, genie, glop, hu.dwim.def, hu.dwim.partial-eval, hu.dwim.presentation, hu.dwim.quasi-quote, hu.dwim.reiterate, hu.dwim.util, hu.dwim.web-server, inquisitor, ironclad, json-streams, jsonrpc, kenzo, legit, let-plus, lichat-serverlib, lisp-namespace, lquery, maiden, mcclim, media-types, mito, mito-auth, modularize-hooks, new-op, nineveh, oclcl, opticl, opticl-core, org-davep-dict, osicat, overlord, parse-float, parser.ini, pathname-utils, plump, pngload, postmodern, psychiq, pzmq, qlot, qmynd, qt-libs, qtools, readable, restas, retrospectiff, roan, rtg-math, scalpl, sdl2kit, secret-values, serapeum, shorty, simple-inferiors, simple-logger, simple-rgb, snooze, spinneret, staple, static-vectors, stl, stumpwm, temporal-functions, the-cost-of-nothing, trivial-file-size, trivial-irc, type-r, unix-opts, varjo, websocket-driver.

Removed projects: cl-proj, magicffi, poiu.

Neither cl-proj nor magicffi build for me any more due to foreign library changes. POIU was removed by request of the author.

To get this update, use: (ql:update-dist "quicklisp")

Enjoy!

23 Oct 2017 7:20pm GMT

22 Oct 2017

feedPlanet Lisp

McCLIM: Progress report #10

Dear Community,

We have many important improvements since the last iteration and even more work is pending. I want to apologise for this late progress report - it has been almost three months since the last update. I'll try to improve in this regard.

Some highlights for this iteration:

We want to thank everybody who has contributed to the project (by improving the code base, discussions, issue reporting, providing advice and suggestions, monetary contributions etc). We are especially grateful to the following people: Nisar Ahmad, Alastair Bridgewater, John Carroll, Cyrus Harmon, Philipp Marek, Elias Mårtenson, Piotr Mieszkowski, Jan Moringen, Nick Patrick, Alessandro Serra and last but not least Robert Strandh.

Bounties:

All McCLIM bounties (both active and already solved) may be found here. Default bounty expiration date is 6 months after publishing it (a bounty may be reissued after that time period).

Bounties solved this iteration:

Active bounties ($1800):

Our current financial status is $1089 for bounties and $264 recurring monthly contributions from the supporters (thank you!).

I have been asked a question about who decides which issues have bounties on them and how the reward amounts are decided. If anyone has been wondering about the same here goes the answer: issues and prices are based on my subjective opinion indicated by problems users encounter and what I consider being worth putting bounty on it. Note though, that I'm open to suggestions (see the next paragraph). I hope that despite some potential doubts the community is generally satisfied with the progress and decisions we make. If there is some lack of transparency, please let me know what you want to know and I'll do my best to help.

Suggestions as to which other issues should have a bounty on them are appreciated and welcome. Please note that Bountysource has a functionality "Suggest an Issue" which may be found on the bounties page. If you would like to work on an issue that is not covered by the existing bounties, feel free to suggest a new bounty.

If you have any questions, doubts or suggestions - please contact me either by email (daniel@turtleware.eu) or on IRC (my nick is jackdaniel). McCLIM developers and users hang out on #clim IRC channel on Freenode.

Sincerely yours,
Daniel Kochmański

22 Oct 2017 1:00am GMT

18 Oct 2017

feedPlanet Lisp

Zach Beane: UIOP 3.3.0 problems

Ok, here's something that is causing problems when I build this month's Quicklisp dist.

UIOP 3.3.0 was recently released, and it's causing some stuff to apparently compile over and over again. Here's a real simple thing to try:

$ cd ~/quicklisp/local-projects/
$ curl -O https://common-lisp.net/project/asdf/archives/uiop.tar.gz
$ tar xzvf uiop.tar.gz

Then:

CL-USER> (ql:quickload "circular-streams")

On my setup, I see cffi and babel stuff compiled twice:

To load "circular-streams":
  Load 1 ASDF system:
    circular-streams
; Loading "circular-streams"
[package uiop/package]............................
..................................................
..................................................
[package alexandria.0.dev]........................
..................................................
[package babel-encodings].........................
[package babel]...................................
..................................................
[package cffi-sys]................................
[package cffi]....................................
..................................................
[package cffi-features]...........................
[package alexandria.0.dev]........................
..................................................
[package impl-specific-gray]......................
[package trivial-gray-streams]....................
[package uiop/package]............................
..................................................
..................................................
[package babel-encodings].........................
[package babel]...................................
..................................................
[package cffi-sys]................................
[package cffi]....................................
..................................................
[package cffi-features]...........................
[package static-vectors]..........................
[package fast-io].................................
[package circular-streams]..

If I remove uiop-3.3.0/ from local-projects, the output looks like this:

To load "circular-streams":
  Load 1 ASDF system:
    circular-streams
; Loading "circular-streams"
[package alexandria.0.dev]........................
[package impl-specific-gray]......................
[package trivial-gray-streams]....................
[package uiop/package]............................
..................................................
..................................................
[package babel-encodings].........................
[package babel]...................................
..................................................
[package cffi-sys]................................
[package cffi]....................................
..................................................
[package cffi-features]...........................
[package static-vectors]..........................
[package fast-io].................................
[package circular-streams]..

Any ideas?

update Commit 4ed76c32050753c8a4450c342a1592881e11d63d seems to reference this behavior, with the "fast-io" system given as an example. And indeed, when I try this with fast-io, I see similar recompilation.

18 Oct 2017 12:59pm GMT

07 Oct 2017

feedPlanet Lisp

Nicolas Hafner: Project Listing - Confession 76

header
This is a listing of projects that I've started, some of which I've completed. The intent is to spread awareness about the work I've done, as I speculate that a lot of people don't know about most of it, even though it might prove useful to their own projects. So, hopefully this article will help a bit in that regard.

I won't go into much detail in the descriptions, as that would take too much of both your and my own time. You can however click on the title of each project to get to its "homepage" if you want to find out more. Naturally, you're also free to contact me if you're interested.

Major Projects

Major projects are ones that have no completion in sight. There are always improvements and additions that could be made to improve the project. Generally they serve as a launch pad for other, minor projects that are completable.

Lichat

Lichat is an attempt at a simple, light-weight chat protocol. There's currently full implementations of the protocol available that allow you to host a TCP or WebSockets server, and to write clients in JavaScript and CL. A Java library, with the intent of writing an Android client is planned.

Maiden

Maiden is an event coordination framework and bot construction toolkit. It includes a plethora of pre-made modules to provide common chat bot functionality, as well as a couple of different backends for various chat protocols.

Parasol

Parasol is a native painting application for graphics tablet users. It has been dormant for some years now as I've still not figured out a good way to architecture everything.

Portacle

Portacle is the portable development environment for Common Lisp. It gives you an easy-to-deploy, contained development studio that you can use to program. It is especially suited and geared towards beginners that have not used Lisp before, but it's also convenient for quick setups on unconfigured machines.

Radiance

Radiance is a web application environment, allowing you to easily deploy and run different web applications in the same lisp instance, increasing sharing of common resources. This article is actually hosted on an application running in Radiance. Imagine that!

Trial

Trial is an OpenGL game engine with a heavy focus on modularity. It is supposed to provide a large toolkit of useful bits and pieces from which you can create a game. I use this engine together with some of my co-conspirators to write games for the Ludum Dare game jam. Hopefully the engine will at some point also give birth to bigger games.

Minor Projects

These projects all either spawned out of the requirements of the major projects, or happened simply for fun. Most of them are already complete and thus ready for use.

3d-matrices

A library implementing common matrix calculations, with an emphasis on 2x2,3x3, and 4x4 matrices as commonly used in graphics. It provides some numerical functions as well, but those are not the focus. The library is heavily optimised, so it is not made of pretty code.

3d-vectors

This is the counter-piece to 3d-matrices, providing vector operations optimised for 2, 3, and 4-component vectors. Also just like 3d-matrices, the library is heavily optimised and thus not pretty on the eyes.

CLSS

CSS-Like Simple Selectors implements a DOM search engine using the CSS selectors as the query format. It is reasonably optimised, but only usable with the DOM provided the Plump system.

LASS

Lisp Augmented Style Sheets is a compiler for a custom CSS syntax. It allows you to write CSS files in a much more convenient and homely-feeling syntax. I've been using this to write pretty much all of my CSS for the past couple of years.

array-utils

A small library to provide vector manipulation functions that are sorely missing from the standard. Allows to push to any place in the array while maintaining the proper shifting logic.

autobuild

A Continuous Integration system with a focus on running directly on your machine, rather than in a container or otherwise segregated environment. This is currently being rewritten from scratch.

chatlog

A Radiance application for a web interface to a chatlog database. The database is recorded through the Colleen or Maiden chatlog modules.

chatter

A chat application based on the Twitter direct messages system. Allows you to chat directly with your twitter friends as if it were a regular chat room. Easy to set up, and runs on all major desktop platforms.

chirp

A client library implementing the full Twitter REST API. If you want to interact with Twitter, this is your best bet.

cl-fond

A bindings library to libfond, allowing you to use its functionality easily from Lisp out. Libfond allows the rendering of TrueType fonts onto OpenGL textures.

cl-gamepad

A bindings library to libstem_gamepad, providing easy gamepad and joystick event processing from Lisp. This is useful if you don't want to use some kind of framework that brings all sorts of other baggage with it, not just gamepad processing.

cl-gpio

A wrapper library for the Linux General Purpose IO device present on embedded systems such as the Raspberry Pi. It allows you to conveniently access and control the IO pins on the board.

cl-k8055

A bindings library for the k8055 analog input board. Allows you to read its various values and set outputs.

cl-mixed

A bindings library to libmixed, allowing you to use its functionality from Lisp. Libmixed allows you to perform digital audio mixing and processing. Thus, with this, you can do efficient DSP from Lisp.

cl-monitors

A bindings library to libmonitors, providing convenient access to information about the currently attached monitors, and giving you the ability to control the resolution thereof.

cl-mpg123

A bindings library to libmpg123, giving you fast and easy to use MP3 decoding. This covers the complete API exposed by libmpg123, and thus easily your best bet for MP3 processing.

cl-out123

A bindings library to libout123, giving you cross-platform audio output. The API is very simple to use, and can thus give you a quick start if you need to play some audio.

cl-soloud

A bindings library to SoLoud, an open source C++ sound engine for the use in video games. I've completed this, but dropped it, as it was too hostile to extension from CL. I've since developed Harmony (see below).

cl-spidev

A wrapper library for the Linux Serial Port Interface device. With this you can do serial port input/output, which is present on some embedded devices like the Raspberry Pi.

clip

Clip is an alternative approach to templating, expressing the template within valid HTML. This allows a different development approach, wherein you can hammer out a mock-up for a website in an HTML document, and then simply add templating logic through further tags and attributes, maintaining browser-viewability.

colleen

This is the predecessor to Maiden, with a more narrow focus and feature base. Since it has been superseded, and the code is quite crusty, I heavily urge you to look at the Maiden project instead.

crypto-shortcuts

A tiny library to provide commonly used cryptography functions in a more accessible format, as some of the tools provided by Ironclad & co. can be a bit cumbersome to use.

deeds

Deeds is an Extensible and Event Delivery System. It offers both flexible and performant creation of event systems. Deeds is used heavily in Maiden.

deferred

This was an attempt at making optional dependency wrangling more convenient. It gives you a few tools that attempt to make it possible to write code that is only considered once another system becomes available.

deploy

With Qtools I developed a very convenient mechanism to generate deployments of my systems. This is the evolution of that, allowing you to use it independent from Qt. It takes care of your foreign libraries and the general shutdown and boot sequence, making the whole binary deployment process much smoother.

dissect

Sadly a lot of projects use the "trivial-backtrace" system that just gives them a string with a backtrace. Dissect allows you to capture, step, and completely inspect the stack trace on a variety of Lisp implementations. The introspection abilities allow you to write a good chunk of a portable debugger. It's also very useful for logging and other situations where execution is automatically continued, but the information of the current stack is still useful to store somewhere.

documentation-utils

I like to keep my code nice and clean, and as such docstrings are quite cumbersome clutter. This library allows you to easily and conveniently put all the docstrings in a file outside of the rest of your code.

filebox

This is a Radiance application that provides you with a very simple file storage. Coupled with the filebox-client, you get a Dropbox-like system.

flare

Flare is a particle simulation framework. Unlike most particle systems, it does not focus on the emission of small dots, but rather on the precise coordination of a hierarchy of entities. It allows you to describe sequences of events, and then lets you play those sequences back, performing the actual transformations. You can even rewind time and interactively work on your sequences.

flow

This is a flowchart-like graph library. It gives you access to nodes which, unlike in mathematical graphs, have dedicated ports from which connections are made. These ports can have semantic meaning and syntactic properties. Thus, this gives you a toolkit to make flowchart-like graphs and compute with them.

for

Since I couldn't come to terms with Iterate, I decided to write my own extensible iteration construct. Unlike Loop or Iterate, For has a particular syntax to it that makes extensions feel much more integrated.

form-fiddle

This small library allows you to wrangle lambda forms and destructure them into their individual components (docstring, declarations, arguments, etc).

glsl-toolkit

A toolkit to allow you to process and manipulate OpenGL Shader Language code. It includes a full GLSL4 parser, printer, and code-walker. Using this you can even do stuff like merge separate shaders together automatically, preserving input/output semantics.

halftone

A sample application for the Qtools system, providing you with a minimal, but pretty image viewer. Works on all major desktop platforms.

harmony

Harmony is a fully-fledged audio system, allowing you to control playback of multiple sources, and even to position them in 3D space. It also allows you to build your own sound processing pipelines, to add in effects and other sound processing capabilities.

humbler

This is a client library for the Tumblr REST API. It has full coverage of the documented features and properly wrangles all the oddities and inconsistencies of the API for you.

kanji-tree

This is still in the works, but is intended to become a website (using Radiance) that provides useful information about Kanji, as well as an optimised sequence by which to learn them. Hopefully this will help me and other people to learn Japanese.

keyword-reviews

Another Radiance application that provides a very minimalist site for product reviews. The twist of this site is that your review should be very short, if possible reduced to keywords only. The idea is that this should make for interesting descriptions and interpretations.

lambda-fiddle

The counterpart to form-fiddle, this allows you to wrangle and destructure lambda-lists (argument lists).

legit

An interface to the Git binary. Using this library you can run all the available Git commands using a more convenient and streamlined function interface. An object-oriented high-level interface is also available, but doesn't cover the full API.

libfond

A small C library to allow you to render TrueType fonts to OpenGL textures. Text rendering is something that's often left out of minimal game engines, and so libfond can provide you with that aspect.

libmixed

A small C library to allow you to mix and process digital audio. It is reasonably optimised and comes with a set of processing and mixer components out of the box. One of the components also allows you to integrate LADSPA plugins, so you can use those directly as well.

libmonitors

A small C library to handle the management and information retrieval of connected Monitors. Currently Linux, Windows, and OS X are supported.

lionchat

A native GUI for the Lichat system. While this works well enough as it currently stands, I'd like to rewrite it at some point to use Maiden, and thus allow connecting to other chat systems as well.

lquery

A library modelled after jQuery to allow you to conveniently and succinctly wrangle HTML and XML documents. This is Particularly useful for web scraping tasks.

modularize

This is a system that gives you an extension to the package system, by allowing you to add other metadata to it. This should facilitate the construction of "modules," individual components of a larger system. The metadata can be used to give meaning to the modules and model their relations to the whole system.

modularize-hooks

This augments the modularize system by giving you hooks and triggers. Thus, modules can provide opaque entry points for other modules to provide additional functionality.

modularize-interfaces

This augments the modularize system by giving you "interfaces"- contract-like descriptions of the functionality provided through a package. While the interface description is abstract and only includes the signatures of functions, another module can then opt to implement the actual functionality behind the interface.

north

The successor to the South (Simple OaUTH) library, implementing the full oAuth 1.0a protocol, both client and server sides. Using North you can easily become an oAuth provider or consumer.

parachute

Parachute is a testing framework with an emphasis on being extensible. As proof of this, it includes "compatibility layers" for a couple of other popular testing frameworks. Using such a layer you can immediately convert to using Parachute by just changing a package :use and system :depends-on.

pathname-utils

A small library to help with common pathname wrangling tasks. If you need to work with pathnames a lot, you'll probably find one or two things in here that will prove useful to you. Note that the library is restricted to pathnames, so it won't provide anything that actually touches the file system.

pi-plates

A library implementing the public interface for the PiPlates DAQ plates that you can use in combination with the Raspberry Pi. The library is currently untested, but "should work" as it is a fairly straightforward translation of the official Python code. I haven't yet found the will to actually test it myself.

piping

Piping allows you to write "pipelines." Pipelines are systems of pipe segments that pass along and transform or compute based on the object currently being passed through the pipeline.

plaster

Plaster is another Radiance application. It gives you a usable paste service. The Radiance tutorial even shows you how to write the application from scratch.

plump-bundle

An implementation of a binary storage format for the Plump DOM. It allows you to save a DOM into a more efficiently parseable representation on disk.

plump-sexp

A parser and printer for an s-expression based syntax of an HTML DOM, using the Plump DOM as a back-end.

plump-tex

A parser and printer for a TeX based syntax, using the Plump DOM as a back-end. With this you can parse TeX sources into a DOM and render them as HTML.

plump

A Practically Lenient and Unimpressive Markup Parser for XML and HTML documents. It provides a fast and lenient parser, allowing you to chuck all sorts of malformed data at it. Since it integrates with a bunch of my other systems, it's a pretty good choice for HTML.

post-all

This is a small library to allow you to post content to multiple services at once. I use this to post my art online, as there's a couple of different places I'd otherwise have to upload to manually every time.

purplish

Purplish is yet another Radiance application. It provides you with a slick and simple image board software. If you ever want to run a chan, this could be a good choice.

qt-libs

This system provides you with the Qt4 library binaries. Usually all you have to do is load this system, and you'll be set up and ready to go writing Qt applications. It also includes tools to automatically build the libraries from scratch.

qtools

Qtools allows you to write Qt GUIs in a syntax and manner much more similar to how you'd write any other Lisp code. It provides all sorts of conveniences and abstractions to make life a whole lot easier.

qtools-ui

This is a collection of UI components and systems that Qt does not provide on its own. Being completely written in Lisp, it is also ripe for extension and adaptation in your own projects. If you have a reusable component that you wrote, it would be a great idea to integrate it here, so that others can benefit as well.

random-state

Random-state gives you access to a bunch of different random number generation algorithms, and allows you to portably seed them. This is primarily useful where number generation needs to be controllable.

ratify

This system allows you to verify and parse a variety of string-based formats. It is primarily geared towards validating input from web forms, or other unauthorised sources.

reader

A Radiance application providing you with a simple blogging platform with tags and Atom feeds. This article was published on Reader!

redirect-stream

A tiny library implementing a gray stream that redirects the output written to it to another stream. This is useful when you want to switch out the stream of a particular system on the fly.

simple-tasks

A small library to do simple task issuing and processing. You create tasks that execute some code, and then send them off to be processed on a dedicated background thread.

trivial-arguments

This gives you a single function, which returns the lambda-list of a function, if the list is known. Useful for introspective and descriptive tasks.

trivial-benchmark

A small library to do simple benchmarking work. This library uses CLOS to be easy to extend, which incurs a bit of overhead for the benchmarks themselves. Thus, it is sadly not suitable for micro-benchmarking.

trivial-indent

If you make a macro with a bit of a more advanced syntax, it's likely Slime will not pick up the proper indentation for it. With this, you can help it out by declaring the proper indentation form manually.

trivial-main-thread

Sometimes it's necessary to ensure that code is run in the main thread, especially when you want to do graphics on OS X. This library helps you with that.

trivial-mimes

The detection and handling of mime-types for files is sometimes necessary to validate the correctness of a specified content type. This library implements both a binary lookup, and a file-name lookup.

trivial-thumbnail

This tiny library uses the ImageMagick binaries to create thumbnails of images for you.

ubiquitous

Ubiquitous provides a persistent configuration storage. It gives you convenient traversal through the configuration and offers easy file-based serialisation for a good range of Lisp object types. If you need your application to be configurable through external files, or just need a simple storage, check a look!

Other Stuff

That's about it. I have a bunch of other projects that I haven't mentioned here, either because they're old, abandoned, not even close to finishing, or simply uninteresting.

Since I'm constantly doing things, this list is bound to become outdated before long. So, please be mindful of the date. When in doubt, just look at the project's own page, or contact me directly. I love to chat, so even if you don't care about any of this, I definitely wouldn't mind if you stopped by at #shirakumo on Freenode some time.

07 Oct 2017 3:57pm GMT

28 Sep 2017

feedPlanet Lisp

Quicklisp news: Something to try out: Quicklisp with OpenPGP and SHA verification

I've got a test version of Quicklisp available. It uses pure Common Lisp code to verify file SHA digests and OpenPGP signatures, from bootstrap to library loading.

To try it out, fetch the following file:

https://www.quicklisp.org/tmp/quicklisp.lisp



Load it into a Lisp implementation with (load "quicklisp.lisp") and follow the prompts. It's best to start with a Lisp that doesn't have Quicklisp already loaded automatically from the init file.

The PGP public key for Quicklisp releases is embedded directly in quicklisp.lisp, but you can also fetch it from another source and use :public-key-file "/path/to/separate/key" as an argument to quicklisp-quickstart:install to use a specific key file.

If you do try it, move your existing, working Quicklisp install out of the way first, or use the :path option to install to a test location. Otherwise, you could clobber a working Quicklisp setup.

This verification code slows things down a bit because it does a lot of arithmetic. The slowdown is most dramatic in implementations like ABCL and CLISP.

If everything works as it should, you won't notice anything very different from the normal Quicklisp install, except some slowdown during verification and some output indicating what checks were attempted and passed.

If you run into problems where something doesn't work as you expect, please let me know at zach@quicklisp.org.

Thanks!

28 Sep 2017 4:00pm GMT