21 Oct 2021

feedPlanet Lisp

Quicklisp news: October 2021 Quicklisp dist update now available

New projects:

Updated projects: 3d-matrices, also-alsa, april, architecture.builder-protocol, bdef, beast, bike, bnf, bp, chameleon, check-bnf, chirp, ci-utils, cl+ssl, cl-ana, cl-ansi-term, cl-ansi-text, cl-async, cl-bloggy, cl-collider, cl-colors2, cl-cron, cl-data-structures, cl-dbi, cl-digraph, cl-environments, cl-form-types, cl-forms, cl-gearman, cl-gserver, cl-info, cl-kraken, cl-liballegro-nuklear, cl-libsvm, cl-marshal, cl-megolm, cl-mixed, cl-opencl, cl-opencl-utils, cl-patterns, cl-pdf, cl-permutation, cl-png, cl-readline, cl-schedule, cl-sdl2-mixer, cl-ses4, cl-telebot, cl-utils, cl-wave-file-writer, cl-webdriver-client, cl-webkit, cletris, clj-re, clog, closer-mop, cluffer, clunit2, clx, cmd, colored, common-lisp-jupyter, concrete-syntax-tree, consfigurator, core-reader, croatoan, cytoscape-clj, dartsclhashtree, data-frame, defmain, dfio, djula, dns-client, doc, doplus, easy-routes, eclector, esrap, fare-scripts, fof, fresnel, functional-trees, gadgets, gendl, generic-cl, glacier, gtirb-capstone, gute, harmony, hash-table-ext, helambdap, hunchenissr, imago, ironclad, jingoh, kekule-clj, lack, lambda-fiddle, lass, legit, lisp-namespace, lisp-stat, literate-lisp, log4cl, log4cl-extras, lsx, maiden, markup, math, matrix-case, mcclim, messagebox, mgl-pax, micmac, millet, mito, mnas-graph, mnas-hash-table, mnas-package, mnas-string, mutility, null-package, numerical-utilities, nyxt, omglib, osicat, parachute, petalisp, physical-quantities, plot, portal, postmodern, pp-toml, prompt-for, qlot, query-repl, quilc, read-as-string, resignal-bind, rove, rpcq, salza2, sel, serapeum, sha1, shasht, shop3, sketch, slite, smart-buffer, spinneret, staple, static-dispatch, stealth-mixin, structure-ext, swank-protocol, sycamore, tfeb-lisp-hax, tfeb-lisp-tools, tooter, trace-db, trestrul, trivia, trivial-with-current-source-form, uax-15, uncursed, vellum, vellum-postmodern, vgplot, vk, whirlog, with-c-syntax, zippy.

Removed projects: adw-charting, cl-batis, cl-bunny, cl-dbi-connection-pool, cl-reddit, cl-server-manager, corona, gordon, hemlock, hunchenissr-routes, prepl, s-protobuf, submarine, torta, trivial-swank, weblocks-examples, weblocks-prototype-js, weblocks-tree-widget, weblocks-utils.

To get this update, use (ql:update-dist "quicklisp").

There are a lot of removed projects this month. These projects no longer build with recent SBCLs, and all bug reports have gone ignored for many months. If one of these projects is important to you, consider contributing to its maintenance and help it work again.

Incidentally, this is the eleventh anniversary of the first Quicklisp dist release back in October 2010.

21 Oct 2021 1:47am GMT

TurtleWare: Selective waste collection

When an object in Common Lisp is not reachable it is garbage collected. Some implementations provide the functionality to set finalizers for these objects. A finalizer is a function that is run when the object is not reachable.

Whether the finalizer is run before the object is deallocated or after is a nuance differing between implementations.

On ABCL, CMU CL, LispWorks, Mezzano, SBCL and Scieener CL the finalizer does not accept any arguments and it can't capture the finalized object (because otherwise it will be always reachable); effectively it may be already deallocated. As the least common denominator it is the approach taken in the portability library trivial-garbage.

(let* ((file (open "my-file"))
       (object (make-instance 'pseudo-stream :file file)))
  (flet ((finalize () (close file)))
    (trivial-garbage:set-finalizer object (lambda () (close file))))

On contrary ACL, CCL, clasp, clisp, corman and ECL the finalizer accepts one argument - the finalized object. This relieves the programmer from the concern of what should be captured but puts the burden on the programmer to ensure that there are no circular dependencies between finalized objects.

(let ((object (make-instance 'pseudo-stream :file (open "my-file"))))
  (flet ((finalize (stream) (close (slot-value stream 'file))))
    (another-garbage:set-finalizer object #'finalize)))

The first approach may for instance store weak pointers to objects with registered finalizers and when a weak pointer is broken then the finalizer is called.

The second approach requires more synchronization with GC and for some strategies makes it possible to absolve objects from being collected - i.e by stipulating that finalizers are executed in a topological order one per the garbage collection cycle.

In this post I want to discuss a certain problem related to finalizers I've encountered in an existing codebase. Consider the following code:

(defclass pseudo-stream ()
  ((resource :initarg :resource :accessor resource)))

(defun open-pseudo-stream (uri)
  (make-instance 'pseudo-stream :resource (make-resource uri)))

(defun close-pseudo-stream (object)
  (destroy-resource (resource object))))

(defvar *pseudo-streams* (make-hash-table))

(defun reload-pseudo-streams ()
  (loop for uri in *uris*
        do (setf (gethash uri *pseudo-streams*)
                 (open-pseudo-stream uri))))

The function reopen-pseudo-streams may be executed i.e to invalidate caches. Its main problem is that it leaks resources by not closing the pseudo stream before opening a new one. If the resource consumes a file descriptor then we'll eventually run out of them.

A naive solution is to close a stream after assigning a new one:

(defun reload-pseudo-streams/incorrect ()
  (loop for uri in *uris*
        for old = (gethash uri *pseudo-streams*)
        do (setf (gethash uri *pseudo-streams*)
                 (open-pseudo-stream uri))
           (close-pseudo-stream old)))

This solution is not good enough because it is prone to race conditions. In the example below we witness that the old stream (that is closed) may still be referenced after a new one is put in the hash table.

(defun nom-the-stream (uri)
  (loop
    (let ((stream (gethash uri *pseudo-streams*)))
      (some-long-computation-1 stream)
      ;; reload-pseudo-streams/incorrect called, the stream is closed
      (some-long-computation-2 stream) ;; <-- aaaa
      )))

This is a moment when you should consider abandoning the function reload-pseudo-streams/incorrect and using a finalizer. The new version of the function open-pseudo-stream destroys the resource only when the stream is no longer reachable, so the function nom-the-stream can safely nom.

When the finalizer accepts the object as an argument then it is enough to register the function close-pseudo-stream. Otherwise, since we can't close over the stream, we close over the resource and open-code destroying it.

(defun open-pseudo-stream (uri)
  (let* ((resource (make-resource uri))
         (stream (make-instance 'pseudo-stream :resource resource)))

    #+trivial-garbage ;; closes over the resource (not the stream)
    (flet ((finalizer () (destroy-resource resource)))
      (set-finalizer stream #'finalizer))

    #+another-garbage ;; doesn't close over anything
    (set-finalizer stream #'close-pseudo-stream)

    stream))

Story closed, the problem is fixed. It is late friday afternoon, so we eagerly push the commit to the production system and leave home with a warm feeling of fulfilled duty. Two hours later all hell breaks loose and the system fails. The problem is the following function:

(defun run-client (stream)
  (assert (pseudo-stream-open-p stream))
  (loop for message = (read-message stream)
        do (process-message message)
        until (eql message :server-closed-connection)
        finally (close-pseudo-stream stream)))

The resource is released twice! The first time when the function run-client closes the stream and the second time when the stream is finalized. A fix for this issue depends on the finalization strategy:

#+trivial-garbage ;; just remove the reference
(defun close-pseudo-stream (stream)
  (setf (resource stream) nil))

#+another-garbage ;; remove the reference and destroy the resource
(defun close-pseudo-stream (stream)
  (when-let ((resource (resource steram)))
    (setf (resource stream) nil)
    (destroy-resource resource)))

With this closing the stream doesn't interfere with the finalization. Hurray! Hopefully nobody noticed, it was late friday afternoon after all. This little incident tought us to never push the code before testing it.

We build the application from scratch, test it a little and... it doesn't work. After some investigation we find the culpirt - the function creates a new stream with the same resource and closes it.

(defun invoke-like-a-good-citizen-with-pseudo-stream (original-stream fn)
  (let* ((resource (resource original-stream))
         (new-stream (make-instance 'pseudo-stream :resource resource)))
    (unwind-protect (funcall fn new-stream)
      (close-pseudo-stream new-stream))))

Thanks to our previous provisions closing the stream doesn't collide with finalization however the resource is destroyed for each finalized stream because it is shared between distinct instances.

When the finalizer accepts the collected object as an argument then the solution is easy because all we need is to finalize the resource instead of the pseudo stream (and honestly we should do it from the start!):

#+another-garbage
(defun open-pseudo-stream (uri)
  (let* ((resource (make-resource uri))
         (stream (make-instance 'pseudo-stream :resource resource)))
    (set-finalizer resource #'destroy-resource)
    stream))

#+another-garbage
(defun close-pseudo-stream (stream)
  (setf (resource stream) nil))

When the finalizer doesnt't accept the object we need to do the trick and finalize a shared pointer instead of a verbatim resource. This has a downside that we need to always unwrap it when used.

#+trivial-garbage
(defun open-pseudo-stream (uri)
  (let* ((resource (make-resource uri))
         (wrapped (list resource))
         (stream (make-instance 'pseudo-stream :resource wrapped)))
    (flet ((finalize () (destroy-resource resource)))
      (set-finalizer wrapped #'finalize)
    stream))

#+trivial-garbage
(defun close-pseudo-stream (stream)
  (setf (resource stream) nil))

When writing this post I've got too enthusiastic and dramatized a little about the production systems but it is a fact, that I've proposed a fix similar to the first finalization attempt in this post and when it got merged it broke the production system. That didn't last long though because the older build was deployed almost immedietely. Cheers!

21 Oct 2021 12:00am GMT

19 Oct 2021

feedPlanet Lisp

Eitaro Fukamachi: Day 1: Roswell, as a Common Lisp implementation manager

This is my first public article in English. I've been sending out newsletters about what I've been doing only to sponsors, but there have been requests to publish my know-how on my blog, so I'm writing this way.

However, my English skills are still developing, so I can't suddenly deliver a lot of information at once. So instead, I'm going to start writing fragments of knowledge in the form of technical notes, little by little. The articles may not be in order. But I suppose each one would help somehow as a tip for your Common Lisp development.

When I thought of what I should start from, "Roswell" seemed appropriate, because most of the topics I want to tell depends on it.

It's been six years since Roswell was born. Although its usage has been expanding, I still feel that Roswell is underestimated, especially among the English community.

Not because of you. I think a lot of the reason for this is that the author is Japanese, like me, and has neglected to send out information in English.

If you are not familiar with Roswell or have tried it before but didn't get as much use out of it as you wanted, I hope this article will make you interested.

What's Roswell

Roswell has the following features:

It would be too much work to explain everything in a single article, so I will explain from the first one today: installation of Common Lisp implementations.

Installation

See the Official Installation Guide.

Installation of Common Lisp implementations

To install implementations with Roswell, use its "install" subcommand.

$ ros help install
Usage:

To install a new Lisp implementaion:
   ros install impl [options]
or a system from the GitHub:
   ros install fukamachi/prove/v2.0.0 [repository... ]
or an asdf system from quicklisp:
   ros install quicklisp-system [system... ]
or a local script:
   ros install ./some/path/to/script.ros [path... ]
or a local system:
   ros install ./some/path/to/system.asd [path... ]

For more details on impl specific options, type:
   ros help install impl

Candidates impls for installation are:
abcl-bin
allegro
ccl-bin
clasp-bin
clasp
clisp
cmu-bin
ecl
mkcl
sbcl-bin
sbcl-head
sbcl
sbcl-source

For instance, SBCL, currently the most popular implementation, can be installed with sbcl-bin or sbcl.

# Install the latest SBCL binary
$ ros install sbcl-bin

# Install the SBCL 2.1.7 binary
$ ros install sbcl-bin/2.1.7

# Build and install the latest SBCL from the source
$ ros install sbcl

Since Roswell author builds and hosts its own SBCL binaries, it can install more versions of binaries than the official binary support. So in most cases, you can just run ros install sbcl-bin/<version> to install a specific version of SBCL.

After installing a new Lisp, it will automatically be in the active one. To switch implementations/versions, ros use command is available.

# Switch to SBCL 2.1.7 binary version
$ ros use sbcl-bin/2.1.7

# Switch to ECL of the latest installed version
$ ros use ecl

To see what implementations/versions are installed, ros list installed is available.

$ ros list installed
Installed implementations:

Installed versions of ecl:
ecl/21.2.1

Installed versions of sbcl-bin:
sbcl-bin/2.1.7
sbcl-bin/2.1.9

Installed versions of sbcl-head:
sbcl-head/21.9.21

To check the active implementation, run ros run -- --version.

# Print the active implementation and its version
$ ros run -- --version
SBCL 2.1.7

Run REPL with Roswell

To start a REPL, execute ros run.

# Start the REPL of the active Lisp
$ ros run

# Start the REPL of a specific implementation/version
$ ros -L sbcl-bin/2.1.7 run

"sbcl" command needed?

For those of you who have been installing SBCL from a package manager, the lack of the sbcl command may be disconcerting. Some people are relying on the "sbcl" command in your editor settings. As a workaround to install the "sbcl" command, such as the following command would help.

$ printf '#!/bin/sh\nexec ros -L sbcl-bin run -- "$@"\n' | \
    sudo tee /usr/local/bin/sbcl \
  && sudo chmod +x /usr/local/bin/sbcl

Though once you get used to it, I'm sure you'll naturally start using ros run.

Conclusion

I introduced the following subcommand/options in this article.

(Rough) Troubleshooting

If you have a problem like "Roswell worked fine at first but won't work after I updated SBCL," simply delete ~/.roswell .

Roswell writes all related files under the directory, like configurations, Lisp implementations, and Quicklisp libraries, etc. When the directory doesn't exist, Roswell creates and initializes it implicitly. So it's okay to delete ~/.roswell.

19 Oct 2021 2:26am GMT

10 Nov 2011

feedPlanet Python

Terry Jones: Emacs buffer mode histogram

Tonight I noticed that I had over 200 buffers open in emacs. I've been programming a lot in Python recently, so many of them are in Python mode. I wondered how many Python files I had open, and I counted them by hand. About 90. I then wondered how many were in Javascript mode, in RST mode, etc. I wondered what a histogram would look like, for me and for others, at times when I'm programming versus working on documentation, etc.

Because it's emacs, it wasn't hard to write a function to display a buffer mode histogram. Here's mine:

235 buffers open, in 23 distinct modes

91               python +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
47          fundamental +++++++++++++++++++++++++++++++++++++++++++++++
24                  js2 ++++++++++++++++++++++++
21                dired +++++++++++++++++++++
16                 html ++++++++++++++++
 7                 text +++++++
 4                 help ++++
 4           emacs-lisp ++++
 3                   sh +++
 3       makefile-gmake +++
 2          compilation ++
 2                  css ++
 1          Buffer-menu +
 1                 mail +
 1                 grep +
 1      completion-list +
 1                   vm +
 1                  org +
 1               comint +
 1              apropos +
 1                 Info +
 1           vm-summary +
 1      vm-presentation +

Tempting as it is, I'm not going to go on about the heady delights of having a fully programmable editor. You either already know, or you can just drool in slack-jawed wonder.

Unfortunately I'm a terrible emacs lisp programmer. I can barely remember a thing each time I use it. But the interpreter is of course just emacs itself and the elisp documentation is in emacs, so it's a really fun environment to develop in. And because emacs lisp has a ton of support for doing things to itself, code that acts on emacs and your own editing session or buffers is often very succinct. See for example the save-excursion and with-output-to-temp-buffer functions below.

(defun buffer-mode-histogram ()
"Display a histogram of emacs buffer modes."
(interactive)
(let* ((totals '())
(buffers (buffer-list()))
(total-buffers (length buffers))
(ht (make-hash-table :test 'equal)))
(save-excursion
(dolist (buffer buffers)
(set-buffer buffer)
(let
((mode-name (symbol-name major-mode)))
(puthash mode-name (1+ (gethash mode-name ht 0)) ht))))
(maphash (lambda (key value)
(setq totals (cons (list key value) totals)))
ht)
(setq totals (sort totals (lambda (x y) (> (cadr x) (cadr y)))))
(with-output-to-temp-buffer "Buffer mode histogram"
(princ (format "%d buffers open, in %d distinct modes\n\n"
total-buffers (length totals)))
(dolist (item totals)
(let
((key (car item))
(count (cadr item)))
(if (equal (substring key -5) "-mode")
(setq key (substring key 0 -5)))
(princ (format "%2d %20s %s\n" count key
(make-string count ?+))))))))

Various things about the formatting could be improved. E.g., not use fixed-width fields for the count and the mode names, and make the + signs indicate more than one buffer mode when there are many.

10 Nov 2011 2:42pm GMT

Mike Driscoll: wxPython: ANN: Namespace Diff Tool

Last night, Andrea Gavana released his new Namespace Diff Tool (NDT) to the world. I got his permission to reprint his announcement here for all those people who don't follow the wxPython mailing list. I think it sounds like a really cool tool. You should check it out and see what you think. Here is the announcement:

Description
===========

The `Namespace Diff Tool` (NDT) is a graphical user interface that can
be used to discover differences between different versions of a library,
or even between different iterations/sub-versions of the same library.

The tool can be used to identify what is missing and still needs to be
implemented, or what is new in a new release, which items do not have
docstrings and so on.

Full description of the original idea by Robin Dunn:

http://svn.wxwidgets.org/viewvc/wx/wxPython/Phoenix/trunk/TODO.txt?vi…

:warning: As most of the widgets in the GUI are owner drawn or custom,
it is highly probable that the interface itself will look messy on other
platforms (Mac, I am talking to you). Please do try and create a patch to
fix any possible issue in this sense.

:note: Please refer to the TODOs section for a list of things that still
need
to be implemented.

Requirements
============

In order to run NDT, these packages need to be installed:

- Python 2.X (where 5 = X = 7);
- wxPython >= 2.8.10;
- SQLAlchemy >= 0.6.4.

More detailed instructions on how to use it, TODO items, list of
libraries/packages I tested NDT against, screenshots and download links can
be found here:

http://xoomer.virgilio.it/infinity77/main/NDT.html

If you stumble upon a bug (which is highly probable), please do let me
know. But most importantly, please do try and make an effort to create a
patch for the bug.

According to the thread, some bugs were already found and fixed.

10 Nov 2011 1:15pm GMT

feedPlanetJava

OSDir.com - Java: Oracle Introduces New Java Specification Requests to Evolve Java Community Process

From the Yet Another dept.:

To further its commitment to the Java Community Process (JCP), Oracle has submitted the first of two Java Specification Requests (JSRs) to update and revitalize the JCP.

10 Nov 2011 6:01am GMT

OSDir.com - Java: No copied Java code or weapons of mass destruction found in Android

From the Fact Checking dept.:

ZDNET: Sometimes the sheer wrongness of what is posted on the web leaves us speechless. Especially when it's picked up and repeated as gospel by otherwise reputable sites like Engadget. "Google copied Oracle's Java code, pasted in a new license, and shipped it," they reported this morning.



Sorry, but that just isn't true.

10 Nov 2011 6:01am GMT

OSDir.com - Java: Java SE 7 Released

From the Grande dept.:

Oracle today announced the availability of Java Platform, Standard Edition 7 (Java SE 7), the first release of the Java platform under Oracle stewardship.

10 Nov 2011 6:01am GMT

feedPlanet Python

Andy Todd: Extracting a discrete set of values

Today's I love Python moment is bought to you by set types.

I have a file, XML naturally, the contains a series of transactions. Each transaction has a reference number, but the reference number may be repeated. I want to pull the distinct set of reference numbers from this file. The way I learnt to build up a discrete set of items (many years ago) was to use a dict and set default.

>>> ref_nos = {}
>>> for record in records:
>>>     ref_nos.setdefault(record.key, 1)
>>> ref_nos.keys()

But Python has had a sets module since 2.3 and the set standard data type since 2.6 so my knowledge is woefully out of date. The latest way to get the unique values from a sequence looks something like this;

>>> ref_nos = set([record.key for record in records])

I think I should get bonus points for using a list comprehension as well.

10 Nov 2011 5:42am GMT

09 Nov 2011

feedPlanet Grep

Thomas Vander Stichele: Mach 1.0.0 “Madera” released

Another November, another Fedora. 16 came out, so it was time to update mach again.

And today I thought, is there any reason mach isn't 1.0 yet ? Am I going to do anything more to this piece of code before I want to call it that ?

And the answer is, no. It's the first Python application I've written, and I'm not particularly proud of the code, but I'm happy I've made good use of it for so long, and that it helped push packaging approaches forward and sparked ideas for the Fedora build system.

Since I didn't like the original code for mach2 (there was a version 1 which was Makefile-based), I started a rewrite with unit tests, better code layout, decent classes for abstracting distro-specific stuff, and so on.

The experience of how mock was created based off mach2 was a slightly sour one however, so I wasn't really motivated to finish the mach3 rewrite. Sometimes that's the drawback of open source - sure, forking is specifically allowed, so don't whine about it when it happens. But when it's done gratuitously, with no serious attempt at collaborating, it doesn't feel like it's in the spirit of open source.

Anyway, that was a long time ago. mach2 as it is today, is done. It really only needs updating for newer versions. As long as it works for me, it's unlikely I will continue mach3, but who knows?

Enjoy the release!

09 Nov 2011 10:56pm GMT

feedPlanet Debian

Matthew Garrett: Properly booting a Mac

This is mostly for my own reference, but since it might be useful to others:

By "Properly booting" I mean "Integrating into the boot system as well as Mac OS X does". The device should be visible from the boot picker menu and should be selectable as a startup disk. For this to happen the boot should be in HFS+ format and have the following files:

That's enough to get it to appear in the startup disk boot pane. Getting it in the boot picker requires it to be blessed. You probably also want a .VolumeIcon.icns in / in order to get an appropriate icon.

Now all I need is an aesthetically appealing boot loader.

comment count unavailable comments

09 Nov 2011 10:06pm GMT

Martin Zobel-Helas: How to read Debian's mailing list archives locally

From time to time i want to answer on mails on Debian mailinglists that i am not subcribed to. To have proper reply-headers set, i usually copied the archive mbox from master.debian.org to my local machine.

Now i found a much nicer way.

apt-get install fuse afuse sshfs
adduser zobel fuse
mkdir ~/fuse/
afuse -o mount_template="sshfs %r:/ %m" -o unmount_template="fusermount -u -z %m" -o timeout=60 ~/fuse
mutt -f /home/zobel/fuse/master.debian.org/home/debian/lists/debian-user/debian-user.201111

09 Nov 2011 10:04pm GMT

Gunnar Wolf: On the social-based Web and my reluctance to give it my time

I recently started getting mails from no-reply@joindiaspora.com. Usually, a mail from no-reply@whatever is enough to make me believe that the admins of said whatever are clueless regarding what e-mail means and how should it work. And in this case, it really amazes me - If I get an invite to Diaspora*, right, I should not pester a hypothetical sysadmin@joindiaspora.com to get me off his list, but I should be able to reply to the person mailing me - Maybe requesting extra details on what he is inviting me to, or allowing me to tell him why I'm not interested. But yes, Diaspora* has fallen to the ease of requiring me to join their network to be able to communicate back with the "friend" who invited me.

Some of the (three?) readers of this site might not be familiar with the Diaspora* project. It is a free reimplementation (as far as I know) of something similar to Facebook - Free not only in the sense that it runs free software, but also because it is federated - Your data will not belong to a specific company (that is, you are not the value object they sell and make money with), but you can choose and switch (or become) the provider for your information. A very interesting proposal, socially and technically.

I find that a gross violation of netiquette. I should be able to reply to the mail - Even if in this case it were to (and sorry - As you are spreading my name/mail, you will excuse me if I spread your name ;-) ) fernando.estrada.invite1501335@joindiaspora.com. Such an (fictional FWIW) address would allow for mail to reach back the submitter by the same medium it was sent, without allowing open spamming into the network.

Now, what prompted me to write this mail (just before adding no-reply@joindiaspora.com to my blacklist) is the message I got (in an ugly HTML-only mail which erroneously promised to be text/plain, sigh...) is that Fernando sent me as the inviting message, «So, at least are you going to give Diaspora a chance?»

The answer is: No..

But not because of being a fundamentalist. Right, I am among what many people qualify as Free Software zealots, but many of my choices (as this one is) is in no way related to the software's freeness. I use non-free Web services, as much as many of you do. Yes, I tend to use them less, rather than more (as the tendency goes).

But the main reason I don't use Twitter is the same reason I don't use Identi.ca, its free counterpart - And the reason I'm not interested in Facebook is the same reason I will not join Diaspora* - Because I lack time for yet another stream of activity, of information, of things to do and think about.

Yes, even if I care about you and I want to follow what's going on in your life: The best way to do it is to sit over a cup of coffee, or have some dinner, or to meet once a year in the most amazing conference ever. Or we can be part of distributed projects together, and we will really interact lots. Or you can write a blog! I do follow the blogs of many of my friends (plus several planets), even if they have fallen out of fashion - A blog post pulls me to read it as it is a unit of information, not too much depending on context (a problem when I read somebody's Twitter/Identica lines: You have to hunt a lot of conversations to understand what's going on), gives a true dump of (at least one aspect of) your state of (mind|life|work), and is a referenceable unit I can forward to other people, or quote if needed.

So, yes, I might look old-fashioned, clinging to the tools of the last-decade for my Social Web presence. I will never be a Social Media Expert. I accept it - But please, don't think it is a Stallmanesque posture from me. It is just that of a person who can lose too much time, and needs to get some work done in the meantime.

(oh, of course: Blog posts also don't have to make much sense or be logically complete. But at least they allow me to post a full argument!)

09 Nov 2011 5:55pm GMT

feedPlanet Grep

Xavier Mertens: Biology Rules Apply to Infosec?

(Source: www.esa.org)

In biology, it is proven that consanguinity between members belonging to the same group (example: people living in the same closed area or animals from the same breed) may affect their resistance to certain diseases or reduce certain physical characteristics. It's important to keep some level of diversity. The latest Juniper story made me remember the talk about "monoculture" presented at BlackHat Europe 2011.

A few days ago, some parts of the Internet were affected by a bug in Juniper routers BGP update code. If you have a look at the market of the core-routers, it is dominated by two manufacturers: Cisco & Juniper. Routers operated by major ISPs are crucial to maintain the Internet reliable. If most of those devices are coming from a unique manufacturer (or a very limited number of them), you increase the risks to face big issues if they are affected by a bug or a security flaw.

Now, speaking about devices or applications in general (the core-routers were just an example) and from a business point of view, monoculture is positive:

But, putting the layer-8 (the "political layer") aside, monoculture has side effects:

Like in biology, monoculture can generate catastrophic situations in case of a successful attack or major bug. I don't say that big players do a bad job (otherwise they could never reach such part of the market). Just don't behave like a lemming. Choose the solution which match your requirements and not just because "it's a big name".

Do you remember the French movie "Les Rivières Pourpre" ("The Crimson Rivers") with the closed society of Guernon?

09 Nov 2011 1:44pm GMT

Thomas Bouve: Flynn has to spend 10 days at the hospital due to a kidney &...



Flynn has to spend 10 days at the hospital due to a kidney & blood infection :_( (Taken with Instagram at AZ Sint-Lucas)

09 Nov 2011 12:18pm GMT

08 Nov 2011

feedfosdem - Google Blog Search

papupapu39 (papupapu39)'s status on Tuesday, 08-Nov-11 00:28 ...

papupapu39 · http://identi.ca/url/56409795 #fosdem #freeknowledge #usamabinladen · about a day ago from web. Help · About · FAQ · TOS · Privacy · Source · Version · Contact. Identi.ca is a microblogging service brought to you by Status.net. ...

08 Nov 2011 12:28am GMT

05 Nov 2011

feedfosdem - Google Blog Search

Write and Submit your first Linux kernel Patch | HowLinux.Tk ...

FOSDEM (Free and Open Source Development European Meeting) is a European event centered around Free and Open Source software development. It is aimed at developers and all interested in the Free and Open Source news in the world. ...

05 Nov 2011 1:19am GMT

03 Nov 2011

feedfosdem - Google Blog Search

Silicon Valley Linux Users Group – Kernel Walkthrough | Digital Tux

FOSDEM (Free and Open Source Development European Meeting) is a European event centered around Free and Open Source software development. It is aimed at developers and all interested in the Free and Open Source news in the ...

03 Nov 2011 3:45pm GMT

28 Oct 2011

feedPlanet Ruby

O'Reilly Ruby: MacRuby: The Definitive Guide

Ruby and Cocoa on OS X, the iPhone, and the Device That Shall Not Be Named

28 Oct 2011 8:00pm GMT

14 Oct 2011

feedPlanet Ruby

Charles Oliver Nutter: Why Clojure Doesn't Need Invokedynamic (Unless You Want It to be More Awesome)

This was originally posted as a comment on @fogus's blog post "Why Clojure doesn't need invokedynamic, but it might be nice". I figured it's worth a top-level post here.

Ok, there's some good points here and a few misguided/misinformed positions. I'll try to cover everything.

First, I need to point out a key detail of invokedynamic that may have escaped notice: any case where you must bounce through a generic piece of code to do dispatch -- regardless of how fast that bounce may be -- prevents a whole slew of optimizations from happening. This might affect Java dispatch, if there's any argument-twiddling logic shared between call sites. It would definitely affect multimethods, which are using a hand-implemented PIC. Any case where there's intervening code between the call site and the target would benefit from invokedynamic, since invokedynamic could be used to plumb that logic and let it inline straight through. This is, indeed, the primary benefit of using invokedynamic: arbitrarily complex dispatch logic folds away allowing the dispatch to optimize as if it were direct.

Your point about inference in Java dispatch is a fair one...if Clojure is able to infer all cases, then there's no need to use invokedynamic at all. But unless Clojure is able to infer all cases, then you've got this little performance time bomb just waiting to happen. Tweak some code path and obscure the inference, and kablam, you're back on a slow reflective impl. Invokedynamic would provide a measure of consistency; the only unforeseen perf impact would be when the dispatch turns out to *actually* be polymorphic, in which case even a direct call wouldn't do much better.

For multimethods, the benefit should be clear: the MM selection logic would be mostly implemented using method handles and "leaf" logic, allowing hotspot to inline it everywhere it is used. That means for small-morphic MM call sites, all targets could potentially inline too. That's impossible without invokedynamic unless you generate every MM path immediately around the eventual call.

Now, on to defs and Var lookup. Depending on the cost of Var lookup, using a SwitchPoint-based invalidation plus invokedynamic could be a big win. In Java 7u2, SwitchPoint-based invalidation is essentially free until invalidated, and as you point out that's a rare case. There would essentially be *no* cost in indirecting through a var until that var changes...and then it would settle back into no cost until it changes again. Frequently-changing vars could gracefully degrade to a PIC.

It's also dangerous to understate the impact code size has on JVM optimization. The usual recommendation on the JVM is to move code into many small methods, possibly using call-through logic as in multimethods to reuse the same logic in many places. As I've mentioned, that defeats many optimizations, so the next approach is often to hand-inline logic everywhere it's used, to let the JVM have a more optimizable view of the system. But now we're stepping on our own feet...by adding more bytecode, we're almost certainly impacting the JVM's optimization and inlining budgets.

OpenJDK (and probably the other VMs too) has various limits on how far it will go to optimize code. A large number of these limits are based on the bytecoded size of the target methods. Methods that get too big won't inline, and sometimes won't compile. Methods that inline a lot of code might not get inlined into other methods. Methods that inline one path and eat up too much budget might push out more important calls later on. The only way around this is to reduce bytecode size, which is where invokedynamic comes in.

As of OpenJDK 7u2, MethodHandle logic is not included when calculating inlining budgets. In other words, if you push all the Java dispatch logic or multimethod dispatch logic or var lookup into mostly MethodHandles, you're getting that logic *for free*. That has had a tremendous impact on JRuby performance; I had previous versions of our compiler that did indeed infer static target methods from the interpreter, but they were often *slower* than call site caching solely because the code was considerably larger. With invokedynamic, a call is a call is a call, and the intervening plumbing is not counted against you.

Now, what about negative impacts to Clojure itself...

#0 is a red herring. JRuby supports Java 5, 6, and 7 with only a few hundred lines of changes in the compiler. Basically, the compiler has abstract interfaces for doing things like constant lookup, literal loading, and dispatch that we simply reimplement to use invokedynamic (extending the old non-indy logic for non-indified paths). In order to compile our uses of invokedynamic, we use Rémi Forax's JSR-292 backport, which includes a "mock" jar with all the invokedynamic APIs stubbed out. In our release, we just leave that library out, reflectively load the invokedynamic-based compiler impls, and we're off to the races.

#1 would be fair if the Oracle Java 7u2 early-access drops did not already include the optimizations that gave JRuby those awesome numbers. The biggest of those optimizations was making SwitchPoint free, but also important are the inlining discounting and MutableCallSite improvements. The perf you see for JRuby there can apply to any indirected behavior in Clojure, with the same perf benefits as of 7u2.

For #2, to address the apparent vagueness in my blog post...the big perf gain was largely from using SwitchPoint to invalidate constants rather than pinging a global serial number. Again, indirection folds away if you can shove it into MethodHandles. And it's pretty easy to do it.

#3 is just plain FUD. Oracle has committed to making invokedynamic work well for Java too. The current thinking is that "lambda", the support for closures in Java 7, will use invokedynamic under the covers to implement "function-like" constructs. Oracle has also committed to Nashorn, a fully invokedynamic-based JavaScript implementation, which has many of the same challenges as languages like Ruby or Python. I talked with Adam Messinger at Oracle, who explained to me that Oracle chose JavaScript in part because it's so far away from Java...as I put it (and he agreed) it's going to "keep Oracle honest" about optimizing for non-Java languages. Invokedynamic is driving the future of the JVM, and Oracle knows it all too well.

As for #4...well, all good things take a little effort :) I think the effort required is far lower than you suspect, though.

14 Oct 2011 2:40pm GMT

07 Oct 2011

feedPlanet Ruby

Ruby on Rails: Rails 3.1.1 has been released!

Hi everyone,

Rails 3.1.1 has been released. This release requires at least sass-rails 3.1.4

CHANGES

ActionMailer

ActionPack

ActiveModel

ActiveRecord

ActiveResource

ActiveSupport

Railties

SHA-1

You can find an exhaustive list of changes on github. Along with the closed issues marked for v3.1.1.

Thanks to everyone!

07 Oct 2011 5:26pm GMT

26 Jul 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Update your RSS link

If you see this message in your RSS reader, please correct your RSS link to the following URL: http://fosdem.org/rss.xml.

26 Jul 2008 5:55am GMT

25 Jul 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Archive of FOSDEM 2008

These pages have been archived.
For information about the latest FOSDEM edition please check this url: http://fosdem.org

25 Jul 2008 4:43pm GMT

09 Mar 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Slides and videos online

Two weeks after FOSDEM and we are proud to publish most of the slides and videos from this year's edition.

All of the material from the Lightning Talks has been put online. We are still missing some slides and videos from the Main Tracks but we are working hard on getting those completed too.

We would like to thank our mirrors: HEAnet (IE) and Unixheads (US) for hosting our videos, and NamurLUG for quick recording and encoding.

The videos from the Janson room were live-streamed during the event and are also online on the Linux Magazin site.

We are having some synchronisation issues with Belnet (BE) at the moment. We're working to sort these out.

09 Mar 2008 3:12pm GMT