fidzu : software

Although LLMs perform a straightforward operation - they predict the next tokens from a sequence of tokens - they can be almost magical in their results if the stars are aligned. And from the look of it, the stars align often enough to be useful. But if you're unlucky, you can end up with a useless pile of garbage. My LLM started spitting out such gems as Cascadescontaminantsunnatural and exquisiteacquire the other day when I requested it imagine some dialog. Your mileage will vary, a lot.

The question is whether the magic outweighs the glossolalia. Can we keep the idiot savant LLM from evangelically speaking in tongues?

Many people at work are reluctant to use LLMs as an aid to programming, preferring to hand craft all their code. I understand the sentiment, but I think it is a mistake. LLMs are a tool of extraordinary power, but you need to develop the skill to use them, and that takes a lot of time and practice.

The initial key to using LLMs is to get good at prompting them. Here a trained programmer has a distinct advantage over a layperson. When you program at a high level, you are not only thinking about how to solve your problem, but also all the ways you can screw up. This is "defensive programming". You check your inputs, you write code to handle "impossible" cases, you write test cases that exercise the edge cases. (I'm no fan of test-driven development, but if I have code that is supposed to exhibit some complex behavior, I'll often write a few test cases to prove that the code isn't egregiously broken.)

When you prompt an LLM, it helps a lot to think in the same way you program. You need to be aware of the ways the LLM can misinterpret your prompt, and you need to write your prompt so that it is as clear as possible. You might think that this defeats the purpose. You are essentially performing the act of programming with an extra natural language translation step in the middle. This is true, and you will get good results if you approach the task with this in mind. Learning to effectively prompt an LLM is very similar to learning a new programming language. It is a skill that a trained programmer will have honed over time. Laypeople will find it possible to generate useful code with an LLM, but they will encounter bugs and problems that they will have difficulty overcoming. A trained programmer will know precisely how to craft additional clauses to the prompt to avoid these problems.

Context engineering is the art of crafting a series of prompts to guide the LLM to produce the results you want. If you know how to program, you don't necessarily know how to engineer large systems. If you know how to prompt, you don't necessarily know how to engineer the context. Think of Mickey Mouse in Fantasia. He quickly learns the prompts that get the broom to carry the water, but he doesn't foresee the consequences of exponential replication.

Ever write a program that seems to be taking an awfully long time to run? You do a back-of-the-envelope calculation and realize that the expected runtime will be on the order of 10⁵⁰ seconds. This sort of problem won't go away with an LLM, but the relative number of people ill-equipped to diagnose and deal with the problem will certainly go up. Logical thinking and foreseeing of consequences will be skills in higher demand than ever in the future.

You won't be able to become a "machine whisperer" without a significant investment of time and effort. As a programmer, you already have a huge head start. Turn on the LLM and use it in your daily workflow. Get a good feel for its strengths and weaknesses (they'll surprise you). Then leverage this crazy tool for your advantage. It will make you a better programmer.

31 Jul 2025 2:14am GMT

I'm a novice to the LLM API, and I'm assuming that at least some of my readers are too. I'm not the very last person to the party, am I?

When integrating the LLM with Lisp, we want to allow the LLM to direct queries back to the Lisp that is invoking it. This is done through the function call protocol. The client supplies to the LLM a list of functions that the LLM may invoke. When the LLM wants to invoke the function, instead of returing a block of generated text, it returns a JSON object indicating a function call. This contains the name of the function and the arguments. The client is supposed to invoke the function, but to return an answer, it actually makes a new call into the LLM and it concatenates the entire conversation so far along with the result of the function call. It is bizarro continuation-passing-style where the client acts as a trampoline and keeps track of the continuation.

So, for example, by exposing lisp-implementation-type and lisp-implementation-version, we can then query the LLM:

> (invoke-gemini "gemini-2.5-flash" "What is the type and version of the lisp implementation?")
"The Lisp implementation is SBCL version 2.5.4."

30 Jul 2025 2:49pm GMT

The following contributors got their Debian Developer accounts in the last two months:

• Cordell Bloor (cgmb)
• Enkelena Haxhija (enkelenah)

The following contributors were added as Debian Maintainers in the last two months:

• Karsten Schöke
• Lorenzo Puliti
• Nick Rosbrook
• Nicolas Peugnet
• Yifei Zhan
• Glenn Strauss
• Fab Stz
• Matheus Polkorny
• Manuel Elias Guerra Figueroa

Congratulations!

30 Jul 2025 12:00pm GMT

After I wrote last week that OEIS A286874 would stop at a(12) and that computing (verifying) a(13) would take about 4-5000 CPU years, the changes have finally been approved, and… the sequence includes a(13) = 26. What happened?

Well, first of all, I am indeed not a mathematical genius (the last post even forgot the "not"); I had a stupid conversion error in the estimation, causing a factor 25 or so. But the rest came from actual speedups.

First of all, I improved one of the existing symmetry detectors a bit (the one described last in the previous post was not fully rejecting the possible symmetries when multiple new bits were introduced in one value). But I also made a more universal symmetry detector; if switching the order of certain neighboring bits and re-sorting the sequence made it lexicographically smaller, then we can abort the search. This is pretty expensive and only rejects ~5% of candidates, so it's only worth it at higher levels, but it's much cheaper than checking all n! arbitrary permutations and catches maybe 90% of a full rejection. (Also, if you can reject 5% at multiple levels, those percentages tend to add up. We're down from hundreds of thousands of duplicate solutions, to only a bit over 100, so the amount of speedup available from reducing symmetries is rapidly dwindling.)

Also, surprisingly to me, before going on to run the next level, doing a population count to check if there were too few bits to ever be a solution was seemingly a large win (e.g. are have three values so far, but only 21 bits left; we can never generate a sequence larger than 24 even if all the stars align, and can abort immediately). You would think that this counting, which takes very real CPU time even with vectorization, wouldn't be worth it compared to just running through the base layers of the recursion very quickly, but evidently, it is by a large margin. I guess it's a very common case to have many more than 1 bit left but less than 26-n, and it also means you can just stop iterating a bit before you get to the end.

But perhaps the most impactful optimization was a microoptimization. Recall that we spent most of our time ANDing 8192-bit vectors (which would be 16384-bit vectors for a(13)) with each other. Some looking at performance metrics suggested that the RAM bandwidth was completely maxed out, with ~80% of theoretical bandwidth in use; only faster RAM or more memory channels would have made a reasonable dent in the performance of this kind of architecture.

But pretty early, most of those bits will be zero. If you've already decided on the first five values in a sequence, you will not have 8187 options left; in most cases, you'll have more like 3-400. And since the bit sets only ever shrink, we can simply compress away all those known zeros. For most of our purposes, it doesn't really decide what each bit signifies (an important exception is the point where we have a valid solution and need to print it out, but it's not hard to store the mapping), as we mostly use the values for looking up pregenerated vectors to AND together. This means that when we start a new sub-job, we can find which future values are possible, and then map those into new numbers 0 through 511 (or whatever). This means we can use 512-bit vectors instead of 8192-bit vectors, with all the obvious advantages; less ALU work, less memory traffic, and better cache locality. (It's interesting that we started by being extremely ALU-bound, then moved to being very RAM-bound, and then ended up in fairly normal territory.)

Of course, there are situations where you could have more than 512 valid values. In that case, you can either recompile with larger bit sets (typically a multiple of 128, to get good use of SIMD), or you can split into smaller sub-jobs; find all valid ways of extending the sequence by one element (trivial; we already did that to make the bit sets), and then make one job for each. This splitting is also good for variance; no longer do you have some sub-jobs that finish in milliseconds and some that require days.

There are some downsides too, of course. In particular, we can no longer pregenerate one universal 8192*8192*8192 bit LUT (well, 8192*8191/2*8192); every sub-job needs to make its own set of LUTs before starting. But since this is O(n³) and we just cut n from 8192 to 512, it's not really a blocker (although of course far from zero); and importantly, it cuts our total RAM usage. For n=8192, we already needed a bit over 32 GB (though sharable between all jobs), and each next element in the sequence (a(13), a(14), etc.) is a factor 8 extra, so it starts becoming a real problem fast. But on the flipside, I think this extra precalc makes the algorithm much less amenable to a theoretical GPU implementation (~8 MB private data per instance, as opposed to one large shared static pool of constants and then just 1 kB of state per instance), which would otherwise be nontrivial but probably possible (the problem itself is so parallel). Interestingly enough, it's possible to use bitslicing to speed up this precalc, which is a technique I cannot remember when I last used.

All in all, it took only about 114 CPU-days (or, well, thread-days, as hyperthreading now makes sense again) to calculate a(13), which was eminently possible; and many of the optimizations came late in the process, so a rerun would be faster than that. So, could we get to a(14)? Well, maybe. I'm less convinced that it would be impossible than I was with a(13) earlier. :-) But I started looking at it, and it turns out there are literally trillions (possibly more!) of sub-jobs if you want to split deeply enough to get each down into the 512-bit range. And even at ~8 ms per core per job (ignoring the cost of splitting and just looking at the cost of processing the jobs themselves), it just becomes too unwieldy for me, especially since Postgres isn't really that great at storing billions of rows efficiently. But impossible? Definitely not.

30 Jul 2025 7:45am GMT

If your bank is like mine, its website doesn't allow you to copy your password and paste it by performing a simple Ctrl+V. I tried the Don't Fuck With Paste extension in Firefox, which could paste my bank account's profile password but not the login password.

Therefore, I asked on Mastodon a couple of days ago and got some responses. The solution that worked for me was to use Shift+Insert to paste the password. It worked for me in LibreWolf and Firefox, and that's all I needed.

Furthermore, this behavior by bank websites leads to users choosing insecure and memorable passwords. Using this trick will help you choose strong passwords for your bank account.

I prefer to use random and strong passwords generated using the password manager pass. It is a freedom-respecting software, unlike popular proprietary password managers promoted by YouTubers. Feel free to check out their webpage here. The reason I use pass is that it stores all the passwords locally (and optionally in a remote Git repository) in encrypted form, which can only be decrypted using your private GPG keys.

29 Jul 2025 7:38am GMT

I was wondering what it would look like if a large language model were part of your programming language. I'm not talking about calling the model as an API, but rather embedding it as a language construct. I came up with this idea as a first cut.

The pseudo macro allows you to embed pseudocode expressions in your Common Lisp code. It takes a string description and uses an LLM to expand it into an s-expression. You can use pseudo anywhere an expression would be expected.

(defun my-func (a b)
  (pseudo "multiply b by factorial of a."))
MY-FUNC

(my-func 5 3)
360

(defun quadratic (a b c)
  (let ((d (sqrt (pseudo "compute discriminant of quadratic equation"))))
    (values (/ (+ (- b) d) (* 2 a)) (/ (- (- b) d) (* 2 a)))))
QUADRATIC

(quadratic 1 2 -3)
1.0
-3.0

The pseudo macro gathers contextual information and packages it up in a big set of system instructions to the LLM. The instructions include

• the lexically visible variables in the macro environment
• fbound symbols
• bound symbols
• overall directives to influence code generation
• directives to influence the style of the generated code (functional vs. imperative)
• directives to influence the use of the loop macro (prefer vs. avoid)
• the source code of the file currently being compiled, if there is one

pseduo sets the LLM to use a low temperature for more predictable generation. It prints the "thinking" of the LLM.

Lisp is a big win here. Since Lisp's macro system operates at the level of s-expressions, it has more contextual information available to it than a macro system that is just text expansion. The s-expression representation means that we don't need to interface with the language's parser or compiler to operate on the syntax tree of the code. Adding pseudo to a language like Java would be a much more significant undertaking.

pseudo has the usual LLM caveats:

• The LLM is slow.
• The LLM can be expensive.
• The LLM can produce unpredictable and unwanted code.
• The LLM can produce incorrect code; the more precise you are in your pseudocode, the more likely you are to get the results you want.
• You would be absolutely mad to use this in production.

pseudo has one dependency on SBCL which is a function to extract the lexically visible variables from the macro environment. If you port it to another Common Lisp, you'll want to provide an equivalent function.

pseudo was developed using Google's Gemini as the back end, but there's no reason it couldn't be adapted to use other LLMs. To try it out, you'll need the gemini library, available at https://github.com/jrm-code-project/gemini, and a Google API key.

Download pseudo from https://github.com/jrm-code-project/pseudo.

You'll also need these dependencies.

• alexandria - available from Quicklisp
• cl-json - available from Quicklisp
• dexador - available from Quicklisp
• fold - https://github.com/jrm-code-project/fold
• function - https://github.com/jrm-code-project/function
• named-let - https://github.com/jrm-code-project/named-let
• uiop - available from Quicklisp

If you try it, let me know how it goes.

28 Jul 2025 9:41am GMT

Treasure hunters, we have an update! Unfortunately, some of our signs have been removed or stolen, but don't worry-the hunt is still on! To ensure everyone can continue, we will be posting all signs online so you can still access the riddles and keep progressing. However, there is one exception: the 4th riddle must still be heard in person at Building H, as it includes an important radio message. Keep your eyes on our updates, stay determined, and don't let a few missing signs stop you from cracking the code! Good luck, and see you at Infodesk K with舰

31 Jan 2025 11:00pm GMT

Are you ready for a challenge? We're hosting a treasure hunt at FOSDEM, where participants must solve six sequential riddles to uncover the final answer. Teamwork is allowed and encouraged, so gather your friends and put your problem-solving skills to the test! The six riddles are set up across different locations on campus. Your task is to find the correct locations, solve the riddles, and progress to the next step. No additional instructions will be given after this announcement, it's up to you to navigate and decipher the clues! To keep things fair, no hints or tips will be given舰

29 Jan 2025 11:00pm GMT

The regular FOSDEM lightning talk track isn't chaotic enough, so this year we're introducing Lightning Lightning Talks (now with added lightning!). Update: we've had a lot of proposals, so submissions are now closed! Thought of a last minute topic you want to share? Got your interesting talk rejected? Has something exciting happened in the last few weeks you want to talk about? Get that talk submitted to Lightning Lightning Talks! This is an experimental session taking place on Sunday afternoon (13:00 in k1105), containing non-stop lightning fast 5 minute talks. Submitted talks will be automatically presented by our Lightning舰

26 Jan 2025 11:00pm GMT