04 Jul 2025
Planet Debian
Sahil Dhiman: Secondary Authoritative Name Server Options for Self-Hosted Domains
In the past few months, I have moved authoritative name servers (NS) of two of my domains (sahilister.net and sahil.rocks) in house using PowerDNS. Subdomains of sahilister.net see roughly 320,000 hits/day across my IN and DE mirror nodes, so adding secondary name servers with good availability (in addition to my own) servers was one of my first priorities.
I explored the following options for my secondary NS, which also didn't cost me anything:
1984 Hosting
- 1984 Hosting Company FreeDNS.
- Hosting provider from Iceland.
- AXFR over IPv4 only.
- Following secondaries are offered:
- Not all of NS support IPv6.
- Personally, I use ns1.1984.is which is hosted by Netnod, one of 13 root name servers and .SE ccTLD operator.
- Same infrastructure serves 1984.hosting as well.
Hurriance Electric
- Hurricane Electric Free DNS Hosting.
- One has to delegate NS towards one or more of ns[1-5]he.net to verify ownership. It does lead to a minor lame server period between NS addition and first zone transfer.
- Supports TSIG and DNSSEC pre-signed zones.
- Following secondaries are offered:
- The service went down when he.net domain was put on hold. NANOG thread and Hurricane Electric's response there. Better not depend on just one external provider.
- Same infrastructure serves he.net as well.
Afraid.org
- FreeDNS at Afraid.org.
- Backup DNS option on left side menu on their website.
- Following secondary offered:
Puck
- PUCK Free Secondary DNS service.
- One person show, been long-standing though there seems to be manual approval of each account, which did take some time.
- Following secondary offered:
NS-Global
- NS-Global DNS Service.
- From FAQ, anycast with 16 POP, including 1 POP in Tokyo.
- Kenneth Finnegan's blog post carries how this came to be. Same person who also pulled off the Fremont Cabal Internet Exchange and MicroMirror CDN project.
- Following secondary is offered:
- ns-global.kjsl.com uses Afraid.org, Puck and their NS for their own zone.
Asking friends
Two of my friends and fellow mirror hosts have their own authoritative name server setup, Shrirang (ie albony) and Luke. Shirang gave me another POP in IN and through Luke (who does have an insane amount of in-house NS, see dig ns jing.rocks +short
), I added a JP POP.
If we know each other, I would be glad to host a secondary NS for you in (IN and/or DE locations).
Some notes
-
Adding a third-party secondary is putting trust that the third party would serve your zone right.
-
Hurricane Electric and 1984 hosting provide multiple NS. One can use some or all of them. Ideally, you can get away with just using your own with full set from any of these two. Play around with adding and removing secondaries, which gives you the best results. . Using everyone is anyhow overkill, unless you have specific reasons for it.
-
Moving NS in-house isn't that hard. Though, be prepared to get it wrong a few times (and some more). I have already faced partial outages because:
- Recursive resolvers (RR) in the wild behave in a weird way and cache the wrong NS response for longer time than in TTL.
- NS expiry took more than time. 2 out of 3 of my Netim's NS (my domain registrar) had stopped serving my domain, while RRs in the wild hadn't picked up my new in-house NS. I couldn't really do anything about it, though.
- Dot is pretty important at the end.
- With HE.net, I forgot to delegate my domain on their panel and just added in my NS set, thinking I've already done so (which I did but for another domain), leading to a lame server situation.
-
In terms of serving traffic, there's no distinction between primary and secondary NS. RR don't really care who they're asking the query to. So one can have hidden primary too.
-
I initially thought of adding periodic RIPE Atlas measurements from the global set but thought against it as I already host a termux mirror, which brings in thousands of queries from around the world leading to a diverse set of RRs querying my domain already.
-
In most cases, query resolution time would increase with out of zone NS servers (which most likely would be in external secondary). 1 query vs. 2 queries. Pay close attention to ADDITIONAL SECTION Shrirang's case followed by mine:
$ dig ns albony.in
; <<>> DiG 9.18.36 <<>> ns albony.in
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60525
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 9
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;albony.in. IN NS
;; ANSWER SECTION:
albony.in. 1049 IN NS ns3.albony.in.
albony.in. 1049 IN NS ns4.albony.in.
albony.in. 1049 IN NS ns2.albony.in.
albony.in. 1049 IN NS ns1.albony.in.
;; ADDITIONAL SECTION:
ns3.albony.in. 1049 IN AAAA 2a14:3f87:f002:7::a
ns1.albony.in. 1049 IN A 82.180.145.196
ns2.albony.in. 1049 IN AAAA 2403:44c0:1:4::2
ns4.albony.in. 1049 IN A 45.64.190.62
ns2.albony.in. 1049 IN A 103.77.111.150
ns1.albony.in. 1049 IN AAAA 2400:d321:2191:8363::1
ns3.albony.in. 1049 IN A 45.90.187.14
ns4.albony.in. 1049 IN AAAA 2402:c4c0:1:10::2
;; Query time: 29 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Fri Jul 04 07:57:01 IST 2025
;; MSG SIZE rcvd: 286
vs mine
$ dig ns sahil.rocks
; <<>> DiG 9.18.36 <<>> ns sahil.rocks
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64497
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;sahil.rocks. IN NS
;; ANSWER SECTION:
sahil.rocks. 6385 IN NS ns5.he.net.
sahil.rocks. 6385 IN NS puck.nether.net.
sahil.rocks. 6385 IN NS colin.sahilister.net.
sahil.rocks. 6385 IN NS marvin.sahilister.net.
sahil.rocks. 6385 IN NS ns2.afraid.org.
sahil.rocks. 6385 IN NS ns4.he.net.
sahil.rocks. 6385 IN NS ns2.albony.in.
sahil.rocks. 6385 IN NS ns3.jing.rocks.
sahil.rocks. 6385 IN NS ns0.1984.is.
sahil.rocks. 6385 IN NS ns1.1984.is.
sahil.rocks. 6385 IN NS ns-global.kjsl.com.
;; Query time: 24 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Fri Jul 04 07:57:20 IST 2025
;; MSG SIZE rcvd: 313
- Theoretically speaking, a small increase/decrease in resolution would occur based on the chosen TLD and the popularity of the TLD in query originators area (already cached vs. fresh recursion).
- One can get away with having only 3 NS (or be like Google and have 4 anycast NS or like Amazon and have 8 or like Verisign and make it 13 :P).
- Nowhere it's written, your NS needs not to be called dns* or ns1, ns2 etc. Get creative with naming NS; be deceptive with the naming :D.
- A good understanding of RR behavior can help engineer a good authoritative NS system.
Further reading
- RFC 2182: Selection and Operation of Secondary DNS Servers is a good read on what to consider while choosing secondaries.
- RFC 1537: Common DNS Operational and Configuration Errors.
- DNS Nameservers by Geoff Huston gives a good overview of how common RRs behave. Another link from the same article is Recursives in the Wild:Engineering Authoritative DNS Servers.
- DNS Nameservers: Service Platforms and Resilience by Geoff Huston.
- Looking at Centrality in the DNS by Geoff Huston.
04 Jul 2025 2:36am GMT
03 Jul 2025
Planet Debian
Russell Coker: The Fuss About “AI”
There are many negative articles about "AI" (which is not about actual Artificial Intelligence also known as "AGI"). Which I think are mostly overblown and often ridiculous.
Resource Usage
Complaints about resource usage are common, training Llama 3.1 could apparently produce as much pollution as "10,000 round trips by car between Los Angeles and New York City". That's not great but when you compare to the actual number of people doing such drives in the US and the number of people taking commercial flights on that route it doesn't seem like such a big deal. Apparently commercial passenger jets cause CO2 emissions per passenger about equal to a car with 2 people. Why is it relevant whether pollution comes from running servers, driving cars, or steel mills? Why not just tax polluters for the damage they do and let the market sort it out? People in the US make a big deal about not being communist, so why not have a capitalist solution, make it more expensive to do undesirable things and let the market sort it out?
ML systems are a less bad use of compute resources than Bitcoin, at least ML systems give some useful results while Bitcoin has nothing good going for it.
The Dot-Com Comparison
People often complain about the apparent impossibility of "AI" companies doing what investors think they will do. But this isn't anything new, that all happened before with the "dot com boom". I'm not the first person to make this comparison, The Daily WTF (a high quality site about IT mistakes) has an interesting article making this comparison [1]. But my conclusions are quite different.
The result of that was a lot of Internet companies going bankrupt, the investors in those companies losing money, and other companies then bought up their assets and made profitable companies. The cheap Internet we now have was built on the hardware from bankrupt companies which was sold for far less than the manufacture price. That allowed it to scale up from modem speeds to ADSL without the users paying enough to cover the purchase of the infrastructure. In the early 2000s I worked for two major Dutch ISPs that went bankrupt (not my fault) and one of them continued operations in the identical manner after having the stock price go to zero (I didn't get to witness what happened with the other one). As far as I'm aware random Dutch citizens and residents didn't suffer from this and employees just got jobs elsewhere.
There are good things being done with ML systems and when companies like OpenAI go bankrupt other companies will buy the hardware and do good things.
NVidia isn't ever going to have the future sales that would justify a market capitalisation of almost 4 Trillion US dollars. This market cap can support paying for new research and purchasing rights to patented technology in a similar way to the high stock price of Google supported buying YouTube, DoubleClick, and Motorola Mobility which are the keys to Google's profits now.
The Real Upsides of ML
Until recently I worked for a company that used ML systems to analyse drivers for signs of fatigue, distraction, or other inappropriate things (smoking which is illegal in China, using a mobile phone, etc). That work was directly aimed at saving human lives with a significant secondary aim of saving wear on vehicles (in the mining industry drowsy drivers damage truck tires and that's a huge business expense).
There are many applications of ML in medical research such as recognising cancer cells in tissue samples.
There are many less important uses for ML systems, such as recognising different types of pastries to correctly bill bakery customers - technology that was apparently repurposed for recognising cancer cells.
The ability to recognise objects in photos is useful. It can be used for people who want to learn about random objects they see and could be used for helping young children learn about their environment. It also has some potential for assistance for visually impaired people, it wouldn't be good for safety critical systems (don't cross a road because a ML system says there are no cars coming) but could be useful for identifying objects (is this a lemon or a lime). The Humane AI pin had some real potential to do good things but there wasn't a suitable business model [2], I think that someone will develop similar technology in a useful way eventually.
Even without trying to do what the Humane AI Pin attempted, there are many ways for ML based systems to assist phone and PC use.
ML systems allow analysing large quantities of data and giving information that may be correct. When used by a human who knows how to recognise good answers this can be an efficient way of solving problems. I personally have solved many computer problems with the help of LLM systems while skipping over many results that were obviously wrong to me. I believe that any expert in any field that is covered in the LLM input data could find some benefits from getting suggestions from an LLM. It won't necessarily allow them to solve problems that they couldn't solve without it but it can provide them with a set of obviously wrong answers mixed in with some useful tips about where to look for the right answers.
Jobs and Politics
I don't think it's reasonable to expect ML systems to make as much impact on society as the industrial revolution, and the agricultural revolutions which took society from more than 90% farm workers to less than 5%. That doesn't mean everything will be fine but it is something that can seem OK after the changes have happened. I'm not saying "apart from the death and destruction everything will be good", the death and destruction are optional. Improvements in manufacturing and farming didn't have to involve poverty and death for many people, improvements to agriculture didn't have to involve overcrowding and death from disease. This was an issue of political decisions that were made.
The Real Problems of ML
Political decisions that are being made now have the aim of making the rich even richer and leaving more people in poverty and in many cases dying due to being unable to afford healthcare. The ML systems that aim to facilitate such things haven't been as successful as evil people have hoped but it will happen and we need appropriate legislation if we aren't going to have revolutions.
There are documented cases of suicide being inspired by Chat GPT systems [4]. There have been people inspired towards murder by ChatGPT systems but AFAIK no-one has actually succeeded in such a crime yet. There are serious issues that need to be addressed with the technology and with legal constraints about how people may use it. It's interesting to consider the possible uses of ChatGPT systems for providing suggestions to a psychologist, maybe ChatGPT systems could be used to alleviate mental health problems.
The cases of LLM systems being used for cheating on assignments etc isn't a real issue. People have been cheating on assignments since organised education was invented.
There is a real problem of ML systems based on biased input data that issue decisions that are the average of the bigotry of the people who provided input. That isn't going to be worse than the current situation of bigoted humans making decisions based on hate and preconceptions but it will be more insidious. It is possible to search for that so for example a bank could test it's mortgage approval ML system by changing one factor at a time (name, gender, age, address, etc) and see if it changes the answer. If it turns out that the ML system is biased on names then the input data could have names removed. If it turns out to be biased about address then there could be weights put in to oppose that.
For a long time there has been excessive trust in computers. Computers aren't magic they just do maths really fast and implement choices based on the work of programmers - who have all the failings of other humans. Excessive trust in a rule based system is less risky than excessive trust in a ML system where no-one really knows why it makes the decisions it makes.
Self driving cars kill people, this is the truth that Tesla stock holders don't want people to know.
Companies that try to automate everything with "AI" are going to be in for some nasty surprises. Getting computers to do everything that humans do in any job is going to be a large portion of an actual intelligent computer which if it is achieved will raise an entirely different set of problems.
I've previously blogged about ML Security [5]. I don't think this will be any worse than all the other computer security problems in the long term, although it will be more insidious.
How Will It Go?
Companies spending billions of dollars without firm plans for how to make money are going to go bankrupt no matter what business they are in. Companies like Google and Microsoft can waste some billions of dollars on AI Chat systems and still keep going as successful businesses. Companies like OpenAI that do nothing other than such chat systems won't go well. But their assets can be used by new companies when sold at less than 10% the purchase price.
Companies like NVidia that have high stock prices based on the supposed ongoing growth in use of their hardware will have their stock prices crash. But the new technology they develop will be used by other people for other purposes. If hospitals can get cheap diagnostic ML systems because of unreasonable investment into "AI" then that could be a win for humanity.
Companies that bet their entire business on AI even when it's not necessarily their core business (as Tesla has done with self driving) will have their stock price crash dramatically at a minimum and have the possibility of bankruptcy. Having Tesla go bankrupt is definitely better than having people try to use them as self driving cars.
- [1] https://thedailywtf.com/articles/ai-the-bad-the-worse-and-the-ugly
- [2] https://etbe.coker.com.au/2024/04/26/humane-ai-pin/
- [3] https://www.noemamag.com/how-ai-could-help-rebuild-the-middle-class/
- [4] https://tinyurl.com/2b95xdzs
- [5] https://etbe.coker.com.au/2025/05/30/machine-learning-security/
03 Jul 2025 10:21am GMT
02 Jul 2025
Planet Debian
Dirk Eddelbuettel: RcppArmadillo 14.6.0-1 on CRAN: New Upstream Minor Release
Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language-and is widely used by (currently) 1241 other packages on CRAN, downloaded 40.4 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 634 times according to Google Scholar.
Conrad released a minor version 4.6.0 yesterday which offers new accessors for non-finite values. And despite being in Beautiful British Columbia on vacation, I had wrapped up two rounds of reverse dependency checks preparing his 4.6.0 release, and shipped this to CRAN this morning where it passed with flying colours and no human intervention-even with over 1200 reverse dependencies. The changes since the last CRAN release are summarised below.
Changes in RcppArmadillo version 14.6.0-1 (2025-07-02)
Upgraded to Armadillo release 14.6.0 (Caffe Mocha)
Added
balance()
to transform matrices so that column and row norms are roughly the sameAdded
omit_nan()
andomit_nonfinite()
to extract elements while omitting NaN and non-finite valuesAdded
find_nonnan()
for finding indices of non-NaN elementsAdded standalone
replace()
functionThe
fastLm()
help page now mentions that options tosolve()
can control its behavior.
Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub.
02 Jul 2025 9:21pm GMT
30 Jun 2025
Planet Lisp
Joe Marshall: You Are The Compiler
Consider a complex nested function call like
(foo (bar (baz x)) (quux y))
This is a tree of function calls. The outer call to foo
has two arguments, the result of the inner call to bar
and the result of the inner call to quux
. The inner calls may themselves have nested calls.
One job of the compiler is to linearize this call tree into a sequential series of calls. So the compiler would generate some temporaries to hold the results of the inner calls, make each inner call in turn, and then make the outer call.
temp1 = baz(x) temp2 = bar(temp1) temp3 = quux(y) return foo (temp2, temp3)
Another job of the compiler is to arrange for each call to follow the calling conventions that define where the arguments are placed and where the results are returned. There may be additional tasks done at function call boundaries, for example, the system might insert interrupt checks after each call. These checks are abstracted away at the source code level. The compiler takes care of them automatically.
Sometimes, however, you want to want modify the calling conventions. For example, you might want to write in continuation passing style. Each CPS function will take an additional argument which is the continuation. The compiler won't know about this convention, so it will be incumbent on the programmer to write the code in a particular way.
If possible, a macro can help with this. The macro will ensure that the modified calling convention is followed. This will be less error prone than expecting the programmer to remember to write the code in a particular way.
The Go language has two glaring omissions in the standard calling conventions: no dynamic (thread local) variables and no error handling. Users are expected to impose their own calling conventions of passing an additional context argument between functions and returning error objects upon failures. The programmer is expected to write code at the call site to check the error object and handle the failure.
This is such a common pattern of usage that we can consider it to be the de facto calling convention of the language. Unfortunately, the compiler is unaware of this convention. It is up to the programmer to explicitly write code to assign the possible error object and check its value.
This calling convention breaks nested function calls. The user has to explicitly linearize the calls.
temp1, err1 := baz(ctx, x) if err1 != nil { return nil, err1 } temp2, err2 := bar(ctx, temp1) if err2 != nil { return nil, err2 } temp3, err3 := quux(ctx, y) if err2 != nil { return nil, err2 } result, err4 := foo(ctx, temp2, temp3) if err4 != nil { return nil, err4 } return result, nil
Golang completely drops the ball here. The convention of returning an error object and checking it is ubiquitous in the language, but there is no support for it in the compiler. The user ends up doing what is normally considered the compiler's job of linearizing nested calls and checking for errors. Of course users are less disciplined than the compiler, so unconventional call sequences and forgetting to handle errors are common.
30 Jun 2025 6:15pm GMT
29 Jun 2025
Planet Lisp
Neil Munro: Ningle Tutorial 8: Mounting Middleware
Contents
- Part 1 (Hello World)
- Part 2 (Basic Templates)
- Part 3 (Introduction to middleware and Static File management)
- Part 4 (Forms)
- Part 5 (Environmental Variables)
- Part 6 (Database Connections)
- Part 7 (Envy Configuation Switching)
- Part 8 (Mouning Middleware)
Introduction
Welcome back to this Ningle tutorial series, in this part we are gonna have another look at some middleware, now that we have settings and configuration done there's another piece of middleware we might want to look at; application mounting
, many web frameworks have the means to use apps within other apps, you might want to do this because you have some functionality you use over and over again in many projects, it makes sense to make it into an app and simply include it in other apps. You might also might want to make applications available for others to use in their applications.
Which is exactly what we are gonna do here, we spent some time building a registration view, but for users we might want to have a full registration system that will have:
- Register
- Login
- Logout
- Account Verification
- Account Reset
- Account Deletion
Creating the auth app
We will begin by building the basic views that return a simple template and mount them into our main application, we will then fill the actual logic out in another tutorial. So, we will create a new Ningle project that has 6 views that simply handle get
requests, the important thing to bear in mind is that we will have to adjust the layout of our templates, we need our auth app to use its own templates, or use the templates of a parent app, this means we will have to namespace our templates, if you have use django before this will seem familiar.
Using my project builder set up a new project for our authentication application.
(nmunro:make-project #p"~/quicklisp/local-projects/ningle-auth/")
This will create a project skeleton, complete with an asd
file, a src
, and tests
directory. In the asd
file we need to add some packages (we will add more in a later tutorial).
:depends-on (:cl-dotenv
:clack
:djula
:envy-ningle
:mito
:ningle)
In the src/main.lisp
file, we will add the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
(defpackage ningle-auth (:use :cl) (:export #:*app* #:start #:stop)) (in-package ningle-auth) (defvar *app* (make-instance 'ningle:app)) (djula:add-template-directory (asdf:system-relative-pathname :ningle-auth "src/templates/")) (setf (ningle:route *app* "/register") (lambda (params) (format t "Test: ~A~%" (mito:retrieve-by-sql "SELECT 2 + 3 AS result")) (djula:render-template* "auth/register.html" nil :title "Register"))) (setf (ningle:route *app* "/login") (lambda (params) (djula:render-template* "auth/login.html" nil :title "Login"))) (setf (ningle:route *app* "/logout") (lambda (params) (djula:render-template* "auth/logout.html" nil :title "Logout"))) (setf (ningle:route *app* "/reset") (lambda (params) (djula:render-template* "auth/reset.html" nil :title "Reset"))) (setf (ningle:route *app* "/verify") (lambda (params) (djula:render-template* "auth/verify.html" nil :title "Verify"))) (setf (ningle:route *app* "/delete") (lambda (params) (djula:render-template* "auth/delete.html" nil :title "Delete"))) (defmethod ningle:not-found ((app ningle:<app>)) (declare (ignore app)) (setf (lack.response:response-status ningle:*response*) 404) (djula:render-template* "error.html" nil :title "Error" :error "Not Found")) (defun start (&key (server :woo) (address "127.0.0.1") (port 8000)) (djula:add-template-directory (asdf:system-relative-pathname :ningle-auth "src/templates/")) (djula:set-static-url "/public/") (clack:clackup (lack.builder:builder (envy-ningle:build-middleware :ningle-auth/config *app*)) :server server :address address :port port)) (defun stop (instance) (clack:stop instance)) |
Just as we did with our main application, we will need to create a src/config.lisp
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
(defpackage ningle-auth/config (:use :cl :envy)) (in-package ningle-auth/config) (dotenv:load-env (asdf:system-relative-pathname :ningle-auth ".env")) (setf (config-env-var) "APP_ENV") (defconfig :common `(:application-root ,(asdf:component-pathname (asdf:find-system :ningle-auth)))) (defconfig |test| `(:debug T :middleware ((:session) (:mito (:sqlite3 :database-name ,(uiop:getenv "SQLITE_DB_NAME")))))) |
Now, I mentioned that the template files need to be organised in a certain way, we will start with the new template layout in our auth application, the directory structure should look like this:
➜ ningle-auth git:(main) tree .
.
├── ningle-auth.asd
├── README.md
├── src
│ ├── config.lisp
│ ├── main.lisp
│ └── templates
│ ├── ningle-auth
│ │ ├── delete.html
│ │ ├── login.html
│ │ ├── logout.html
│ │ ├── register.html
│ │ ├── reset.html
│ │ └── verify.html
│ ├── base.html
│ └── error.html
└── tests
└── main.lisp
So in your src/templates
directory there will be a directory called ningle-auth
and two files base.html
and error.html
, it is important that this structure is followed, as when the app is used as part of a larger app, we want to be able to layer templates, and this is how we do it.
base.html
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<!doctype html> <html lang="en"> <head> <title>{{ title }}</title> <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet"> </head> <body> <div class="container mt-4"> {% block content %} {% endblock %} </div> <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script> </body> </html> |
error.html
1 2 3 4 5 6 7 8 9 10 11 |
{% extends "base.html" %} {% block content %} <div class="container"> <div class="row"> <div class="col-12"> <h1>{{ error }}</h1> </div> </div> </div> {% endblock %} |
Now the rest of the html files are similar, with only the title changing. Using the following html, create files for:
delete.html
1 2 3 4 5 6 7 8 9 10 11 |
{% extends "base.html" %} {% block content %} <div class="container"> <div class="row"> <div class="col-12"> <h1>Delete</h1> </div> </div> </div> {% endblock %} |
login.html
1 2 3 4 5 6 7 8 9 10 11 |
{% extends "base.html" %} {% block content %} <div class="container"> <div class="row"> <div class="col-12"> <h1>Login</h1> </div> </div> </div> {% endblock %} |
logout.html
1 2 3 4 5 6 7 8 9 10 11 |
{% extends "base.html" %} {% block content %} <div class="container"> <div class="row"> <div class="col-12"> <h1>Logout</h1> </div> </div> </div> {% endblock %} |
register.html
1 2 3 4 5 6 7 8 9 10 11 |
{% extends "base.html" %} {% block content %} <div class="container"> <div class="row"> <div class="col-12"> <h1>Register</h1> </div> </div> </div> {% endblock %} |
reset.html
1 2 3 4 5 6 7 8 9 10 11 |
{% extends "base.html" %} {% block content %} <div class="container"> <div class="row"> <div class="col-12"> <h1>Reset</h1> </div> </div> </div> {% endblock %} |
verify.html
1 2 3 4 5 6 7 8 9 10 11 |
{% extends "base.html" %} {% block content %} <div class="container"> <div class="row"> <div class="col-12"> <h1>Verify</h1> </div> </div> </div> {% endblock %} |
There is one final file to create, the .env
file! Even though this application wont typically run on its own, we will use one to test it is all working, since we did write src/config.lisp
afterall!
1 2 |
APP_ENV=test SQLITE_DB_NAME=ningle-auth.db |
Testing the auth app
Now that the auth application has been created we will test that it at least runs on its own, once we have confirmed this, we can integrate it into our main app. Like with our main application, we will load the system and run the start function that we defined.
(ql:quickload :ningle-auth)
To load "ningle-auth":
Load 1 ASDF system:
ningle-auth
; Loading "ningle-auth"
..................................................
[package ningle-auth/config].
(:NINGLE-AUTH)
(ningle-auth:start)
NOTICE: Running in debug mode. Debugger will be invoked on errors.
Specify ':debug nil' to turn it off on remote environments.
Woo server is started.
Listening on 127.0.0.1:8000.
#S(CLACK.HANDLER::HANDLER
:SERVER :WOO
:SWANK-PORT NIL
:ACCEPTOR #<BT2:THREAD "clack-handler-woo" {1203E4E3E3}>)
*
If this works correctly, you should be able to access the defined routes in your web browser, if not, and there is an error, check that another web server isn't running on port 8000 first! When you are able to access the simple routes from your web browser, we are ready to integrate this into our main application!
Integrating the auth app
Made it this far? Congratulations, we are almost at the end, I'm sure you'll be glad to know, there isn't all that much more to do, but we do have to ensure we follow the structure we set up in the auth app, which we will get to in just a moment, first, lets remember to add the ningle-auth
app to our dependencies in our project asd
file.
:depends-on (:cl-dotenv
:clack
:djula
:cl-forms
:cl-forms.djula
:cl-forms.ningle
:envy
:envy-ningle
:ingle
:mito
:mito-auth
:ningle
:ningle-auth) ;; add this
Next, we need to move most of our template files into a directory called main
, to make things easy, the only two templates we will not move are base.html
and error.html
; create a new directory src/templates/main
and put everything else in there.
For reference this is what your directory structure should look like:
➜ ningle-tutorial-project git:(main) tree .
.
├── ningle-tutorial-project.asd
├── ntp.db
├── README.md
├── src
│ ├── config.lisp
│ ├── forms.lisp
│ ├── main.lisp
│ ├── migrations.lisp
│ ├── models.lisp
│ ├── static
│ │ ├── css
│ │ │ └── main.css
│ │ └── images
│ │ ├── logo.jpg
│ │ └── lua.jpg
│ └── templates
│ ├── base.html
│ ├── error.html
│ └── main
│ ├── index.html
│ ├── login.html
│ ├── logout.html
│ ├── people.html
│ ├── person.html
│ └── register.html
└── tests
└── main.lisp
With the templates having been moved, we must find all areas in src/main.lisp
where we reference one of these templates and point to the new location, thankfully there's only 4 lines that need to be changed, the render-template* calls, below is what they should be changed to.
(djula:render-template* "main/index.html" nil :title "Home" :user user :posts posts)
(djula:render-template* "main/people.html" nil :title "People" :users users)
(djula:render-template* "main/person.html" nil :title "Person" :user user)
(djula:render-template* "main/register.html" nil :title "Register" :form form)
Here is a complete listing of the file in question.
(defpackage ningle-tutorial-project
(:use :cl :sxql)
(:import-from
:ningle-tutorial-project/forms
#:email
#:username
#:password
#:password-verify
#:register)
(:export #:start
#:stop))
(in-package ningle-tutorial-project)
(defvar *app* (make-instance 'ningle:app))
(setf (ningle:route *app* "/")
(lambda (params)
(let ((user (list :username "NMunro"))
(posts (list (list :author (list :username "Bob") :content "Experimenting with Dylan" :created-at "2025-01-24 @ 13:34")
(list :author (list :username "Jane") :content "Wrote in my diary today" :created-at "2025-01-24 @ 13:23"))))
(djula:render-template* "main/index.html" nil :title "Home" :user user :posts posts))))
(setf (ningle:route *app* "/people")
(lambda (params)
(let ((users (mito:retrieve-dao 'ningle-tutorial-project/models:user)))
(djula:render-template* "main/people.html" nil :title "People" :users users))))
(setf (ningle:route *app* "/people/:person")
(lambda (params)
(let* ((person (ingle:get-param :person params))
(user (first (mito:select-dao
'ningle-tutorial-project/models:user
(where (:or (:= :username person)
(:= :email person)))))))
(djula:render-template* "main/person.html" nil :title "Person" :user user))))
(setf (ningle:route *app* "/register" :method '(:GET :POST))
(lambda (params)
(let ((form (cl-forms:find-form 'register)))
(if (string= "GET" (lack.request:request-method ningle:*request*))
(djula:render-template* "main/register.html" nil :title "Register" :form form)
(handler-case
(progn
(cl-forms:handle-request form) ; Can throw an error if CSRF fails
(multiple-value-bind (valid errors)
(cl-forms:validate-form form)
(when errors
(format t "Errors: ~A~%" errors))
(when valid
(cl-forms:with-form-field-values (email username password password-verify) form
(when (mito:select-dao 'ningle-tutorial-project/models:user
(where (:or (:= :username username)
(:= :email email))))
(error "Either username or email is already registered"))
(when (string/= password password-verify)
(error "Passwords do not match"))
(mito:create-dao 'ningle-tutorial-project/models:user
:email email
:username username
:password password)
(ingle:redirect "/people")))))
(error (err)
(djula:render-template* "error.html" nil :title "Error" :error err))
(simple-error (csrf-error)
(setf (lack.response:response-status ningle:*response*) 403)
(djula:render-template* "error.html" nil :title "Error" :error csrf-error)))))))
(defmethod ningle:not-found ((app ningle:<app>))
(declare (ignore app))
(setf (lack.response:response-status ningle:*response*) 404)
(djula:render-template* "error.html" nil :title "Error" :error "Not Found"))
(defun start (&key (server :woo) (address "127.0.0.1") (port 8000))
(djula:add-template-directory (asdf:system-relative-pathname :ningle-tutorial-project "src/templates/"))
(djula:set-static-url "/public/")
(clack:clackup
(lack.builder:builder (envy-ningle:build-middleware :ningle-tutorial-project/config *app*))
:server server
:address address
:port port))
(defun stop (instance)
(clack:stop instance))
The final step we must complete is actually mounting our ningle-auth
application into our main app, which is thankfully quite easy. Mounting middleware exists for ningle
and so we can configure this in src/config.lisp
, to demonstrate this we will add it to our sqlite
config:
1 2 3 4 5 6 |
(defconfig |sqlite| `(:debug T :middleware ((:session) (:mito (:sqlite3 :database-name ,(uiop:getenv "SQLITE_DB_NAME"))) (:mount "/auth" ,ningle-auth:*app*) ;; This line! (:static :root ,(asdf:system-relative-pathname :ningle-tutorial-project "src/static/") :path "/public/")))) |
You can see on line #5 that a new mount
point is being defined, we are mounting all the routes that ningle-auth
has, onto the /auth
prefix. This means that, for example, the /register
route in ningle-auth
will actually be accessed /auth/register
.
If you can check that you can access all the urls to confirm this works, then we have assurances that we are set up correctly, however we need to come back to the templates one last time.
The reason we changed the directory structure, because ningle-auth is now running in the context of our main app, we can actually override the templates, so if we wanted to, in our src/templates
directory, we could create a ningle-auth
directory and create our own register.html
, login.html
, etc, allowing us to style and develop our pages as we see fit, allowing complete control to override, if that is our wish. By NOT moving the base.html
and error.html
files, we ensure that templates from another app can inherit our styles and layouts in a simple and predictable manner.
Conclusion
Wow, what a ride... Thanks for sticking with it this month, although, next month isn't going to be much easier as we begin to develop a real authentication application for use in our microblog app! As always, I hope you have found this helpful and you have learned something.
In this tutorial you should be able to:
- Explain what mounting an application means
- Describe how routes play a part in mounting an application
- Justify why you might mount an application into another
- Develop and mount an application inside another
Github
- The link for this tutorials code is available here.
- The link for the auth app code is available here.
Resources
29 Jun 2025 11:30am GMT
11 Jun 2025
Planet Lisp
Joe Marshall: No Error Handling For You
According to the official Go blog, there are no plans to fix the (lack of) error handling in Go. Typical. Of course they recognize the problem, and many people have suggested solutions, but no one solution seems to be obviously better than the others, so they are going to do nothing. But although no one solution appears obviously better than the others, it's pretty clear that the status quo is worse than any of the proposed solutions.
But the fundamental problem isn't error handling. The fundamental problem is that the language cannot be extended and modified by the user. Error handling requires a syntactic change to the language, and changes to the language have to go through official channels.
If Go had a macro system, people could write their own error handling system. Different groups could put forward their proposals independently as libraries, and you could choose the error handling library that best suited your needs. No doubt a popular one would eventually become the de facto standard.
But Go doesn't have macros, either. So you are stuck with limitations that are baked into the language. Naturally, there will be plenty of people who will argue that this is a good thing. At least the LLMs will have a lot of training data for if err != nil
.
11 Jun 2025 2:25pm GMT
31 Jan 2025
FOSDEM 2025
FOSDEM Treasure Hunt Update – Signs Stolen, But the Hunt Continues!
Treasure hunters, we have an update! Unfortunately, some of our signs have been removed or stolen, but don't worry-the hunt is still on! To ensure everyone can continue, we will be posting all signs online so you can still access the riddles and keep progressing. However, there is one exception: the 4th riddle must still be heard in person at Building H, as it includes an important radio message. Keep your eyes on our updates, stay determined, and don't let a few missing signs stop you from cracking the code! Good luck, and see you at Infodesk K with舰
31 Jan 2025 11:00pm GMT
29 Jan 2025
FOSDEM 2025
Join the FOSDEM Treasure Hunt!
Are you ready for a challenge? We're hosting a treasure hunt at FOSDEM, where participants must solve six sequential riddles to uncover the final answer. Teamwork is allowed and encouraged, so gather your friends and put your problem-solving skills to the test! The six riddles are set up across different locations on campus. Your task is to find the correct locations, solve the riddles, and progress to the next step. No additional instructions will be given after this announcement, it's up to you to navigate and decipher the clues! To keep things fair, no hints or tips will be given舰
29 Jan 2025 11:00pm GMT
26 Jan 2025
FOSDEM 2025
Introducing Lightning Lightning Talks
The regular FOSDEM lightning talk track isn't chaotic enough, so this year we're introducing Lightning Lightning Talks (now with added lightning!). Update: we've had a lot of proposals, so submissions are now closed! Thought of a last minute topic you want to share? Got your interesting talk rejected? Has something exciting happened in the last few weeks you want to talk about? Get that talk submitted to Lightning Lightning Talks! This is an experimental session taking place on Sunday afternoon (13:00 in k1105), containing non-stop lightning fast 5 minute talks. Submitted talks will be automatically presented by our Lightning舰
26 Jan 2025 11:00pm GMT