20 Nov 2019

feedPlanet Debian

Antoine Beaupré: a quick review of file watchers

File watchers. I always forget about those and never use then, but I constantly feel like I need them. So I made this list to stop searching everywhere for those things which are surprisingly hard to find in a search engine.

Generic

Those tools an watch files or trees of files and execute whatever.

direvent

https://www.gnu.org.ua/software/direvent/

entr

http://eradman.com/entrproject/

fswatch

http://emcrisostomo.github.io/fswatch/

gamin

https://people.gnome.org/~veillard/gamin/

watchexec

https://github.com/watchexec/watchexec

watchman

http://facebook.github.io/watchman/

Purpose-specific

grip (markdown)

https://github.com/joeyespo/grip

conttest (unit tests)

https://github.com/eigenhombre/continuous-testing-helper

kfmon (kobo launcher)

https://github.com/NiLuJe/kfmon

tdaemon (unit tests)

https://github.com/brunobord/tdaemon

Framework-specific

Many frameworks automatically reload files when they change. A few examples:

I've probably missed half a million such programs so this might be expanded with time and your comments.

20 Nov 2019 9:04pm GMT

Birger Schacht: Converting ikiwiki to hugo

Sometimes I play around with Tails and on rare occasions I also build a Tails image myself. One thing that makes the build of Tails a bit tedious is that it a also builds the Tails Website, which contains the whole documentation (which is really cool, because that way users have the most up to date documentation on their desktop!). The problem is, that the website takes a looooong time to build- on my Laptop (i7-5600U) it takes around 11 minutes.

I was curious if it was possible to convert the whole website, which is based on ikiwiki, to the hugo static site generator which is known to be pretty fast ("with its amazing speed and flexibility, Hugo makes building websites fun again" as the hugo website puts it ;)). I did some research if there was some tooling to do so- the Hugo website lists some migration tools but nothing for ikiwiki, but I stumbled upon anarcat's conversion notes which has a lot of information and also links to the write up jak did on his conversion. Anarcat also published a python script to convert ikiwiki to hugo which I tried, but there were some important parts missing. As it happened I also had to prepare for an exam at that time and I am a bit of a procrastinator, so I started adapting the script for the Tails wiki and then rewrote some parts and now its just another ikiwiki2hugo script:

What does it do?

For every ikiwiki *.mdwn or *.html file, it creates a Markdown file with hugo frontmatter. The information for the hugo frontmatter comes from parsing the [[!meta directives in the files.

If there is a *.po file with the same name, the script creates a *.XX.md file with the strings translated.

Then the content is converted:

The content of the Markdown files is a copy of the content of the ikiwiki files with the ikiwiki directives replaced by hugo shortcodes. There are a lot of directives out there and I only implemented replacements for some of them. My test ikiwiki instance was the the Tails ikiwiki source, so I mostly implemented directives used there. Merge requests or patches for additional directives are welcome- to add a replacement for a directive, look in the directives folder, all the replacement python modules inherit from the Directive class. The shortcode files are assets/shortcodes folder.

What does it not do?

Probably the more important question ;)

There are a lot of directives whose replacements are not implemented. Most important probably the [[!inline directive with multiple (and negative) arguments and the [[!map directive.

Also, Hugo is more strict regarding Markdown syntax. It does not convert markdown syntax that is embedded in <div>...</div> and other block-level elements.

The result

Disclaimer: This comparison only looks at the build time- there is a lot of functionality of ikiwiki that hugo does not provide, for example the possibility to edit websites through the browser, which is used for the Tails blueprints. Another feature are the traillink directives which i simple replaced by normal markdown links.

Anarcat also provides a nice oneliner to check which directives are used in the ikiwiki repository:

grep -h -r '\[\[!' * | sed 's/\[\[!/\n[[!/g' | grep '\[\[!' | sed 's/ .*//' | sort | uniq -c | sort -n

Before running the script (I've removed listings of directives that occure less than 100 times):

    ...
    ...
    122 [[!wikipedia\n"
    124 [[!debsa2012
    130 [[!debsa2018
    134 [[!tails_roadmap
    152 [[!debsa2014
    188 [[!debsa2015
    204 [[!debsa2017
    219 [[!tails_gitweb_branch
    222 [[!debsa2016
    247 [[!tails_website
    283 [[!mfsa
    604 [[!cve
    655 [[!toggleable
    696 [[!tails_roadmap]]
    775 [[!"
    806 [[!tails_gitweb
   1143 [[!toggle
   1203 [[!wikipedia
   1704 [[!traillink
   2411 [[!pagetemplate
   2434 [[!toc
   2572 [[!tag
   5627 [[!inline
   5747 [[!tails_ticket
   8673 [[!img
  13577 [[!meta

After the conversion :

      1 [[!immagine
      1 [[!parent]]`.
      1 [[!tails_gitweb
      1 [[!tails_ticket
     30 [[!map
     66 [[!inline

(I think the immagine is a typo in a translation; I also found a couple of other typos which are fixed now; the tails_ticket directive is a code-quote from the release process document and the tails_gitweb is the one occurence of this directive which lists the description before the path ;))

Hugo takes nearly two seconds to build the website:

                   | DE  |  EN  | ES  | FA  | FR  | IT  | PT
+------------------+-----+------+-----+-----+-----+-----+-----+
  Pages            | 286 | 1023 | 286 | 286 | 286 | 286 | 286
  Paginator pages  |   0 |    0 |   0 |   0 |   0 |   0 |   0
  Non-page files   |   9 |  400 |   9 |   9 |  14 |   9 |   9
  Static files     |   0 |    0 |   0 |   0 |   0 |   0 |   0
  Processed images |   0 |    0 |   0 |   0 |   0 |   0 |   0
  Aliases          |   1 |    0 |   0 |   0 |   0 |   0 |   0
  Sitemaps         |   2 |    1 |   1 |   1 |   1 |   1 |   1
  Cleaned          |   0 |    0 |   0 |   0 |   0 |   0 |   0

Total in 1711 ms

Finally, a screeshot of the Tails website built with hugo- it looks pretty broken. Most of the syntax that does not work is because its embedded in HTML tags. Apparently hugo thinks about switching to another markdown renderer, so that could make a conversion from systems with a lot of mixed HTML/Markdown code easier.

Screenshot of the Tails website built with hugo

20 Nov 2019 5:30pm GMT

Rhonda D'Vine: Trans Day of Remembrance 2019

Today, November 20th, is Trans Day of Remembrance. It is about remembering victims of hate crimes that aren't amongst us anymore. Last year we learned about at least 331 murdered trans people, the real number is like always higher. Also like always it affects mostly trans women of color who are the target of multiple discriminatory patterns.

What is also a pattern is that Brazil has a fair chunk of those murders. Unfortunately the country since a while is on the top in that statistic, but the election of an right winged outspoken queer hating person as president of the country last year did make those who feel the hate having some sort of legitimacy to it, which makes it obviously harder to survive these days. My thoughts thus are specifically with the people of Brazil who fight for their survival.

Right-winged parties though rise all around the globe spreading hate, and as our Debian Free Software Guidelines say in #5, "No Discrimination Against Persons or Groups", and this is something that we can't limit only to software licenses but also have to extend to the way we work as community.

If you ask what you can do: Support your local community spaces and support groups. I had the pleasure to meet Grupo Dignidade during my stay in Curitiba for DebConf 19, and was very thankful for a representative of that group to join my Debian Diversity BoF. Thanks again, Ananda, it was lovely having you!

Meu Corpo é Político - my body is political.

/personal | permanent link | Comments: 0 | Flattr this

20 Nov 2019 1:33pm GMT