21 May 2018

feedPlanet Debian

Steve Kemp: This month has been mostly golang-based

This month has mostly been about golang. I've continued work on the protocol-tester that I recently introduced:

This has turned into a fun project, and now all my monitoring done with it. I've simplified the operation, such that everything uses Redis for storage, and there are now new protocol-testers for finger, nntp, and more.

Sample tests are as basic as this:

  mail.steve.org.uk must run smtp
  mail.steve.org.uk must run smtp with port 587
  mail.steve.org.uk must run imaps
  https://webmail.steve.org.uk/ must run http with content 'Prayer Webmail service'

Results are stored in a redis-queue, where they can picked off and announced to humans via a small daemon. In my case alerts are routed to a central host, via HTTP-POSTS, and eventually reach me via the pushover

Beyond the basic network testing though I've also reworked a bunch of code - so the markdown sharing site is now golang powered, rather than running on the previous perl-based code.

As a result of this rewrite, and a little more care, I now score 99/100 + 100/100 on Google's pagespeed testing service. A few more of my sites do the same now, thanks to inline-CSS, inline-JS, etc. Nothing I couldn't have done before, but this was a good moment to attack it.

Finally my "silly" Linux security module, for letting user-space decide if binaries should be executed, can-exec has been forward-ported to v4.16.17. No significant changes.

Over the coming weeks I'll be trying to move more stuff into the cloud, rather than self-hosting. I'm doing a lot of trial-and-error at the moment with Lamdas, containers, and dynamic-routing to that end.

Interesting times.

21 May 2018 7:00am GMT

20 May 2018

feedPlanet Debian

Andrej Shadura: Porting inputplug to XCB

5 years ago I wrote inputplug, a tiny daemon which connects to your X server and monitors its input devices, running an external command each time a device is connected or disconnected.

I have used a custom keyboard layout and a fairly non-standard settings for my pointing devices since 2012. I always annoyed me those settings would be re-set every time the device was disconnected and reconnected again, for example, when the laptop was brought back up from the suspend mode. I usually solved that by putting commands to reconfigure my input settings into the resume hook scripts, but that obviously didn't solve the case of connecting external keyboards and mice. At some point those hook scripts stopped to work because they would run too early when the keyboard and mice were not they yet, so I decided to write inputplug.

Inputplug was the first program I ever wrote which used X at a low level, and I had to use Xlib to access the low-level features I needed. More specifically, inputplug uses XInput X extension and listens to XIHierarchyChanged events. In June 2014, Vincent Bernat contributed a patch to rely on XInput2 only.

During the MiniDebCamp, I had a typical case of yak shaving despite not having any yaks around: I wanted to migrate inputplug's packaging from Alioth to Salsa, and I had an idea to update the package itself as well. I had an idea of adding optional systemd user session integration, and the easiest way to do that would be to have inputplug register a D-Bus service. However, if I just registered the service, introspecting it would cause annoying delays since it wouldn't respond to any of the messages the clients would send to it. Handling messages would require me to integrate polling into the event loop, and it turned out it's not easy to do while sticking to Xlib, so I decided to try and port inputplug to XCB.

For those unfamiliar with XCB, here's a bit of background: XCB is a library which implements the X11 protocol and operates on a slightly lower level than Xlib. Unlike Xlib, it only works with structures which map directly to the wire protocol. The functions XCB provides are really atomic: in Xlib, it not unusual for a function to perform multiple X transactions or to juggle the elements of the structures a bit. In XCB, most of the functions are relatively thin wrappers to enable packing and unpacking of the data. Let me give you an example.

In Xlib, if you wanted to check whether the X server supports a specific extension, you would write something like this:

XQueryExtension(display, "XInputExtension", &xi_opcode, &event, &error)

Internally, XQueryExtension would send a QueryExtension request to the X server, wait for a reply, parse the reply and return the major opcode, the first event code and the first error code.

With XCB, you need to separately send the request, receive the reply and fetch the data you need from the structure you get:

const char ext[] = "XInputExtension";

xcb_query_extension_cookie_t qe_cookie;
qe_cookie = xcb_query_extension(conn, strlen(ext), ext);

xcb_query_extension_reply_t *rep;
rep = xcb_query_extension_reply(conn, qe_cookie, NULL);

At this point, rep has its field preset set to true if the extension is present. The rest of the things are in the structure as well, which you have to free yourself after the use.

Things get a bit more tricky with requests returning arrays, like XIQueryDevice. Since the xcb_input_xi_query_device_reply_t structure is difficult to parse manually, XCB provides an iterator, xcb_input_xi_device_info_iterator_t which you can use to iterate over the structure: xcb_input_xi_device_info_next does the necessary parsing and moves the pointer so that each time it is run the iterator points to the next element.

Since replies in the X protocol can have variable-length elements, e.g. device names, XCB also provides wrappers to make accessing them easier, like xcb_input_xi_device_info_name.

Most of the code of XCB is generated: there is an XML description of the X protocol which is used in the build process, and the C code to parse and generate the X protocol packets is generated each time the library is built. This means, unfortunately, that the documentation is quite useless, and there aren't many examples online, especially if you're going to use rarely used functions like XInput hierarchy change events.

I decided to do the porting the hard way, changing Xlib calls to XCB calls one by one, but there's an easier way: since Xlib is now actually based on XCB, you can #include <X11/Xlib-xcb.h> and use XGetXCBConnection to get an XCB connection object corresponding to the Xlib's Display object. Doing that means there will still be a single X connection, and you will be able to mix Xlib and XCB calls.

When porting, it often is useful to have a look at the sources of Xlib: it becomes obvious what XCB functions to use when you know what Xlib does internally (thanks to Mike Gabriel for pointing this out!).

Another thing to remember is that the constants and enums Xlib and XCB define usually have the same values (mandated by the X protocol) despite having slightly different names, so you can mix them too. For example, since inputplug passes the XInput event names to the command it runs, I decided to keep the names as Xlib defines them, and since I'm creating the corresponding strings by using a C preprocessor macro, it was easier for me to keep using XInput2.h instead of defining those strings by hand.

If you're interested in the result of this porting effort, have a look at the code in the Mercurial repo. Unfortunately, it cannot be packaged for Debian yet since the Debian package for XCB doesn't ship the module for XInput (see bug #733227).

P.S. Thanks again to Mike Gabriel for providing me important help - and explaining where to look for more of it ;)

20 May 2018 7:50pm GMT

Sune Vuorela: Where KDEInstallDirs points to

The other day, some user of Extra CMake Modules (A collection of utilities and find modules created by KDE), asked if there was an easy way to query cmake for wherever the KDEInstallDirs points to (KDEInstallDirs is a set of default paths that mostly is good for your system, iirc based upon GNUInstallDirs but with some extensions for various Qt, KDE and XDG common paths, as well as some cross platform additions). I couldn't find an easy way of doing it without writing a couple of lines of CMake code.

Getting the KDE_INSTALL_(full_)APPDIR with default options is:

$ cmake -DTYPE=APPDIR ..
KDE_INSTALL_FULL_APPDIR:/usr/local/share/applications

and various other options can be set as well.

$ cmake -DCMAKE_INSTALL_PREFIX=/opt/mystuff -DTYPE=BINDIR ..
KDE_INSTALL_FULL_BINDIR: /opt/mystuff/bin

This is kind of simple, but let's just share it with the world:

cmake_minimum_required(VERSION 3.0)
find_package(ECM REQUIRED)
set (CMAKE_MODULE_PATH ${ECM_MODULE_PATH})

include(KDEInstallDirs)

message("KDE_INSTALL_FULL_${TYPE}: " ${KDE_INSTALL_FULL_${TYPE}})

I don't think it is complex enough to claim any sorts of copyrights, but if you insist, you can use it under one of the following licenses: CC0, Public Domain (if that's in your juristiction), MIT/X11, WTFPL (any version), 3-clause BSD, GPL (any version), LGPL (any version) and .. erm. whatever.

I was trying to get it to work as a cmake -P script, but some of the find_package calls requires working CMakeCache. Comments welcome.

20 May 2018 5:28pm GMT

feedPlanet Grep

Joost Damad: madparts-rs 1.0 release

History

Back in Februari 2013 then coworker Romain S. showed me the new trend of programming editors that do continuous compilation while you type, showing you immediate feedback on your code. In parallel I also worked on 3D modeling for my 3D printer using the OpenSCAD program. OpenSCAD works by writing code in its custom language and then have it rendered.

An example:

openSCAD image

I had this idea of combining these two approaches to make an electronics footprint generator. And so the development of the original madparts program started.

To go quick I decided to write the program in python, but I wanted a compact language for the footprints so there I decided to use coffeescript, a functional javascript like language that compiles to javascript. Also I used openGL shading language for most of the drawing, because I found it interesting and hadn't touched it before.

Several changes were made among the way, and end of March 2013 a first release was made. It supported both Kicad and Eagle and linux, mac and windows.

After this a command line version of the program was added, and Debian packaging and 1.1 was released in May 2013.

In August 2013 1.2 was released which added support for the then brand new symbolic expressions based file format in Kicad.

In August 2015 the 2.0 release was done with mostly bugfixes and an update to the file format, but it also completely removed the whole file management system which existed, simplifying the program to just work on one file.

At this point the program pretty much did all I needed so further development stalled except for some minor bugfixes.

madparts image

Rust rewrite

In August 2016 I had been playing with the then pretty new rust programming language and decided a rewrite in it and simplifying the program even further would be fun to do.

The following were my initial goals:

Some brief searching showed that Qt (which I used in the python version) support was limited at best but GTK+ and cairo support seemed quite good with reasonable APIs (gtk-rs), and python support seemed to be ok with PyO3.

Practical

file monitoring

By monitoring the file being edited by an external editor via inotify the program can see when a change is saved and render the file. There is a practical caveat there though: most editors actually write the content to a temporary file and move that file over the old file on save (to avoid loss on case of system crash or power loss). This means in practice instead of monitoring the file, you have to monitor the containing directory.

    let _file_watch = ino.add_watch(
        &filedir,
        WatchMask::CREATE | WatchMask::MOVED_TO | WatchMask::CLOSE_WRITE,
    ).unwrap();

python interfacing

Using PyO3 running a python interpreter inside of rust is pretty straightforward. The biggest issue I ran into was dealing with the error situations:

While it is possible to get errors out of python into rust this is tedious and verbose. After some testing I came with a simple solution: do it all in python, providing python the filename to process, and when it fails, capture this in python as well and convert it in a simple Error object meaning from the rust perspective the python code always succeeds, it just has to check for this error object and display then contained message it when it is there instead of drawing the footprint.

def handle_load_python(filename):
    try:
        exec(open(filename).read(), globals(), globals())
        return flatten(footprint())
    except:
        import sys, traceback
        exc_type, exc_value, exc_traceback = sys.exc_info()
        message = "".join(traceback.format_exception(exc_type, exc_value, exc_traceback))
        e = PythonError(message)
        return [e]

rendering

Using Cairo rendering is pretty straightforward as well. A few traits on the Element type make it work:

trait BoundingBox {
    fn bounding_box(&self) -> Bound;
}

pub trait DrawElement {
    fn draw_element(&self, &cairo::Context, layer: Layer);
}


BoundingBox calculates the Bound of an element. By knowing all the bounds and combining them the program can automatically scale the drawing canvas correctly.

DrawElement allows an element to draws on a certain Layer. This is called Layer by layer for each element to have proper z-axis stacking of the drawings.

openSCAD image

KLC

KLC is supported by just executing the python KLC tool check_kicad_mod.py. The result is displayed in the KLC tab.

openSCAD image

exporting

Exporting just saves the footprint as a .kicad_mod file for usage in Kicad.

Release

This is the release 1.0 and the first public release. This means the program works, but is still far from feature complete. I'm adding more features as I need them for footprints. Documentation is not available yet, but I suggest looking at the examples in the footprint/ subdirectory or look in src/prelude.py directly to see what is supported by the python format.

For now the program is only tested in Linux but it should also run with perhaps minimal changes on OSX or Windows. I'm always happy to get a pull request for that on github.

Further plans

More features will be added as needed. Other things planned:

(this blog post was originally posted at rustit.be)

20 May 2018 7:43am GMT

19 May 2018

feedPlanet Grep

Xavier Mertens: [SANS ISC] Malicious Powershell Targeting UK Bank Customers

I published the following diary on isc.sans.org: "Malicious Powershell Targeting UK Bank Customers":

I found a very interesting sample thanks to my hunting rules… It is a PowerShell script that was uploaded on VT for the first time on the 16th of May from UK. The current VT score is still 0/59. The upload location is interesting because the script targets major UK bank customers as we will see below… [Read more]

[The post [SANS ISC] Malicious Powershell Targeting UK Bank Customers has been first published on /dev/random]

19 May 2018 11:45am GMT

17 May 2018

feedPlanet Grep

Dries Buytaert: Working toward a JavaScript-driven Drupal administration interface

As web applications have evolved from static pages to application-like experiences, end-users' expectations of websites have become increasingly demanding. JavaScript, partnered with effective user-experience design, enable the seamless, instantaneous interactions that users now expect.

The Drupal project anticipated this trend years ago and we have been investing heavily in making Drupal API-first ever since. As a result, more organizations are building decoupled applications served by Drupal. This approach allows organizations to use modern JavaScript frameworks, while still benefiting from Drupal's powerful content management capabilities, such as content modeling, content editing, content workflows, access rights and more.

While organizations use JavaScript frameworks to create visitor-facing experiences with Drupal as a backend, Drupal's own administration interface has not yet embraced a modern JavaScript framework. There is high demand for Drupal to provide a cutting-edge experience for its own users: the site's content creators and administrators.

At DrupalCon Vienna, we decided to start working on an alternative Drupal administrative UI using React. Sally Young, one of the initiative coordinators, recently posted a fantastic update on our progress since DrupalCon Vienna.

Next steps for Drupal's API-first and JavaScript work

While we made great progress improving Drupal's web services support and improving our JavaScript support, I wanted to use this blog post to compile an overview of some of our most important next steps:

1. Stabilize the JSON API module

JSON API is a widely-used specification for building web service APIs in JSON. We are working towards adding JSON API to Drupal core as it makes it easier for JavaScript developers to access the content and configuration managed in Drupal. There is a central plan issue that lists all of the blockers for getting JSON API into core (comprehensive test coverage, specification compliance, and more). We're working hard to get all of them out of the way!

2. Improve our JavaScript testing infrastructure

Drupal's testing infrastructure is excellent for testing PHP code, but until now, it was not optimized for testing JavaScript code. As we expect the amount of JavaScript code in Drupal's administrative interface to dramatically increase in the years to come, we have been working on improving our JavaScript testing infrastructure using Headless Chrome and Nightwatch.js. Nightwatch.js has already been committed for inclusion in Drupal 8.6, however some additional work remains to create a robust JavaScript-to-Drupal bridge. Completing this work is essential to ensure we do not introduce regressions, as we proceed with the other items in our roadmap.

3. Create designs for a React-based administration UI

Having a JavaScript-based UI also allows us to rethink how we can improve Drupal's administration experience. For example, Drupal's current content modeling UI requires a lot of clicking, saving and reloading. By using React, we can reimagine our user experience to be more application-like, intuitive and faster to use. We still need a lot of help to design and test different parts of the Drupal administration UI.

4. Allow contributed modules to use React or Twig

We want to enable modules to provide either a React-powered administration UI or a traditional Twig-based administration UI. We are working on an architecture that can support both at the same time. This will allow us to introduce JavaScript-based UIs incrementally instead of enforcing a massive paradigm shift all at once. It will also provide some level of optionality for modules that want to opt-out from supporting the new administration UI.

5. Implement missing web service APIs

While we have been working for years to add web service APIs to many parts of Drupal, not all of Drupal has web services support yet. For our React-based administration UI prototype we decided to implement a new permission screen (i.e. https://example.com/admin/people/permissions). We learned that Drupal lacked the necessary web service APIs to retrieve a list of all available permissions in the system. This led us to create a support module that provides such an API. This support module is a temporary solution that helped us make progress on our prototype; the goal is to integrate these APIs into core itself. If you want to contribute to Drupal, creating web service APIs for various Drupal subsystems might be a great way to get involved.

6. Make the React UI extensible and configurable

One of the benefits of Drupal's current administration UI is that it can be configured (e.g. you can modify the content listing because it has been built using the Views module) and extended by contributed modules (e.g. the Address module adds a UI that is optimized for editing address information). We want to make sure that in the new React UI we keep enough flexibility for site builders to customize the administrative UI.

All decoupled builds benefit

All decoupled applications will benefit from the six steps above; they're important for building a fully-decoupled administration UI, and for building visitor-facing decoupled applications.

Useful for decoupling of visitor-facing front-ends Useful for decoupling of the administration backend
1. Stabilize the JSON API module
2. Improve our JavaScript testing infrastructure
3. Create designs for a React-based administration UI
4. Allow contributed modules to use React or Twig
5. Implement missing web service APIs
6. Make the React UI extensible and configurable

Conclusion

Over the past three years we've been making steady progress to move Drupal to a more API-first and JavaScript centric world. It's important work given a variety of market trends in our industry. While we have made excellent progress, there are more challenges to be solved. We hope you like our next steps, and we welcome you to get involved with them. Thank you to everyone who has contributed so far!

Special thanks to Matt Grill and Lauri Eskola for co-authoring this blog post and to Wim Leers, Gabe Sullice, Angela Byron, and Preston So for their feedback during the writing process.

17 May 2018 5:42pm GMT

08 Nov 2011

feedfosdem - Google Blog Search

papupapu39 (papupapu39)'s status on Tuesday, 08-Nov-11 00:28 ...

papupapu39 · http://identi.ca/url/56409795 #fosdem #freeknowledge #usamabinladen · about a day ago from web. Help · About · FAQ · TOS · Privacy · Source · Version · Contact. Identi.ca is a microblogging service brought to you by Status.net. ...

08 Nov 2011 12:28am GMT

05 Nov 2011

feedfosdem - Google Blog Search

Write and Submit your first Linux kernel Patch | HowLinux.Tk ...

FOSDEM (Free and Open Source Development European Meeting) is a European event centered around Free and Open Source software development. It is aimed at developers and all interested in the Free and Open Source news in the world. ...

05 Nov 2011 1:19am GMT

03 Nov 2011

feedfosdem - Google Blog Search

Silicon Valley Linux Users Group – Kernel Walkthrough | Digital Tux

FOSDEM (Free and Open Source Development European Meeting) is a European event centered around Free and Open Source software development. It is aimed at developers and all interested in the Free and Open Source news in the ...

03 Nov 2011 3:45pm GMT

26 Jul 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Update your RSS link

If you see this message in your RSS reader, please correct your RSS link to the following URL: http://fosdem.org/rss.xml.

26 Jul 2008 5:55am GMT

25 Jul 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Archive of FOSDEM 2008

These pages have been archived.
For information about the latest FOSDEM edition please check this url: http://fosdem.org

25 Jul 2008 4:43pm GMT

09 Mar 2008

feedFOSDEM - Free and Open Source Software Developers' European Meeting

Slides and videos online

Two weeks after FOSDEM and we are proud to publish most of the slides and videos from this year's edition.

All of the material from the Lightning Talks has been put online. We are still missing some slides and videos from the Main Tracks but we are working hard on getting those completed too.

We would like to thank our mirrors: HEAnet (IE) and Unixheads (US) for hosting our videos, and NamurLUG for quick recording and encoding.

The videos from the Janson room were live-streamed during the event and are also online on the Linux Magazin site.

We are having some synchronisation issues with Belnet (BE) at the moment. We're working to sort these out.

09 Mar 2008 3:12pm GMT