fidzu : Fedora

Last year I wrote about some enhancements made to Okular's annotation tool and in one of those, Simone Gaiarin commented that he was working on redesigning the Annotation toolbar altogether. I was quite interested and was also thinking of 'modernizing' the tool - only, I had no idea how much work it would be.

The existing annotation tool works, but it had some quirks and had many advanced options which were documented pretty well in the Handbook but not obvious to an unscrupulous user. For instance, if the user would like to highlight some part of the text, she selects (single-clicks) the highlighter tool, applies it to a block of text. When another part of text is to be highlighted, you'd expect the highlighter tool to apply directly; but it didn't 'stick' - tool was unselected after highlighting the first block of text. There is an easy way to make the annotation tool 'stick' - instead of single-click to select the tool, simply double-click, and it persists. Another instance is the 'Strikeout' annotation which is not displayed by default, but can be added to the tools list.

Simone, with lots of inputs, testing and reviews from David Hurka, Nate Graham and Albert Astals Cid et al., has pulled off a magnificent rewrite of Okular's annotation toolbar. To get an idea of the amount of work went into this, see this phabricator task and this invent code review. The result of many months of hardwork is a truly modern, easy to explore-and-use annotation support. I am not aware of any other libre PDF reader with such good annotation features.

<figure class="wp-block-image size-large"><figcaption>Annotation toolbar in Okular 20.08.</figcaption></figure>

Starting from the left, default tools are: Highlight (brush icon), Underline (straight line) and Squiggle (wobbly line), Strike out, Insert text (Typewriter), Inline note, Popup note, Freehand drawing and Shapes (arrows, lines, rectangles etc.). The line thickness, colour, opacity and font of the tools can be customized easily from the drawer. Oh, and the selected annotation tool 'sticks' by default (see the 'pin' icon at the right end of toolbar).

<figure class="wp-block-gallery columns-2 is-cropped">

• <figure></figure>
• <figure></figure>

<figcaption class="blocks-gallery-caption">Line width and colour of 'Arrow' tool.</figcaption></figure>

When upgrading to okular-20.08 from a previous version, it will preserve the customized annotation tools created by the user and make those available under 'Quick annotations', and these can be quickly applied using Alt+n (Alt-1, Alt-2 etc.) short cuts. It did reset my custom shortcuts keys for navigation (I use Vim keys gg to go to the first page and G to go to the last page), which can be manually added back.

<figure class="wp-block-image size-large is-resized"><figcaption>Custom tools (Quick annotations) can be applied with short cuts.</figcaption></figure>

Here is the new toolbar in action.

<figure class="wp-block-embed is-type-wp-embed is-provider-screencast wp-block-embed-screencast wp-embed-aspect-4-3 wp-has-aspect-ratio">

</figure>

20 Sep 2020 8:27am GMT

Release Candidate versions are available in testing repository for Fedora and Enterprise Linux (RHEL / CentOS) to allow more people to test them. They are available as Software Collections, for a parallel installation, perfect solution for such tests, and also as base packages.

RPM of PHP version 7.4.11RC1 are available as SCL in remi-test repository and as base packages in the remi-test repository for Fedora 32-33 or remi-php74-test repository for Fedora 31 and Enterprise Linux 7-8.

RPM of PHP version 7.3.23RC1 are available as SCL in remi-test repository and as base packages in the remi-test repository for Fedora 31 or remi-php73-test repository for Enterprise Linux.

PHP version 7.2 is now in security mode only, so no more RC will be released.

Installation : read the Repository configuration and choose your version.

Parallel installation of version 7.4 as Software Collection:

yum --enablerepo=remi-test install php74

Parallel installation of version 7.3 as Software Collection:

yum --enablerepo=remi-test install php73

Update of system version 7.4:

yum --enablerepo=remi-php74,remi-php74-test update php\*

or, the modular way (Fedora and RHEL 8):

dnf module reset php
dnf module enable php:remi-7.4
dnf --enablerepo=remi-modular-test update php\*

Update of system version 7.3:

yum --enablerepo=remi-php73,remi-php73-test update php\*

or, the modular way (Fedora and RHEL 8):

dnf module reset php
dnf module enable php:remi-7.3
dnf --enablerepo=remi-modular-test update php\*

Notice: version 7.4.11RC1 is also in Fedora rawhide for QA.

x86_64 builds now use Oracle Client version 19.8

EL-8 packages are built using RHEL-8.2

EL-7 packages are built using RHEL-7.8

RC version is usually the same as the final version (no change accepted after RC, exception for security fix).

Version 8.0.0beta4 is also available as Software Collections

Software Collections (php73, php74)

Base packages (php)

18 Sep 2020 8:34am GMT

The post A reflection: Gabriele Trombini (mailga) appeared first on Justin W. Flory's blog.

Justin W. Flory's blog - Free Software, music, travel, and life reflections

Trigger warning: Grief, death.

Two years passed since we last met in Bolzano. I remember you traveled in for a day to join the 2018 Fedora Mindshare FAD. You came many hours from your home to see us, and share your experiences and wisdom from both the global and Italian Fedora Community. And this week, I learned that you, Gabriele Trombini, passed away from a heart attack. To act like the news didn't affect me denies my humanity. In 2020, a year that feels like it has taken away so much already, we are greeted by another heart-breaking loss.

But to succumb to the despair and sadness of this year would deny the warm, happy memories we shared together. We shared goals of supporting the Fedora Project but also learning from each other.

So, this post is a brief reflection of your life as I knew you. A final celebration of the great memories we shared together, that I only wish I could have shared with you while you were still here.

Ciao!

We had a unique privilege of meeting first in person before meeting online. At Flock 2015, of course I remember coming to your Fedora-Join session. This was my first introduction to the volunteer-supported mentorship community that exists in Fedora. Even though there was one particularly disruptive audience member, I remember learning from you and noting your long-time experience in the Fedora Community.

After that, we would come to know each other better. As I began a new chapter of my life at my university, we would become frequent collaborators. The Fedora Marketing team was always interesting to me, as part of the group of people who helped our community talk about and share the Fedora Project with others. Underneath your gentle mentorship, I learned the focus areas and history of the Fedora Marketing team.

At some point in 2015 or 2016, you asked me if I would like to chair a Marketing Team meeting. Thus began an early step in my journey from a participant to a facilitator. In a tragically ironic way, it strikes me how I did not see your guidance as mentorship at the time. I always saw our conversations as two friends discussing a shared hobby or interest. Such is the subtle art of teaching and mentorship.

Your many contributions

You were a cornerstone community member of Fedora for many years. Since our connection was from Fedora, it is worth noting the many contributions you made over the years. Long before Fedora or Linux were anything I knew about.

You and Robert Mayr co-authored a book together about Fedora 9, I think for the Italian Linux community. You were a one-time steward of the Fedora Join and Marketing teams. You were an influential member in shaping what Mindshare is today, from the days of the Fedora Outreach Steering Committee, the Fedora Ambassador Steering Committee before that, and grassroots community organizing in Italy even before that.

Beyond the source

But perhaps the memories I treasure most are the ones that don't have much to do with Fedora at all. I remember learning that "in real life" you were a co-owner of a heating and air conditioning business in Italy. For many years, my family ran a heating and air conditioning company of our own. This was an experience I could always understand. I remember the times when you would go offline for some time. Then I would hear from you eventually, and you would tell me how the busy season kept you away from helping out in Fedora. And in a few words in IRC private messages, I simply knew and smiled.

We would meet at Flock events, but I find Flock is usually tough to get 1×1 time with others. I remember the day you came up and joined us in Bolzano for the 2018 Mindshare FAD. On a weekend day in March, you came and sat in a wine cellar converted to a conference room, where we spent the day recounting pain points and how Mindshare would address them.

And then, our small group went out for dinner. The food we ate and words we said are now faded memories, but the experience lives warmly in my heart as I think about what your life meant to me.

I was saddened to find no photographs or pictures of us together. But I went looking for our last conversations and found these final messages on IRC:

**** BEGIN LOGGING AT Sun Dec  4 17:49:56 2016

Dec 04 17:49:56 <jflory7>   That would be fantastic... I'll definitely let you know if I have plans to visit Italy. :)

Dec 05 07:00:32 <mailga>    jflory7 hope it happens. :)

**** ENDING LOGGING AT Wed Dec  7 00:28:51 2016

I never got to take you up on your offer to visit your home and meet your family. But I am happy that I had the opportunity to partially fulfill that old promise of meeting together in Italy.

Why write this?

I didn't write this post with an outline, or a template. These words came to me while sitting with my own emotions and feelings. I am writing this because this is an effective coping mechanism for me to process what is lost, but also how to move forward from the loss.

The Fedora Project has given me a lot over the last five years. I have met many wonderful people and contributed to things that matter a great deal to me. But Fedora has also taught me about loss. There are many lessons in life that have nothing to do with work, code, software, or engineering, but have everything to do with how we look at the world.

In the wake of losing you, I think of the kind words and memories we shared that I did not tell you were important to me. I think of how the opportunity is permanently missed for me to share my appreciation of your kindness and friendship. The tragedy of youth is perhaps that I failed to fully appreciate our connection until after you passed.

When writing this, I came to realize something for me. And this will be different for everyone. But I like to think for Gabrielle and me, Fedora was never just about building an operating system. It was about collaborating with other people, human beings, on a digital infrastructure project that mattered, and to share kindness unto others - especially beginners and newcomers.

Rest in peace, amico.

18 Sep 2020 2:17am GMT

Just got around to watching the last two episodes of Clone Wars. There can be no doubt that Asoka is the real hero of the whole "Skywalker" story arc, and maybe more.

17 Sep 2020 4:04am GMT

It's that time of year again: a new GNOME release, and with it, a new Epiphany. The pace of Epiphany development has increased significantly over the last few years thanks to an increase in the number of active contributors. Most notably, Jan-Michael Brummer has solved dozens of bugs and landed many new enhancements, Alexander Mikhaylenko has polished numerous rough edges throughout the browser, and Andrei Lisita has landed several significant improvements to various Epiphany dialogs. That doesn't count the work that Igalia is doing to maintain WebKitGTK, the WPE graphics stack, and libsoup, all of which is essential to delivering quality Epiphany releases, nor the work of the GNOME localization teams to translate it to your native language. Even if Epiphany itself is only the topmost layer of this technology stack, having more developers working on Epiphany itself allows us to deliver increased polish throughout the user interface layer, and I'm pretty happy with the result. Let's take a look at what's new.

Intelligent Tracking Prevention

Intelligent Tracking Prevention (ITP) is the headline feature of this release. Safari has had ITP for several years now, so if you're familiar with how ITP works to prevent cross-site tracking on macOS or iOS, then you already know what to expect here. If you're more familiar with Firefox's Enhanced Tracking Protection, or Chrome's nothing (crickets: chirp, chirp!), then WebKit's ITP is a little different from what you're used to. ITP relies on heuristics that apply the same to all domains, so there are no blocklists of naughty domains that should be targeted for content blocking like you see in Firefox. Instead, a set of innovative restrictions is applied globally to all web content, and a separate set of stricter restrictions is applied to domains classified as "prevalent" based on your browsing history. Domains are classified as prevalent if ITP decides the domain is capable of tracking your browsing across the web, or non-prevalent otherwise. (The public-friendly terminology for this is "Classification as Having Cross-Site Tracking Capabilities," but that is a mouthful, so I'll stick with "prevalent." It makes sense: domains that are common across many websites can track you across many websites, and domains that are not common cannot.)

ITP is enabled by default in Epiphany 3.38, as it has been for several years now in Safari, because otherwise only a small minority of users would turn it on. ITP protections are designed to be effective without breaking too many websites, so it's fairly safe to enable by default. (You may encounter a few broken websites that have not been updated to use the Storage Access API to store third-party cookies. If so, you can choose to turn off ITP in the preferences dialog.)

For a detailed discussion covering ITP's tracking mitigations, see Tracking Prevention in WebKit. I'm not an expert myself, but the short version is this: full third-party cookie blocking across all websites (to store a third-party cookie, websites must use the Storage Access API to prompt the user for permission); cookie-blocking latch mode ("once a request is blocked from using cookies, all redirects of that request are also blocked from using cookies"); downgraded third-party referrers ("all third-party referrers are downgraded to their origins by default") to avoid exposing the path component of the URL in the referrer; blocked third-party HSTS ("HSTS […] can only be set by the first-party website […]") to stop abuse by tracker scripts; detection of cross-site tracking via link decoration and 24-hour expiration time for all cookies created by JavaScript on the landing page when detected; a 7-day expiration time for all other cookies created by JavaScript (yes, this applies to first-party cookies); and a 7-day extendable lifetime for all other script-writable storage, extended whenever the user interacts with the website (necessary because tracking companies began using first-party scripts to evade the above restrictions). Additionally, for prevalent domains only, domains engaging in bounce tracking may have cookies forced to SameSite=strict, and Verified Partitioned Cache is enabled (cached resources are re-downloaded after seven days and deleted if they fail certain privacy tests). Whew!

WebKit has many additional privacy protections not tied to the ITP setting and therefore not discussed here - did you know that cached resources are partioned based on the first-party domain? - and there's more that's not very well documented which I don't understand and haven't mentioned (tracker collusion!), but that should give you the general idea of how sophisticated this is relative to, say, Chrome (chirp!). Thanks to John Wilander from Apple for his work developing and maintaining ITP, and to Carlos Garcia for getting it working on Linux. If you're interested in the full history of how ITP has evolved over the years to respond to the changing threat landscape (e.g. tracking prevention tracking), see John's WebKit blog posts. You might also be interested in WebKit's Tracking Prevention Policy, which I believe is the strictest anti-tracking stance of any major web engine. TL;DR: "we treat circumvention of shipping anti-tracking measures with the same seriousness as exploitation of security vulnerabilities. If a party attempts to circumvent our tracking prevention methods, we may add additional restrictions without prior notice." No exceptions.

Updated Website Data Preferences

As part of the work on ITP, you'll notice that Epiphany's cookie storage preferences have changed a bit. Since ITP enforces full third-party cookie blocking, it no longer makes sense to have a separate cookie storage preference for that, so I replaced the old tri-state cookie storage setting (always accept cookies, block third-party cookies, block all cookies) with two switches: one to toggle ITP, and one to toggle all website data storage.

Previously, it was only possible to block cookies, but this new setting will additionally block localStorage and IndexedDB, web features that allow websites to store arbitrary data in your browser, similar to cookies. It doesn't really make much sense to block cookies but allow other types of data storage, so the new preferences should better enforce the user's intent behind disabling cookies. (This preference does not yet block media keys, service workers, or legacy offline web application cache, but it probably should.) I don't really recommend disabling website data storage, since it will cause compatibility issues on many websites, but this option is there for those who want it. Disabling ITP is also not something I want to recommend, but it might be necessary to access certain broken websites that have not yet been updated to use the Storage Access API.

Accordingly, Andrei has removed the old cookies dialog and moved cookie management into the Clear Personal Data dialog, which is a better place because anyone clearing cookies for a particular website is likely to also want to clear other personal data. (If you want to delete a website's cookies, then you probably don't want to leave its SQL databases intact, right?) He had to remove the ability to clear data from a particular point in time, because WebKit doesn't support this operation for cookies, but that function is probably rarely-used and I think the benefit of the change should outweigh the cost. (We could bring it back in the future if somebody wants to try implementing that feature in WebKit, but I suspect not many users will notice.) Treating cookies as separate and different from other forms of website data storage no longer makes sense in 2020, and it's good to have finally moved on from that antiquated practice.

New HTML Theme

Carlos Garcia has added a new Adwaita-based HTML theme to WebKitGTK 2.30, and removed support for rendering HTML elements using the GTK theme (except for scrollbars). Trying to use the GTK theme to render web content was fragile and caused many web compatibility problems that nobody ever managed to solve. The GTK developers were never very fond of us doing this in the first place, and the foreign drawing API required to do so has been removed from GTK 4, so this was also good preparation for getting WebKitGTK ready for GTK 4. Carlos's new theme is similar to Adwaita, but gradients have been toned down or removed in order to give a flatter, neutral look that should blend in nicely with all pages while still feeling modern.

This should be a fairly minor style change for Adwaita users, but a very large change for anyone using custom themes. I don't expect everyone will be happy, but please trust that this will at least result in better web compatibility and fewer tricky theme-related bug reports.

<figure aria-describedby="caption-attachment-9212" class="wp-caption alignnone" id="attachment_9212" style="width: 1920px"><figcaption class="wp-caption-text" id="caption-attachment-9212">Left: Adwaita GTK theme controls rendered by WebKitGTK 2.28. Right: hardcoded Adwaita-based HTML theme with toned down gradients.</figcaption></figure>

Although scrollbars will still use the GTK theme as of WebKitGTK 2.30, that will no longer be possible to do in GTK 4, so themed scrollbars are almost certain to be removed in the future. That will be a noticeable disappointment in every app that uses WebKitGTK, but I don't see any likely solution to this.

Media Permissions

Jan-Michael added new API in WebKitGTK 2.30 to allow muting individual browser tabs, and hooked it up in Epiphany. This is good when you want to silence just one annoying tab without silencing everything.

Meanwhile, Charlie Turner added WebKitGTK API for managing autoplay policies. Videos with sound are now blocked from autoplaying by default, while videos with no sound are still allowed. Charlie hooked this up to Epiphany's existing permission manager popover, so you can change the behavior for websites you care about without affecting other websites.

<figure aria-describedby="caption-attachment-9140" class="wp-caption aligncenter" id="attachment_9140" style="width: 525px"><figcaption class="wp-caption-text" id="caption-attachment-9140">Configure your preferred media autoplay policy for a website near you today!</figcaption></figure>

Improved Dialogs

In addition to his work on the Clear Data dialog, Andrei has also implemented many improvements and squashed bugs throughout each view of the preferences dialog, the passwords dialog, and the history dialog, and refactored the code to be much more maintainable. Head over to his blog to learn more about his accomplishments. (Thanks to Google for sponsoring Andrei's work via Google Summer of Code, and to Alexander for help mentoring.)

Additionally, Adrien Plazas has ported the preferences dialog to use HdyPreferencesWindow, bringing a pretty major design change to the view switcher:

<figure aria-describedby="caption-attachment-9146" class="wp-caption aligncenter" id="attachment_9146" style="width: 1920px"><figcaption class="wp-caption-text" id="caption-attachment-9146">Left: Epiphany 3.36 preferences dialog. Right: Epiphany 3.38. Note the download settings are present in the left screenshot but missing from the right screenshot because the right window is using flatpak, and the download settings are unavailable in flatpak.</figcaption></figure>

User Scripts

User scripts (like Greasemonkey) allow you to run custom JavaScript on websites. WebKit has long offered user script functionality alongside user CSS, but previous versions of Epiphany only exposed user CSS. Jan-Michael has added the ability to configure a user script as well. To enable, visit the Appearance tab in the preferences dialog (a somewhat odd place, but it really needs to be located next to user CSS due to the tight relationship there). Besides allowing you to do, well, basically anything, this also significantly enhances the usability of user CSS, since now you can apply certain styles only to particular websites. The UI is a little primitive - your script (like your CSS) has to be one file that will be run on every website, so don't try to design a complex codebase using your user script - but you can use conditional statements to limit execution to specific websites as you please, so it should work fairly well for anyone who has need of it. I fully expect 99.9% of users will never touch user scripts or user styles, but it's nice for power users to have these features available if needed.

HTTP Authentication Password Storage

Jan-Michael and Carlos Garcia have worked to ensure HTTP authentication passwords are now stored in Epiphany's password manager rather than by WebKit, so they can now be viewed and deleted from Epiphany, which required some new WebKitGTK API to do properly. Unfortunately, WebKitGTK saves network passwords using the default network secret schema, meaning its passwords (saved by older versions of Epiphany) are all leaked: we have no way to know which application owns those passwords, so we don't have any way to know which passwords were stored by WebKit and which can be safely managed by Epiphany going forward. Accordingly, all previously-stored HTTP authentication passwords are no longer accessible; you'll have to use seahorse to look them up manually if you need to recover them. HTTP authentication is not very commonly-used nowadays except for internal corporate domains, so hopefully this one-time migration snafu will not be a major inconvenience to most users.

New Tab Animation

Jan-Michael has added a new animation when you open a new tab. If the newly-created tab is not visible in the tab bar, then the right arrow will flash to indicate success, letting you know that you actually managed to open the page. Opening tabs out of view happens too often currently, but at least it's a nice improvement over not knowing whether you actually managed to open the tab or not. This will be improved further next year, because Alexander is working on a completely new tab widget to replace GtkNotebook.

New View Source Theme

Jim Mason changed view source mode to use a highlight.js theme designed to mimic Firefox's syntax highlighting, and added dark mode support.

<figure aria-describedby="caption-attachment-9122" class="wp-caption aligncenter" id="attachment_9122" style="width: 997px"><figcaption class="wp-caption-text" id="caption-attachment-9122">Embrace the dark.</figcaption></figure>

And More…

• WebKitGTK 2.30 now supports video formats in image elements, thanks to Philippe Normand. You'll notice that short GIF-style videos will now work on several major websites where they previously didn't.
• I added a new WebKitGTK 2.30 API to expose the paste as plaintext editor command, which was previously internal but fully-functional. I've hooked it up in Epiphany's context menu as "Paste Text Only." This is nice when you want to discard markup when pasting into a rich text editor (such as the WordPress editor I'm using to write this post).
• Jan-Michael has implemented support for reordering pinned tabs. You can now drag to reorder pinned tabs any way you please, subject to the constraint that all pinned tabs stay left of all unpinned tabs.
• Jan-Michael added a new import/export menu, and the bookmarks import/export features have moved there. He also added a new feature to import passwords from Chrome. Meanwhile, ignapk added support for importing bookmarks from HTML (compatible with Firefox).
• Jan-Michael added a new preference to web apps to allow running them in the background. When enabled, closing the window will only hide the the window: everything will continue running. This is useful for mail apps, music players, and similar applications.
• Continuing Jan-Michael's list of accomplishments, he removed Epiphany's previous hidden setting to set a mobile user agent header after discovering that it did not work properly, and replaced it by adding support in WebKitGTK 2.30 for automatically setting a mobile user agent header depending on the chassis type detected by logind. This results in a major user experience improvement when using Epiphany as a mobile browser. Beware: this functionality currently does not work in flatpak because it requires the creation of a new desktop portal.
• Stephan Verbücheln has landed multiple fixes to improve display of favicons on hidpi displays.
• Zach Harbort fixed a rounding error that caused the zoom level to display oddly when changing zoom levels.
• Vanadiae landed some improvements to the search engine configuration dialog (with more to come) and helped investigate a crash that occurs when using the "Set as Wallpaper" function under Flatpak. The crash is pretty tricky, so we wound up disabling that function under Flatpak for now. He also updated screenshots throughout the user help.
• Sabri Ünal continued his effort to document and standardize keyboard shortcuts throughout GNOME, adding a few missing shortcuts to the keyboard shortcuts dialog.

Epiphany 3.38 will be the final Epiphany 3 release, concluding a decade of releases that start with 3. We will match GNOME in following a new version scheme going forward, dropping the leading 3 and the confusing even/odd versioning. Onward to Epiphany 40!

16 Sep 2020 5:00pm GMT

Version 3.29 of syslog-ng was released recently including a user-contributed feature: the panos-parser(). It is parsing log messages from PAN-OS (Palo Alto Networks Operating System). Unlike some other networking devices, the message headers of PAN-OS syslog messages are standards-compliant. However, if you want to act on your messages (filtering, alerting), you still need to parse the message part. The panos-parser() helps you create name-value pairs from the message part of the logs.

From this blog you can learn why it is useful to parse PAN-OS log messages and how to use the panos-parser().

Before you begin

In order to use the panos-parser(), you need to install syslog-ng 3.29 or later. Most Linux distributions feature earlier versions, but the https://www.syslog-ng.com/3rd-party-binaries page of the syslog-ng website has some pointers to 3^rd party repositories featuring up-to-date binaries.

You also need some PAN-OS log messages. If you are reading this blog, you most likely have some Palo Alto Networks devices at hand and your ultimate goal is to collect logs from those devices. In this blog I will use the sample log messages found in the configuration snippet implementing the panos-parser(). Even if you have "real" PAN-OS logs, it is easier to get started configuring and testing syslog-ng this way.

Why is it useful?

As mentioned earlier, PAN-OS devices send completely valid syslog messages. You are not required to do any additional parsing on them. Syslog-ng can interpret the message headers without any additional configuration and save the logs properly:

Apr 14 16:48:54 localhost 1,2020/04/14 16:48:54,unknown,SYSTEM,auth,0,2020/04/14 16:48:54,,auth-fail,,0,0,general,medium,failed authentication for user \'admin\'. Reason: Invalid username/password. From: 10.0.10.55.,1718,0x0,0,0,0,0,,paloalto
Apr 14 16:54:18 localhost 1,2020/04/14 16:54:18,unknown,CONFIG,0,0,2020/04/14 16:54:18,10.0.10.55,,set,admin,Web,Succeeded, deviceconfig system,127,0x0,0,0,0,0,,paloalto

If you look at these logs, you can see that the message part is a list of comma-separated values. That should be easy for the csv-parser(), but the two lists above have a different set of fields and there are a few more message types not included here. You can find the parsers describing these message types in /usr/share/syslog-ng/include/scl/paloalto/panos.conf or in the same directory under /usr/local on most Linux distributions. The panos-parser() can detect what type of log it is and create name-value pairs accordingly. If none of the types match, the parser drops the log. Once you have name-value pairs, it is much easier to create alerts (filters) in syslog-ng or reports in Kibana (if you use the Elasticsearch destination of syslog-ng).

Configuring syslog-ng

In most Linux distributions, syslog-ng is configured in a way so that you can extend it by dropping a configuration file with a .conf extension into the /etc/syslog-ng/conf.d/ directory. In other cases, simply append the below configuration to syslog-ng.conf.

source s_regular { tcp(port(5141)); };
source s_net {
    default-network-drivers(flags(store-raw-message));
};
source s_panosonly { tcp(port(5140) flags(no-parse,store-raw-message)); };

template t_jsonfile {
    template("$(format-json --scope rfc5424 --scope dot-nv-pairs
        --rekey .* --shift 1 --scope nv-pairs --key ISODATE)\n\n");
};
parser p_panos { panos-parser(); };
destination d_frompanos {
    file("/var/log/frompanos" template(t_jsonfile));
};
destination d_other {
    file("/var/log/other");
};
destination d_raw {
    file("/var/log/raw" template("${RAWMSG}\n"));
};
log {
    source(s_regular);
    destination(d_other);
};
log {
    source(s_net);
    destination(d_raw);
    if ("${.app.name}" eq "panos") {
        destination(d_frompanos);
    } else {
        destination(d_other);
    };
};
log {
    source(s_panosonly);
    destination(d_raw);
    parser(p_panos);
    destination(d_frompanos);
};

If you follow my blogs, the above configuration will look familiar: it is based on the configuration I prepared for Cisco in one of my recent blogs, I just replaced the Cisco-specific parts. Some of the text below is also reused, but there are some obvious differences, as the purpose of the parsers is different.

In the following section we will go over this configuration in detail, in the order the statements appear in the configuration. To make your life easier, I copied the relevant snippets below before explaining them.

Sources

source s_regular { tcp(port(5141)); };

This is a pretty regular TCP legacy syslog source on a random high port that I used to create the logs in the "Why is it useful" section. By default, the tcp() source handles all incoming log messages as if they were legacy (RFC3164) formatted and in case of PAN-OS logs it results in properly formatted logs.

source s_net {
    default-network-drivers(flags(store-raw-message));
};

The default-network-drivers() source driver is a kind of a wild card. It opens different UDP and TCP ports using both the legacy and the new RFC5424 syslog protocols. Instead of just expecting everything to be regular syslog, it attempts a number of different parsers on incoming logs, including the panos-parser(). Of course, the extra parsing creates some overhead, but it is not a problem unless you have a very high message rate.

The store-raw-message flag means that syslog-ng preserves the original log message as is. It might be useful for debugging or if a log analysis software expects unmodified PAN-OS messages.

source s_panosonly { tcp(port(5140) flags(no-parse,store-raw-message)); };

The third source, as its name also implies, is only for PAN-OS log messages. Use it when you have a high message rate, you only send PAN-OS log messages at the given port and you are sure that the panos-parser() of syslog-ng can process all of your PAN-OS logs correctly. The no-parse flag means that incoming messages are not parsed automatically as they arrive. As you will see later, panos-parser() parses the incoming messages.

Templates

template t_jsonfile {
    template("$(format-json --scope rfc5424 --scope dot-nv-pairs
        --rekey .* --shift 1 --scope nv-pairs --key ISODATE)\n\n");
};

This is the template, which is often used together with Elasticsearch (without the line breaks at the end). This blog does not cover Elasticsearch, as there are many other blogs covering the topic. However, this template using the JSON template function is also useful here because it shows the name-value pairs parsed from PAN-OS log messages. These JSON formatted logs include all syslog-related fields, name-value pairs parsed from the message, and the date using the ISO standardized format. There is one little trick that might confuse you: the rekey and shift part removes the dots from the front of the name-value pair names. Syslog-ng uses the dot for name-value pairs created by parsers in the syslog-ng configuration library (SCL).

Parsers

parser p_panos { panos-parser(); };

This parser can extract name-value pairs from the log messages of PAN-OS devices. By default, these are stored in name-value pairs starting with .panos., but you can change the prefix using the prefix() parameter. Note that the panos-parser() drops the message if it does not match the rules. This can be a problem when you use it directly instead of using the default-network-drivers() where messages go through a long list of parsers.

Destinations

destination d_frompanos {
    file("/var/log/frompanos" template(t_jsonfile));
};

This is a file destination where logs are JSON-formatted using the template from above. This way you can see all the name-value pairs syslog-ng creates. We use it for PAN-OS log messages.

destination d_other {
    file("/var/log/other");
};

This is a flat file destination using regular syslog formatting. We use it to store non-PAN-OS log messages.

destination d_raw {
    file("/var/log/raw" template("${RAWMSG}\n"));
};

This is yet another file destination. The difference here is that it uses a special template defined in-line. Using the RAWMSG macro, syslog-ng stores the log message without any modifications. This possibility is enabled by utilizing the store-raw-message flag on the source side and it is useful for debugging or when a SIEM or any other analysis software needs the original log message.

Log statements

Previously we defined the building blocks of the configuration. Using the log statements below,we are going to connect these building blocks together. The different building blocks can be used multiple times in the same configuration.

log {
    source(s_regular);
    destination(d_other);
};

This is the simplest log statement: it connects the regular tcp() source with a flat file destination. You can see the results from this in the "Why is it useful" section. The logs look OK, but at a closer look you can also see that they contain tons of structured information. That is where the panos-parser() comes in handy.

log {
    source(s_net);
    destination(d_raw);
    if ("${.app.name}" eq "panos") {
        destination(d_frompanos);
    } else {
        destination(d_other);
    };
};

Unless you have a high message rate, above 50,000 to 200,000 events per second (EPS), or not enough hardware capacity, the recommended way of receiving PAN-OS messages is using the default-network-drivers(). It uses slightly more resources, but if a message does not match the expectations of the panos-parser(), it is still kept. The above log statement receives the message and then stores it immediately in raw format. It can be used for debugging and disabled later.

The if statement below checks if the message is recognized as a PAN-OS log message. If it is, the log is saved as a JSON formatted file. If not, it is saved as a flat file.

Note that the name-value pair is originally called .app.name, but in the output it appears as app.name, as the template removes the dot in front.

log {
    source(s_panosonly);
    destination(d_raw);
    parser(p_panos);
    destination(d_frompanos);
};

If you have a high message rate and you are sure that the panos-parser() detects all of your logs, you can use this solution to collect logs from your PAN-OS devices. For safety and debugging purposes I inserted the raw file destination in front of the parser. This way you can compare the number of lines in the two different file destinations. If they are not equal, check the logs that the panos-parser() discarded.

Testing

For testing I used a few sample log messages from the syslog-ng configuration snippet containing the panos-parser() and saved these messages into a file in /root/panoslogs.txt. Logger generated the regular syslog messages and netcat submitted the PAN-OS messages. Here is the content of /root/panoslogs.txt:

<12>Apr 14 16:48:54 paloalto.test.net 1,2020/04/14 16:48:54,unknown,SYSTEM,auth,0,2020/04/14 16:48:54,,auth-fail,,0,0,general,medium,failed authentication for user \'admin\'. Reason: Invalid username/password. From: 10.0.10.55.,1718,0x0,0,0,0,0,,paloalto
<14>Apr 14 16:54:18 paloalto.test.net 1,2020/04/14 16:54:18,unknown,CONFIG,0,0,2020/04/14 16:54:18,10.0.10.55,,set,admin,Web,Succeeded, deviceconfig system,127,0x0,0,0,0,0,,paloalto

You will get different results epending on to which port you send the logs. You have already seen what happens when you send logs to port 5141. It looks great, but you can also see that a bit of parsing could do wonders to it.

Here is an example of sending logs to port 514:

logger -T --rfc3164 -n 127.0.0.1 -P 514 this is a regular syslog message
cat /root/panoslogs.txt | netcat -4 -n -N -v 127.0.0.1 514

In this case you should see in the files:

localhost:/etc/syslog-ng/conf.d # cat /var/log/other
Sep 14 15:25:54 localhost root: this is a regular syslog message
localhost:/etc/syslog-ng/conf.d # cat /var/log/frompanos
{"panos":{"vsys_name":"","vsys":"","type":"SYSTEM","time_generated":"2020/04/14 16:48:54","subtype":"auth","severity":"medium","serial":"unknown","seqno":"1718","receive_time":"2020/04/14 16:48:54","opaque":"failed authentication for user \\'admin\\'. Reason: Invalid username/password. From: 10.0.10.55.","object":"","module":"general","future_use4":"0","future_use3":"0","future_use2":"0","future_use1":"1","eventid":"auth-fail","dg_hier_level_4":"0","dg_hier_level_3":"0","dg_hier_level_2":"0","dg_hier_level_1":"0","device_name":"paloalto","actionflags":"0x0"},"app":{"name":"panos"},"SOURCE":"s_net","RAWMSG":"<12>Apr 14 16:48:54 paloalto.test.net 1,2020/04/14 16:48:54,unknown,SYSTEM,auth,0,2020/04/14 16:48:54,,auth-fail,,0,0,general,medium,failed authentication for user \\'admin\\'. Reason: Invalid username/password. From: 10.0.10.55.,1718,0x0,0,0,0,0,,paloalto","PROGRAM":"paloalto_panos","PRIORITY":"warning","MESSAGE":"1,2020/04/14 16:48:54,unknown,SYSTEM,auth,0,2020/04/14 16:48:54,,auth-fail,,0,0,general,medium,failed authentication for user \\'admin\\'. Reason: Invalid username/password. From: 10.0.10.55.,1718,0x0,0,0,0,0,,paloalto","ISODATE":"2020-04-14T16:48:54+02:00","HOST_FROM":"localhost","HOST":"paloalto.test.net","FACILITY":"user","DATE":"Apr 14 16:48:54"}

{"panos":{"vsys_name":"","vsys":"","type":"CONFIG","time_generated":"2020/04/14 16:54:18","subtype":"0","serial":"unknown","seqno":"127","result":"Succeeded","receive_time":"2020/04/14 16:54:18","path":" deviceconfig system","host":"10.0.10.55","future_use2":"0","future_use1":"1","dg_hier_level_4":"0","dg_hier_level_3":"0","dg_hier_level_2":"0","dg_hier_level_1":"0","device_name":"paloalto","cmd":"set","client":"Web","admin":"admin","actionflags":"0x0"},"app":{"name":"panos"},"SOURCE":"s_net","RAWMSG":"<14>Apr 14 16:54:18 paloalto.test.net 1,2020/04/14 16:54:18,unknown,CONFIG,0,0,2020/04/14 16:54:18,10.0.10.55,,set,admin,Web,Succeeded, deviceconfig system,127,0x0,0,0,0,0,,paloalto","PROGRAM":"paloalto_panos","PRIORITY":"info","MESSAGE":"1,2020/04/14 16:54:18,unknown,CONFIG,0,0,2020/04/14 16:54:18,10.0.10.55,,set,admin,Web,Succeeded, deviceconfig system,127,0x0,0,0,0,0,,paloalto","ISODATE":"2020-04-14T16:54:18+02:00","HOST_FROM":"localhost","HOST":"paloalto.test.net","FACILITY":"user","DATE":"Apr 14 16:54:18"}

localhost:/etc/syslog-ng/conf.d # cat /var/log/raw
<13>Sep 14 15:25:54 localhost root: this is a regular syslog message
<12>Apr 14 16:48:54 paloalto.test.net 1,2020/04/14 16:48:54,unknown,SYSTEM,auth,0,2020/04/14 16:48:54,,auth-fail,,0,0,general,medium,failed authentication for user \'admin\'. Reason: Invalid username/password. From: 10.0.10.55.,1718,0x0,0,0,0,0,,paloalto
<14>Apr 14 16:54:18 paloalto.test.net 1,2020/04/14 16:54:18,unknown,CONFIG,0,0,2020/04/14 16:54:18,10.0.10.55,,set,admin,Web,Succeeded, deviceconfig system,127,0x0,0,0,0,0,,paloalto

When you send logs to port 5140 where the panos-parser() is the only parser, the results should be pretty similar. The only difference is that the regular syslog message is only saved to the raw file used for debugging, as it is discarded by the panos-parser().

If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For contact information, check our GitHub page under the "Community" section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik.

16 Sep 2020 11:48am GMT

Fedora 33 will switch to systemd-resolved for name resolution.

Resolved has a bundled list of DNS used in case of network settings misconfiguration, i.e. the DHCP doesn't provide the DNS address and probably other cases, for instance when you don't intentionally set a DNS address in the network configuration.

16 Sep 2020 7:42am GMT

Cockpit is the modern Linux admin interface. We release regularly. Here are the release notes from Cockpit version 228.

Accounts: Allow setting weak passwords

Cockpit now allows users to set up a weak password. The user gets notified that the password is weak, but clicking the submit button again will use the selected password anyway. This is similar to how weak passwords are handled in the Anaconda installer.

Changes to remote host logins

Cockpit used to try to log into remote hosts with your initial login password when "Reuse my password for remote connections" was selected during login. This option has been removed, and Cockpit does not promote password reuse across accounts anymore. Instead, Cockpit now helps with setting up SSH keys for users that want automatic and password-less login to remote hosts.

There is also an earlier demo video to see how it looks like in action.

Machines: Add support for reverting to and deleting VM snapshots

One can now restore a VM guest to a previously created snapshot. Existing snapshots can be deleted as well. Deleting a single snapshot preserves the current state of the VM and does not affect any other snapshot.

Drop cockpit-docker code

As announced half a year ago in Cockpit 215, cockpit-docker is deprecated in favor of cockpit-podman. None of the currently supported operating systems builds the cockpit-docker package any more, thus it was finally removed upstream as well. If you still use it on another distribution or from upstream builds, please migrate now, or build it from an older upstream release.

Try it out

Cockpit 228 is available now:

• For your Linux system
• Cockpit Source Tarball
• Cockpit Fedora 32
• Cockpit Fedora 33

16 Sep 2020 12:00am GMT

The first extension I always install in Chrome is "uBlock Origin" of course to try and prevent as many wasteful ads as possible but it doesn't specifically target entire web properties such as all of facebooks sub domains that exist out there (for example, if someone puts a fb image or like button on their site and your browser loads that content, it's another signal they can use with your information even though I don't have a fb account).

I found a cool extension for Chrome called "Domain Blocker" which lets you specify wildcard sub domain names in a simple list to block any web requests at the browser level directly (no messy etc/hosts file setups or maintenance needed). For example, you can grab a master list of facebook domain names and place some basic regex in it to produce a nice short list to block automatically:

$ curl -sL 'https://raw.githubusercontent.com/jmdugan/blocklists/master/corporations/facebook/all' | tr '.' ' ' | awk '{ print $(NF-1)"."$NF }' | sort | uniq | while read d ; do echo "$d" ; echo "*.$d" ; done

facebook.com
*.facebook.com
facebook.de
*.facebook.de
facebook.fr
*.facebook.fr
facebook.net
*.facebook.net
fb.com
*.fb.com
fb.me
*.fb.me
fbcdn.com
*.fbcdn.com
fbcdn.net
*.fbcdn.net
fbsbx.com
*.fbsbx.com
fburl.com
*.fburl.com
foursquare.com
*.foursquare.com
freebasics.com
*.freebasics.com
hootsuite.com
*.hootsuite.com
instagram.com
*.instagram.com
internet.org
*.internet.org
m.me
*.m.me
messenger.com
*.messenger.com
online-metrix.net
*.online-metrix.net
tfbnw.net
*.tfbnw.net
thefacebook.com
*.thefacebook.com
wechat.com
*.wechat.com
whatsapp.com
*.whatsapp.com
whatsapp.net
*.whatsapp.net

This will now produce a blocked message if your browser tries to load any content from any of those domains (links, imgs, scripts, frames, etc.):

www.facebook.com is blocked
Requests to the server have been blocked by an extension.
Try disabling your extensions.
ERR_BLOCKED_BY_CLIENT

Reminder: Also make sure to block third party cookies in Chrome's settings as well, it will help a lot to keep things clean along the way!

15 Sep 2020 5:33am GMT

Josh and Kurt talk about attacking open source. How serious is the threat of developers being targeted or a git repo being watched for secret security fixes? The reality of it all is there are many layers in a security journey, the most important things you can do are also the least exciting.

<audio class="wp-audio-shortcode" controls="controls" id="audio-1923-1" preload="none" style="width: 100%;"><source src="https://traffic.libsyn.com/secure/opensourcesecuritypodcast/Episode_215_Real_security_is_boring.mp3?_=1" type="audio/mpeg">https://traffic.libsyn.com/secure/opensourcesecuritypodcast/Episode_215_Real_security_is_boring.mp3</audio>

Show Notes

• Targeting developers
• XKCD Infrastructure comic
• Hiding security flaws in git
• Mossad vs Not-Mossad (PDF warning)

14 Sep 2020 12:00am GMT

To capture DHCP packets I ran:

 tcpdump port 67 -i vnet0 -vvvv  -w /tmp/packets.bin

That gave me a binary file 940 bytes long. This is actually 2 packets: the request and the response. This has the IP header, the UDP header, and the DHCP packet payload in it.

First, lets let tcpdump interpret the whole thing for us, and then we can starp picking it apart:

#  tcpdump -r /tmp/packets.bin -vvvv
reading from file /tmp/packets.bin, link-type EN10MB (Ethernet)
19:42:46.742937 IP (tos 0x0, ttl 64, id 278, offset 0, flags [none], proto UDP (17), length 428)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 52:54:00:94:9e:f2 (oui Unknown), length 400, xid 0xff77df41, secs 4, Flags [none] (0x0000)
          Client-Ethernet-Address 52:54:00:94:9e:f2 (oui Unknown)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            MSZ Option 57, length 2: 1472
            ARCH Option 93, length 2: 0
            NDI Option 94, length 3: 1.2.1
            Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
            User-Class Option 77, length 4: 
              instance#1: ERROR: invalid option
            Parameter-Request Option 55, length 23: 
              Subnet-Mask, Default-Gateway, Domain-Name-Server, LOG
              Hostname, Domain-Name, RP, MTU
              Vendor-Option, Vendor-Class, TFTP, BF
              Option 119, Option 128, Option 129, Option 130
              Option 131, Option 132, Option 133, Option 134
              Option 135, Option 175, Option 203
            T175 Option 175, length 48: 2969895296,2249199339,50397184,385941794,16847617,17891585,654377241,16846849,35717377,352387352,16852481,17957121
            Client-ID Option 61, length 7: ether 52:54:00:94:9e:f2
            GUID Option 97, length 17: 0.178.35.76.56.225.195.173.69.183.151.210.221.34.14.27.157
            END Option 255, length 0
19:42:50.798338 IP (tos 0x0, ttl 64, id 666, offset 0, flags [none], proto UDP (17), length 428) Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 52:54:00:94:9e:f2 (oui Unknown), length 400, xid 0xff77df41, secs 10, Flags [none] (0x0000)
          Client-Ethernet-Address 52:54:00:94:9e:f2 (oui Unknown)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            MSZ Option 57, length 2: 1472
            ARCH Option 93, length 2: 0
            NDI Option 94, length 3: 1.2.1
            Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
            User-Class Option 77, length 4: 
              instance#1: ERROR: invalid option
            Parameter-Request Option 55, length 23: 
              Subnet-Mask, Default-Gateway, Domain-Name-Server, LOG
              Hostname, Domain-Name, RP, MTU
              Vendor-Option, Vendor-Class, TFTP, BF
              Option 119, Option 128, Option 129, Option 130
              Option 131, Option 132, Option 133, Option 134
              Option 135, Option 175, Option 203
            T175 Option 175, length 48: 2969895296,2249199339,50397184,385941794,16847617,17891585,654377241,16846849,35717377,352387352,16852481,17957121
            Client-ID Option 61, length 7: ether 52:54:00:94:9e:f2
            GUID Option 97, length 17: 0.178.35.76.56.225.195.173.69.183.151.210.221.34.14.27.157
            END Option 255, length 0

My last post was a one liner I use to look at the packets with hexdump. This is the first couple lines of output from the packets I captured.

00000000  212 195 178 161 002 000 004 000  000 000 000 000 000 000 000 000  |................|
00000016  000 000 004 000 001 000 000 000  246 092 093 095 025 086 011 000  |.........\]_.V..|

Another way to look at the packets is using emacs and hexl-mode. Both have their uses. Most valuable is the ability to run the file through tcpdump with various flags to see what it gives.

For example, I can run:

#  tcpdump -r /tmp/packets.bin -X
reading from file /tmp/packets.bin, link-type EN10MB (Ethernet)
19:42:46.742937 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:94:9e:f2 (oui Unknown), length 400
        0x0000:  4500 01ac 0116 0000 4011 782c 0000 0000  E.......@.x,....

This shows the first two bytes of the UDP packet data as the 4500 (hex). The same is true of the second packet:

19:42:50.798338 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:94:9e:f2 (oui Unknown), length 400
        0x0000:  4500 01ac 029a 0000 4011 76a8 0000 0000  E.......@.v.....

Hex 45 is Decimal 69. Looking with Hexdump:

[root@nuzleaf ~]# hexdump  /tmp/packets.bin  -e'"%07.8_ad  " 8/1 "%03d " "  " 8/1 "%03d " "  |"' -e'16/1  "%_p"  "|\n"' -v | grep 69
00000048  000 148 158 242 008 000 069 000  001 172 001 022 000 000 064 017  |......E.......@.|

We see 069 000. That matches the 4500.

Next in the decimal sequence is 001 172. In the hex we have 01ac which matches. This offset is 54. We know the whole file is 940 bytes long. That means each packet is probably 470 bytes. 470 minus 54 is still 416 is still longer than the 300 that we know is the length of the DHCP packet.

tcpdump shows us that the length is 400, which is, I think the length of the IP packet.

Possibly the best pattern to match is the MAC address, which we know to be

82, 84, 0, 230, 8, 49 (converted from hex 0x52,0x54,0x00,0xE6,0x08,0x031)

The MAC address is supposed to be put in the packet at an offset of 32 bytes in. The length is 6. We see this pattern on the line that starts with offset 00000096 and continues onto the next line:

00000096  000 000 000 000 000 000 000 000  000 000 000 000 000 000 082 084  |..............RT|
00000112  000 148 158 242 000 000 000 000  000 000 000 000 000 000 000 000  |................|

Working backwards, now, we should be able to pick out the patter for the start of the packet: 01 for the Opcode, 01 for the Hardware type, and 06 for the HW length. We see that here:

00000080  077 216 001 001 006 000 255 119  223 065 000 004 000 000 000 000  |M......w.A......|

So our DHCP payload starts at offset 82. It should continue for 300 bytes.

00000368  050 048 048 049 077 004 105 080  088 069 055 023 001 003 006 007  |2001M.iPXE7.....|
00000384  012 015 017 026 043 060 066 067  119 128 129 130 131 132 133 134  |....+<bcw.......>

That actually lines up with the error reported in the tcpdump:

instance#1: ERROR: invalid option Parameter-Request Option 55, length 23:

The 55 is followed by the length 23, the two values 001 and 003...and that is the end of the packet. 006 007 012 and so on might be part of the packet, but the BOOTP protocol hardcodes the length at 300 and so they get ignored. If we could find the length of the data section of the UPD header, we might know better.

Note that the bad checksum is reported as well. That value should be in the 2 bytes right before the start of the actual UDP data. And the two bytes prior to that is the UDP data length. Looking at the dump above we see that the line off set 00000080 starts with 077 216 before going into the 001 001 006 of the Boot packet. This is the checksum. The line before that ends with the two values 001 152. In Hex that is 0198 (I looked in hexl-mode) which in decimal is 408.

Our DHCP packet, which is supposed to be 300 Bytes long, is 408 bytes long. Is this legal?

Apparently, yes: RFC1531 extended the BOOTP definition. https://tools.ietf.org/html/rfc1531#section-2 DHCP extended the BOOTP packet size by renaming the vendor-specific area to options and giving it a size of 312.

 The options field is now variable length, with the minimum extended
   to 312 octets.

This seems to mean that tcpdump is working with the older format, which is no longer valid. I also need to extend the size of my packet in rust.

13 Sep 2020 2:55pm GMT

SecureDrop workstation project's packages are reproducible. We use prebuilt wheels (by us) along with GPG signatures to verify and install them using pip during the Debian package building step. But, the way we built those wheels (standard pip command), they were not reproducible.

To fix this problem, Jennifer Helsby (aka redshiftzero) built a tool and the results are available at https://reproduciblewheels.com/. Every night her tool is building the top 100 + our dependency packages on Debian Buster and verifies the reproducibly of them. She has a detailed write up on the steps.

While this issue was fixed, a related issue was to have reproducible source tarballs. python3 setup.py sdist still does not give us a reproducible tarballs. Conor Schaefer, our CTO at the Freedom of the Press Foundation decided to tackle that issue using a few more lines of bash in our build scripts. Now we have reproducible wheels and source tarballs (based on specified timestamps) for our projects.

13 Sep 2020 5:22am GMT

On September 1st, 2020, the maximum lifetime of TLS certificates signed by Public Certificate Authority got reduced to 13 months. How did we arrive here, and what's to come? Let's start from understanding who decides the maximum lifetime of certificates and many other limitations around them. Who decides the TLS certificate guidelines Ultimately, the client (often a browser or an operating system) identifies the certificate as trustable or not (based on the CA that signed it as well as many other parameters), so the client can decide which parameters to look for and which values are acceptable and which are not.

13 Sep 2020 12:00am GMT

Version 8.0.0 Beta 3 is released. It's now enter the stabilisation phase for the developers, and the test phase for the users.

RPM are available in the remi-php80 repository for Fedora ≥ 31 and Enterprise Linux ≥ 7 (RHEL, CentOS), or in the php:remi-8.0 stream, and as Software Collection in the remi-safe repository (or remi for Fedora)

The repository provides development versions which are not suitable for production usage.

Also read: PHP 8.0 as Software Collection

Installation : read the Repository configuration and choose installation mode.

Replacement of default PHP by version 8.0 installation, module way (simplest way on Fedora and EL-8):

dnf module disable php
dnf module install php:remi-8.0
dnf update

Replacement of default PHP by version 8.0 installation, repository way (simplest way on EL-7):

yum-config-manager --enable remi-php80
yum update php\*

Parallel installation of version 8.0 as Software Collection (recommended for tests):

yum install php80

To be noticed :

• EL8 rpm are build using RHEL-8.2
• EL7 rpm are build using RHEL-7.8
• lot of extensions are also available, see the PECL extension RPM status page
• follow the comments on this page for update until final version.

Information, read:

• UPGRADING
• UPGRADING.INTERNALS

Base packages (php)

Software Collections (php74)

12 Sep 2020 6:28am GMT

hexdump  boot-packet.bin  -e'"%07.8_ad  " 8/1 "%03d " "  " 8/1 "%03d " "  |"' -e'16/1  "%_p"  "|\n"' -v

Found here:

12 Sep 2020 3:34am GMT

A few days ago, setuptools 50.0.0 release caused breakage to many projects. SecureDrop package builds was also broken. We use dh-virtualenv tool to build the packages. Initially, we tried to use the experimental build system from dh-virtualenv. We could specify the version of the setuptools to be installed in the virtualenv while creating it.

This approach worked for Xenial builds. As we are working to have proper builds on Focal (still work in progress), that was broken due to the above-mentioned change.

So, we again tried to use Python's venv module itself to create the virtual environment and use the wheels from the /usr/share/python-wheels directory to build the virtual environment. Which works very nicely on Xenial, but on Focal the default setuptools version is 44.0.0, which also failed to install the dependencies.

Now, we are actually getting the setuptools 46.0.0 wheel and replacing the build container's default setuptools wheel. The team spent a lot of time in debugging and finding a proper fix for the package builds. Hopefully, we will not get a similar breakage on the same kind of dependency error soon (the actual package dependencies are pinned via hashes).

11 Sep 2020 12:29pm GMT