23 Oct 2016
The post Pi-Hole: A DNS-based blacklist for ads and tracking for Raspberry Pi appeared first on ma.ttias.be.
I just "upgraded" our home network with a Pi-Hole, an interesting project that implements a DNS server with a known-list of ad- and privacy trackers. The result is that everyone on your network that uses that DNS server gets an adblocker for free, without configuration work.
How DNS-based blacklisting works
The idea is simple and powerful: the project acts as your DNS server, so every time your smartphone/tablet/laptop/pc requests the IP for known adtrackers, the Pi-Hole just responds with a bogus IP address.
Instead of connecting to the server for the adtrackers, you connect to your Pi. Since that's running a webserver, it instantly connects and returns an empty reply.
There's no delay in waiting to connect to a non-existent IP address, there's just an instant "blackholed" answer.
While I usually don't like curl | bash installers, this one seems pretty safe.
I installed it on a Raspberry Pi, but it actually runs on any Ubuntu/Debian based Linux server. The Pi is just convenient as it's a low-power, easy-to-hide-away little server for your home.
On your Pi, run this;
$ curl -L https://install.pi-hole.net | bash
And you're good to go.
Your entire home gets an adblocker, without the hassle
If you want to protect the members of your home (or: anyone connected to your WiFi), it's usually a bit of a hassle: every device needs an adblocker/privacy blocker, it's a pain to get it to work on iOS, ...
Since Pi-Hole acts as a DNS server, all you need to do is to configure your router to stop handing out the DNS server(s) from your ISP, but to configure it to point to the IP of your Raspberry Pi running the Pi-Hole.
Added bonus: since that Pi-Hole can be configured to use Google's upstream namservers (18.104.22.168 & 22.214.171.124) you automatically bypass DNS based blocks implemented by your provider.
So if your ISP is preventing you from visiting certain websites, that's usually DNS based and the Pi-Hole bypasses that by not using your ISPs' nameservers.
This is actually what made me try out Pi-Hole in the first place: their reporting dashboard!
Every DNS query made gets logged: you get graphs, a list of DNS queries, you can add white- or blacklist entries, ...
Top DNS & advertisers
Full DNS query log
Control over convenience
There are already DNS-based blacklist providers you can use (like OpenDNS), but having the DNS server run on your own Raspberry Pi gives you more control and privacy. All the blocking & tracking happens in the comfort of your own home, not by a corporation you don't know.
The downside; you need to configure a Pi and you'll have to troubleshoot why DNS isn't working one day, if the service is stopped/crashed or your Pi has stopped working.
But as a sysadmin, I don't mind -- I'm happy my home gets an adblocker for everyone connected to our WiFi for free.
The post Pi-Hole: A DNS-based blacklist for ads and tracking for Raspberry Pi appeared first on ma.ttias.be.
23 Oct 2016 10:08am GMT
22 Oct 2016
Het is weer eens crisis. Het is altijd crisis. Deze keer is het crisis omdat ik niet weet welke "C++ met Qt" recruiter eerst te beantwoorden. Zelden was het zo erg als nu.
Ik besloot een paar weken geleden eens even niet op Linked-In te antwoorden. Sindsdien heb ik na een ruime schatting ongeveer tien Belgische recruiters die een Qt/C++ ontwikkelaar zoeken en een veelvoud daarvan in Nederland (onder meer voor het bedrijf waar ik nu voor werk). En euh .. voor Duitsland zou ik een database moeten aanleggen om het te kunnen volgen. England negeer en blokkeer ik al een tijdje.
Toegegeven zijn heel wat recruiters voor hetzelfde bedrijf op zoek. Er moet toch iets lucratief aan zijn opdat er zoveel tegelijk los gaan?
Het probleem van C++ met Qt is volgens mij dat er te weinig zielen het kunnen. Dit houdt de explosie van het gebruik er van meer tegen dan goed is, denk ik. Voornamelijk in embedded en industriële toepassingen loopt het goed. Vandaar misschien dat het nogal druk in Duitsland is? Internet of crap dingen ook wel wat. Maar het zijn toch vooral webapp developers die men daar zoekt. Verbazingwekkend is er ook veel pro-Qt belangstelling van alles wat met mediaplayers te maken heeft?
Wij met C++/Qt ervaring zouden beter wat meer lessen en cursussen geven aan de jongeren. We kunnen deze markt toch niet alleen aan.
Iemand nog ideeën?
Misschien moet er toch eens iets komen om met de project managers te gaan overleggen? Ik heb al eens een soort van gilde voorgesteld he. Eind van het jaar is het altijd hetzelfde. Plots hebben ze zicht op de goedkeuring van één of ander budget van hun CEO. En dan zoeken ze allemaal te samen, liefst voor eergisteren, naar twintig programmeurs. Een beetje zoals wilde beesten.
Eigenlijk is het al niet meer enkel het eind van het jaar, maar valt het meestal met de kwartaalovergangen samen.
22 Oct 2016 6:22pm GMT
21 Oct 2016
D'un coup sec, j'enfonce le clou dans la planche vermoulue. Avec un bruit mat, le marteau s'écrase sur le bois, éclaboussant l'obscurité d'un remugle de saumure.
Du bout des doigts, je caresse l'intérieur de la coque, explorant les sillons, les mousses et les algues se frayant un passage à travers les planches.
- Tu n'en as plus pour longtemps, murmuré-je à l'intention du Corsaire.
Dans la pénombre, un long craquement mélancolique me répond. Je souris d'une silencieuse tristesse avant d'être interrompu par le claquement joyeux des bottes résonnant au dessus de ma tête.
Traversant la cale, enjambant par réflexe les éléments de charpente, je me dirige vers l'échelle.
- Oh camarades ! Que nous vaut ce brouhaha, m'exclamé-je ?
- Regarde, Petit père ! Regarde ! Il s'approche, il nous accoste !
Mais face à l'azur adamantin, mes pupilles chthoniennes clignent, se froissent et capitulent. Je ne suis qu'une escarbille embrasée dans l'éclatante blancheur de l'air libre.
- Qui nous accoste ?
- L'Espérance ! Les voilà !
Un choc sourd fait trembler la coque, immédiatement suivi d'une clameur de joie. Des voix nouvelles m'envahissent, des rythmes de pas inconnus se font entendre, des odeurs de vies assaillent mes narines.
- Bravo ! Bienvenue ! Vie l'Espérance ! Vive le Corsaire.
On s'embrasse dans les coursives, on se roule dans les cabestans, on rit, on chante, on mélange les odeurs et on danse au son de l'onde.
Je devine l'approche du contremaître qui, d'un coup sec, arrache la bâche sur notre pont et dévoile la carcasse que nous étions en train de construire. Je sens sa fierté jaillir par tous les pores de sa peau alors qu'il entonne le discours traditionnel.
- Camarades ! Nous, marins du Corsaire, avons construit une carcasse qui nous succédera et continuera notre inlassable quête de rivages. Cependant, il nous manque la coque. Nous avons les mâts, apporterez-vous les voiles ?
- Camarades ! répond une voix nouvelle, feutrée, profonde, chargée d'embruns. Nous, marins de l'Espérance, avons tissé des voiles et avons en suffisance des planches pour réaliser une coque.
- Comment appellerons-nous ce fier navire qui portera vos voiles sur nos mâts ?
- Le Bienvenue !
Comme un seul homme, les deux équipages se lèvent et entonnent un chant fait de vivas, de battements de pieds et d'applaudissements.
Je souris à cet ouranien univers avant de replonger vers le fond de cale afin d'annoncer la nouvelle au Corsaire.
C'est un fameux voyage que le Corsaire et l'Espérance vivent côte-à-côte. Souvent, j'entends la coque de l'Espérance râper un peu plus profondément les planches déjà moisies du Corsaire. J'ai beau l'enduire de goudron, le colmater d'étoupe, il râle, souffle et craque.
- Petit père ! Petit père ! Viens donc voir le Bienvenue !
Une main jeune et ferme me guide sur le pont, mêlant de lumineuses effluves d'épice à la douceur ligneuse du sapin frais.
- Attention Petit père ! Il y a des trous. Nous avons utilisé les planches pour le Bienvenue.
- Attention Petit père, le cordage a été coupé et placé dans le Bienvenue.
Que de changements ! Que de transformations ! Le Corsaire est-il donc désossé ?
- Le Bienvenue me semble si petit. Pourra-t-il emporter beaucoup de monde ?
- Tu sais, Petit père, nous l'agrandirons durant le voyage. Alors, certes, nous serons serrés au début. Mais, très vite, nous prendrons nos aises ! Et puis, le voyage ne sera plus long. Nous sommes convaincus que le Bienvenue accostera !
Je pousse un profond soupir et adresse à cette jeune voix pleine d'énergie ma plus belle larme de sourire.
- Camarades, il est temps de mettre le Bienvenue à l'eau ! Hardi ! Ho Hisse ! Ho Hisse !
Une éclaboussure d'écume me trempe de son vibrant fracas. Par toutes les écoutilles jaillit la joie et la clameur. Une main s'accroche à mon paletot défraîchi.
- Tu embarques Petit père ?
J'hésite. Je déglutis.
- Non. Je reste sur le Corsaire.
- Mais n'as-tu pas dit qu'il n'en avait plus pour très longtemps ?
- Je sais, mais c'est encore suffisamment longtemps pour moi.
- Mais nous avons utilisé ses planches, ses cordages, ses poulies. Il ne peut plus naviguer.
- Il naviguera bien assez pour moi.
- Es-tu sûr Petit père ?
- Oui, certainement.
- Alors, larguez les amarres !
Les voix et les rires se font soudainement lointaines.
- Bonne chance Petit père ! Merci !
- Bon voyage Bienvenue ! Bons rivages !
Pendant de longues minutes, de longues heures, je continue à agiter la main en direction du Bienvenue. Je sais qu'il ne me voient plus mais j'ai l'intime conviction de les sentir, que mon adieu est nécessaire, pertinent.
À pleins poumons, je respire cet air silencieux dans lequel le Corsaire s'est encalminé.
Me guidant prudemment sur les restes dépecés de rambardes, enjambant les outils oubliés et les planches arrachées, je retourne paisiblement vers la confortable moiteur de la cale.
- Dis-moi Corsaire, tu crois qu'ils vont aborder ? Tu crois qu'ils ont une chance de trouver un rivage ?
Le bruit sec d'une planche qui casse me fait sursauter.
- Ou peut-être pourront-ils construire un bateau qui, lui atteindra le rivage ?
Des doigts, je frôle une concrétion marine tandis que l'odeur de la mer me pénètre.
- Le rivage existe-t-il vraiment ? N'est-il pas une invention, une chimère ?
Mes sabots se remplissent d'une eau clapotante, mes doigts s'engourdissent.
- Au fond, cela a-t-il la moindre importance ?
Un grincement humide suivi d'un craquement bref. Le Corsaire se penche brusquement au point de me faire chanceler.
- Au fond Corsaire… Au fond j'ai toujours voulu savoir… Au fond, nous allons…
Mont-Saint-Guibert, le 2 décembre 2015. Photo par Peter Kurdulija.
Ce texte est publié sous la licence CC-By BE.
21 Oct 2016 8:59am GMT
20 Oct 2016
There are really cool things in Zabbix, including Low-Level Discovery. With such discovery, you can create items/prototypes/triggers that will be applied "automagically" for each discovered network interface, or mounted filesystem. For example, the default template (if you still use it) has such item prototypes and also graph for each discovered network interface and show you the bandwidth usage on those network interfaces.
But what happens if you suddenly want to for example to create some calculated item on top of those ? Well, the issue is that from one node to the other, interface name can be eth0, or sometimes eth1, and with CentOS 7 things started to also move to the new naming scheme, so you can have something like enp4s0f0. I wanted to create a template that would fit-them-all, so I had a look at calculated item and thought "well, easy : let's have that calculated item use a user macro that would define the name of the interface we really want to gather stats from ..." .. but it seems I was wrong. Zabbix user macros can be used in multiple places, but not everywhere. (It seems that I wasn't the only one not understanding the doc coverage for this, but at least that bug report will have an effect on the doc to clarify this)
That's when I discussed this in #zabbix (on irc.freenode.net) that RichLV pointed me to something that could be interesting for my case : Alias. I must admit that it's the first time I was hearing about it, and I don't even know when it landed in Zabbix (or if I just overlooked it at first sight).
So cool, now I can just have our config mgmt pushing for example a /etc/zabbix/zabbix_agentd.d/interface-alias.conf file that looks like this and reload zabbix-agent :
That means that now, whatever the interface name will be (as puppet in our case will create that file for us) , we'll be able to get values from net.if.default.out and net.if.default.in keys, automatically. Cool
That also means that if you want to aggregate all this into a single key for a group of nodes (and so graph that too), you can do something always referencing those new keys (example for the total outgoing bandwidth for a group of hosts) :
grpsum["Your group name","net.if.default.out",last,0]
And from that point, you can easily also configure triggers, and graphs too. Now going back to work on some other calculated items for total bandwith usage for a period of time and triggers based on some max_bw_usage user macro.
20 Oct 2016 10:00pm GMT
The third day is already over! I'm just back at home so it's time for a last quick wrap-up before recovering before BruCON which is organized next week! Damien Cauquil started the first batch of talks with a presentation of his new framework: "BTLEJuice: the Bluetooth Smart Man In The Middle Framework".
As the title says, the presentation focused on the BlueTooth LE ("Low Energy") protocol. Damien started with a few slides to review the specs of this protocol. It is designed to be used by low power devices. That's why it is mainly used in embedded devices (IoT). Two important terms are: the peripheral (which accept only one connection) and the central which accepts more connections and is some kind of relay. BTLE uses the same base band as the WiFi (2.4Ghz) and communications are encrypted (via the Security Manager Protocol). Damien explained how the pairing process is performed and how the devices use the different channels available. An important aspect is the ability to sniff the traffic. Not so easy because we must list to all advertising channels to be sure to get all the conversations. You've to be lucky to intercept a connection request ("CONNECT_REQ"). And, as the same frequency is used, WiFi devices may cause false positives.
The next part of the presentation was dedicated to Damien's framework: BTLEJuice. Written in Node.js, it has the following features:
- Live GATT operations and sniffing
- Burp-like intersection and manipulation
- Web UI
- Text and JSON export
- Python & node.js bindings
Some demonstrations were performed. The first one was just sniffing some traffic. The victim was a smart lock that can be unlocked via a smartphone and an app. After sniffing the traffic, we see that the PIN code is exchanged in clear-text mode and that the lock authentication is just based on the BD address. The next one was a replay attack against a small robot. It's easy to send commands to make the robot "beep". Finally, an injection attack was demonstrated based on a blood glucose monitoring system. Can you imagine sending false data without the device itself?
Finally, Damien explained how to detect / block BTLEJuice. Applications should use timing detections (read or write operations take longer than expected when intercepted by the application). Another best practice is to keep the BTLE connections available only when required. The framework is available on Github.
The next presentation was "Where to host my malware?"
Attila presented "Where to host my malware?" by Attila Marosi. It was the same kind of presentation: abusing some hardware. Attila made a research and discovered that a specific malware is infecting specific NAS devices. How? The malware, called Mal/Miner-C, is distributed via open FTP servers. The first step was to identify such vulnerable system. A good start is to use online tools like shodan.io or netdb.io. Attila searched for open FTP servers, than the ones with anonymous access and finally the ones with writable access. Thousands of devices were identified and 70% (!) of them were already infected by Mal/Miner-C. Vulnerable devices are the Seagate Central NAS. On those devices, accounts can't be disabled and if remote access is enable, anonymous access is also activated by default. Based on the capacity of the devices, Attila estimated that the online available storage space was 766TB! The malware drops the following files: photo.src and info.zip. Available webpages are also infected. The next Attila's target was the CCTV solution NetVU and finally some smart street lighting systems. Crazy to see such devices so badly protected! For those who are interested, Attila wrote a nice blog post on the Sophos website.
After a welcome coffee break, Marion Marschalek came with another awesome malware research titled "Interesting malware, No I'm not kidding". Marion's job is to track malware's and when she found one, it is analyzed from A to Z.
If we have masses of malware for windows, most of them are boring and the analyze is always the same. The malware presented by Marion, called "Cheshire Cat", was different. First of all, only 5 samples were detected between 2002 and 2011. Analyzing a malware from 2002 can be challenging because the operating systems internals were different. Marion explained in details how the malware worked: browser injection, C2 communications, key logging, file system enumeration, etc. I liked Marion's conclusion: if you are interesting enough, you could get a malware dedicated to you
Then, Russell Mattioni, from ENISA, presented "Enhancing infrastructure cybersecurity in Europe". By infrastructure, we mean transports, finance, e-health services, smart grids, … Russel presented the ENISA and its daily work to improve the security of infrastructures in Europe. If the talk contained interesting information, it was way too theoretical. IMHO, it was not appropriate for an audience like hack.lu. We all know what are the issues and how to (try to) fix them.
After the lunch, another talk was given with again a physical device as the target. This time, it was the intercom systems that we can find at doors of buildings in many cities. Basically, they provide the following features: they call a resident who can unlock the building from door remotely. The title was "House intercoms attacks: when front doors become backdoors" and presented by Sébastien Dudek.
Sébastien had the idea to investigate how such devices could be abuse. Modern ones do not need cables and rely on 3G connectivity. They are different brands. Amongst them, Linkcom is the cheapest and commonly used in private buildings. They are easy to detect when a nice 3G module is installed close to the door.
Besides the intercoms, Sébastien explained how 3G networks are working. The device configuration can be performed by SMS and the configuration is stored on the SIM card. A first demonstration was to hijack the number stored in the intercom with your own number. You'll then receive calls from visitors, be able to talk to them and unlock the door.
Some intercom's use M2M networks ("Machine 2 Machine"). In this case, there is a centralized server, which means a broader attack surface! Re-use the SIM card in your computer and configure the same M2M network settings then attack the infrastructure. Some of you could also have free Internet access. Note that a botnet of intercom's device could be build to call a premium-rate number for make some profit.
Mahsa Alimardani was the next speaker and came back with a talk that ("A push towards practice informing technology") was really close to the one she gave last year.
I left the event earlier and did not attend the last talk. The 2016 edition was a good one, I met new people and had great times with the already known too. Keep an eye on the archive page to get a copy of slides. Finally, I would like to thank all the readers of my wrap-ups. Feedbacks are always very positive! Stay tuned for more soon
20 Oct 2016 6:58pm GMT
A few months ago, I gave a presentation at LaraconEU in Amsterdam titled "Varnish for PHP developers". The generic title of that presentation is actually Varnish Explained and this is a write-up of that presentation, the video and the slides.
The simplest way to get a grip of the basics of Varnish would be to watch the video recording of that presentation.
Further down you'll find the slides and a transcript of each slide, with accompanying links.
Varnish Explained: presentation video
This is the recording of the talk at LaraconEU on August 2016.
Here's a write-up per slide, with the most important topics.
(Alternatively, if you prefer to just skim through the slides, they are up on Speakerdeck too.)
In order to fully understand Varnish, it's vital that you understand the HTTP protocol (at least the basics) and how its HTTP headers are formed and which purpose they serve.
Later on, I'll cover the Varnish internals, cool acronyms like ESI, Grace, TTL & more.
A quick introduction: I'm Mattias Geniar, I'm the Support/Operations manager at Nucleus Hosting in Belgium.
Part One: What's Varnish?
Varnish can do a lot of things, but it's mostly known as a reverse HTTP proxy. It brands itself as an HTTP accelerator, making HTTP requests faster by caching them.
It can also serve as a TCP load balancer (using
pipe in the VCL code, more on that later), an HTTP load balancer or a generic reverse proxy.
It uses the Varnish Configuration Language (VCL) to allow you as a developer or sysadmin to modify the behaviour of Varnish and to play with the HTTP protocol to shape it so it behaves the way you want.
Varnish is also extendable with VMODs, modules that can be loaded by varnish and provide additional functions/methods and functionality you can call upon in your VCL configurations. Think of these like PHP extensions; binary files that are loaded by Varnish that extend the userspace in which you can create VCL configurations.
A couple of important versions: Varnish 3.x is probably the most widely used, but is now EOL (so: no more security updates). Varnish 4.1 is still supported and Varnish 5 came out a few weeks ago (so the slides are slightly outdated).
If you're upgrading from Varnish 3.x to later: beware, the configurations changed drastically. You'll need to update your syntax/methods in Varnish, or Varnish won't start.
Using Varnish in your infrastructure gives you a couple of advantages: performance, scalability, control & security.
Varnish is usually associated with performance, but it greatly increases your options to scale your infrastructure (load balancing, failover backends etc) and adds a security layer right out of the box: you can easily let Varnish protect you from the httpoxy vulnerability or slowloris type attacks.
Part Deux: HTTP Headers
Let's talk HTTP headers.
A quick summary of HTTP headers: there are request and response headers (basically what your User Agent (=browser) requests and what the Server (Apache/Nginx) responds) and Varnish listens to these by default to determine if requests can or should be cached.
Whenever your browser goes to a certain website, it'll a do a few low-level things: it'll resolve the DNS of the hostname you want to browse to, open a TCP connection to that IP and start sending the HTTP request headers.
They are basically a new-line separated key/value pair. The concepts are the same in HTTP/2, except it's binary and more structured.
The browser describes the kind of response it can receive (plain text, compressed), what site it wants to load (because remember, the browser just connected to the IP, it now needs to tell the webserver which website at that IP address it wants to load).
If all goes well, the server will respond with a similar set of headers: the response headers. It will confirm the HTTP protocol (or down/up-grade the protocol), give a status code and describe the response (plain text / compressed).
Good, that was the HTTP reminder -- now let's get to Varnish.
Part Three: How does Varnish work?
Varnish is a reverse proxy, so it sits between the User Agent (the browser) and the webserver. It makes decisions to either deliver a cached version of the page, or send the request to the backend webserver for processing.
By default, it'll do so by listening to the HTTP headers the client sends as well as the ones the server responds with. But, those usually suck -- creating the need for proper Varnish templates to avoid boilerplate code being re-created all the time.
In a normal scenario, the browser probably connects directly to the webserver, which in turn will let PHP execute the request. The simplest possible setup.
When Varnish is introduced, a few things happen: the webserver is hidden to the user, as that user now only connects to Varnish. Varnish has bound itself on port
:80 and the webserver is either running on a different server or on an alternative port. It's Varnish that will make the connection to the webserver if needed, the user has no idea.
If you follow the official Varnish book, the "internals of Varnish" are described as this. While it's 100% correct, if you're new to Varnish, that image does you no good: it's scary, confusing and it teaches you nothing.
Instead of such a flowchart, I prefer to visualise Varnish slightly different.
Let's start with a basic example: a user tries to connect to a server running Varnish. It'll make a connection to the port where Varnish is running and Varnish can start to process the request.
It'll trigger a routine in Varnish called
It's a routine where you can write custom VCL code to manipulate requests, determine backends, redirect users, ... it gives you full control over the HTTP request of the user.
After that routine, Varnish can do a cache lookup: the page the user is requesting, does it already exist in the cache?
If it does, it can serve that page to the user. Before it does that, it triggers a new internal routine called
vcl_deliver(). It's another place in Varnish where you can manipulate the request, change the HTTP headers etc.
Once that request is finished, Varnish sends the response to the user so he can render it on screen.
Of course, not everything is a cache hit: if Varnish does a cache lookup but finds it doesn't have that object/request in the cache, it has to go and fetch that content from its own backend.
To do so, it'll trigger another internal routine called
You guessed it, it's yet another place where you can further manipulate the request before it gets sent to the backend webserver(s).
That routine will eventually call out to another webserver and have it process the request the user made.
Once the response was created on the backend webserver, Varnish will receive it and fire another routine:
Hey, yet another place to play with the HTTP protocol and change the headers etc. before Varnish will process the request internally.
Varnish can then store the item in its cache (if it's cacheable) and deliver it to the client, via the same
Pfew, what a ride!
Part Four: The Varnish Configuration Language
Now that you have a visual of the flows within Varnish, let's see what the configurations look like.
It looks like the screenshot above, where routines are described using the
sub keyword. For a complete example, have a look at my varnish 4.x configuration templates.
Each of those routines I described earlier allow you to customise the HTTP request with that VCL language.
To do so, it offers you a few useful objects:
req.http, ... Each of those objects represent a part of the HTTP protocol. There's
req.method to indicate a GET, POST, PUT, PURGE, ...
req.http which contains all the raw HTTP headers sent by the client, so there's
req.http.host for the Host-header,
req.http.User-Agent for the User-Agent (Chrome/Firefox) of the client, ...
Just like the HTTP protocol itself, the HTTP headers are case insensitive. So
req.http.HOST is the same as
req.http.host. But do yourself a favour, pick a coding/style standard you like and stick to it.
In the example above we tell Varnish, in the
vcl_recv() routine (the very first routine to be called by Varnish) to only deal with GET or HEAD requests. Anything with a different method (a POST, PUT, ...) will be sent to the backend webserver(s) via the
return (pass); command, indicating that the request should be passed on to the backend.
Further down, we manipulate the cookies the client has set. This is probably the part where Varnish has thought me the most about the HTTP protocol! As a user, every cookie that is set on a particular domain gets sent to the server for every request. It's a single string, semi-colon delimited. You can change the value of cookies in Varnish, remove or add cookies, etc.
You can also use Varnish to be a "dumb load balancer": it can look at the Host header sent by the user and redirect requests to different backends.
This would allow you to set up a single Varnish server which can send requests to multiple backend servers, depending on the site the user is requesting.
set req.backend = xxx; code instructs varnish to use a particular backend for that HTTP request.
A common task in Varnish is dealing with Cookies (more tips on that later by the way) and removing cookies for objects or HTTP requests that don't need them.
Alternatively, we could also modify the response the webserver has sent us, in case it was a cache miss.
If a webserver wants to set a new cookie, it'll send along one or multiple
Set-Cookie headers. In Varnish, we can remove those headers by removing the
bereq.http.Set-Cookie header, making it look like the webserver never even sent those requests in the first place!
Another interesting trick is to tell Varnish to never cache a response when it's an server error, something with a status code in the HTTP/500 range. By telling it to reply with
return (abandon); it'll bail out that routine and exit the flow.
vcl_deliver(); routine you have a final place, right before the HTTP response is sent to your user, to add or remove HTTP headers.
I commonly use this to clean up the request: remove sensitive headers (like Apache or PHP version numbers), add some debug headers to indicate if it was a cache miss or hit, ...
Having so many places to tinker with the request and response can make it hard to visualise where changes were happening and what the outcome would/could be. It can also make you question what's the best place to even begin to manipulate such a request.
To be honest: there's no right or wrong answer here. It depends on your setup. But, there are a couple of tricks to help work with Varnish.
For example, let's run through a request in Varnish and see where/how we can play with it.
In this example, a user wants to connect to our server and load the laravel.com homepage. It sends a request to load the
/ page (homepage) indicating that it's the Chrome User-Agent.
When Varnish receives that request, it'll start its
vcl_recv routine, we can manipulate the request.
For instance, we can change the request so that it no longer wants to load
forge.laravel.com, by setting a new value in the
Important to know here: the user/client has no idea we're doing this. This is purely internally in Varnish. For the user, it still thinks it's going to receive a response for
Varnish will use that new information (the modified
req.http.User-Agent) to check its cache if it has a response it can serve to the client. In this example, we'll assume it was a cache miss.
So now Varnish has to fetch the content from its own backend webserver. But because we modified the request in
vcl_recv(), Varnish will ask the backend for the website behind
forge.laravel.com, claiming to be an IE5 User-Agent.
The webserver does its thing, and if all goes well it can generate the homeapge for
It'll respond with an
HTTP/200 status code, indicating everything was OK. It will also set a PHP session ID, tell that it was generated with the Apache Webserver and indicate that this HTTP response should not be cached, by setting a
Cache-Control: No-Cache header.
Varnish receives that response and triggers its routine
vcl_backend_response(), where we can modify what the webserver sent us.
Even though the webserver said we should not cache the page (remember:
Cache-Control: No-Cache), and Varnish would by default listen to that header and not cache the page, we can overwrite it by setting the
beresp.ttl object. The TTL -- or Time To Live -- determines how long we can cache that object.
Additionally, we will overwrite the status code that the webserver sent us (
HTTP/200) with one that we invented (
And finally, we'll remove that Laravel session ID that was generated server side by removing the
After all that fiddling with the HTTP response, Varnish will store the object in its cache using our modified parameters. So it'll store that object as an
HTTP/123 response without cookies, even though the server generated an
HTTP/200, with cookies and clearly indicated the page shouldn't be cached.
So what eventually gets delivered to the client is a page for
forge.laravel.com (our client requested
laravel.com) with an HTTP status code of
HTTP/123 (the server actually said it was an
HTTP/200), without cookies being set.
Such Varnish power. Much amaze. So many places to fuck things up.
There's another important Varnish routine though, it's called
vcl_hash(). This is where Varnish decides which parameters it will use to determine its cache/hash key. If the key can match an object already in the cache, it will deliver that object.
By default, Varnish looks at 3 key parameters: the
Host-header, the URL being requested and the cookies sent by the client.
Behind the scenes, it will take care of the
Content-Encoding that clients can request: it'll store a compressed version of every page in memory, but if a client requests a plain text -- non-compressed -- version, it'll decompress it on the way out and deliver a plain text version to the client.
If any of the parameters inside the
vcl_hash() routine changes, Varnish will consider it a new HTTP request and the cache lookup will fail.
If you were thinking ahead, you can see a problem with that kind of hash-key determination.
For instance, the URL
/?page=1&languange=nl will show the same content as
/?language=nl&page=1, but since the order of the parameters is different, the hash will be different. To resolve this, we normalise the incoming requests as much as possible.
Varnish offers a good tool for this called
std.querysort(), which will sort the query parameters alphabetically.
Additionally, you might want to strip query parameters that are irrelevant to the backend, like Google Analytics campaign trackers.
Once you normalised the URL, cookies are the Next Big Thing.
Because they are by default used in the hash-key lookup, any change in the cookies client-side will result in a different hash and a new object in the cache.
If every one of your users has a unique PHP session ID, every Cookie set will be different and thus every hash-key will be unique. Just setting a unique PHP session ID for all your users would effectively destroy caching for your site, as every request is unique.
There are a couple of fixes for this, but the biggest one is to only initialise a PHP session when you need it (ie: on the login page), and not for every page. This is called lazy session initialisation.
Alternatively, some XSRF implementations rely on cookies and should be unique per user too.
Varnish offers an interesting VMOD (extension to Varnish) to help manipulate Cookies.
cookie.parse object allows to you do interesting things like whitelisting cookies in Varnish, without crazy mind-breaking regular expressions.
In addition, the
Host header is also a key in your hashing. So a request for
www.laravel.com, even though they're probably the same site, would result in 2 different cached objects in Varnish.
To help Varnish, it's best to redirect users to a single domain and not keep multiple possible domains that would result in the same content being generated.
There are plenty of SEO & technical reasons this is a good move too.
The code example above detects a page without the
www-subdomain and triggers a synthetic response in Varnish, via the new
In that routine, you can detect the
resp.status object and create logic handling that differently. In this case, we set a custom error code of 720 in order to replace the
Location block in the HTTP response and force a 301 redirect.
vcl_hash() can be expended to take any HTTP header into account when determining the cache key. It can add Authorization headers for basic HTTP authentication, custom headers for your API, ...
We also use it at Nucleus to separate cache responses for HTTP vs. HTTPS content, since we often use an Nginx TLS proxy in front of Varnish.
Part Five: Flushing the Cache
There are actually more than 2 hard things in computer science: naming things, cache invalidation, off-by-one errors, DNS, Docker, ...
There are basically 3 ways to flush a cache: the old PURGE HTTP method (which is basically obsoleted), the newer "ban" method (suggested) or the absolute lazy way: restart your varnish service.
The HTTP PURGE method is a "special" HTTP request where you can set the method to PURGE. That in and of itself doesn't actually do anything, but it allows you to catch that special method in varnish's
vcl_recv() routine (the very first routine Varnish will call when it receives a request) and trigger an internal cache flush via the
return (purge); call.
The preferred way is actually to use "bans": this uses the Varnish Administration CLI tool called
varnishadm to connect to a running Varnish daemon on its administrative port and send it control commands.
This syntax gives you more flexibility to flush content based on regex's, either on the host, the URL or arbitrary headers.
Part Six: ESI, TTL's, Grace & Saint
More acronyms! More!
Edge Side Includes are a very powerful but tricky feature of Varnish. By default, Varnish will either cache an entire page, or not cache an entire page.
With ESI, Varnish can cache parts of a page. Funky, right?
In a way, this works very similar to PHP's
require method or the old Server Side Includes in webservers.
Take for instance a default layout for a news website: it'll contain a personal greeting at the top, some ads, some navigation and the main body.
These different "fragments" that usually build a page are what Varnish needs to be able to do partial caching.
Instead of generating the content for each partial directly in the HTML/DOM of the page, you generate special
esi:include tags instead.
Eventually, your server-side generated HTML looks like this. It has several
esi:include blocks, where it would normally have either include/require statements from PHP. But instead of having PHP do the combining of the page, those
esi:include tags are going to be interpreted by Varnish.
When Varnish processes a
vcl_backend_response(), it can look inside the body and detect those
For each of those ESI tags, it will -- on its own -- send a new request to the backend and request those URLs. The backend will in turn process them, reply them back to Varnish, and Varnish will build the page by glueing all the fragments together.
The biggest benefit is that each fragment can have its own caching policy: in the best case scenario, all those
esi:include calls were already in the Varnish cache and it could assemble the page entirely from memory.
The magic of ESI is that the user just requested a single page, it had no idea that it would in turn fire of multiple small requests to the backend to assemble that page before getting it delivered.
Now, by default, Varnish will look at the
Cache-Control headers (or ETAG, Expires, ...) to determine how long it can cache an object. In the absolute best case scenario, the HTTP response headers determine the lifetime of each HTTP response.
But you can artificially overwrite that response by setting a new value in the
The TTL or cache lifetime can be modified even further by introducing the concept of Grace Mode.
You can let Varnish keep objects in its cache that have actually surpassed their lifetime. This is useful in case your backends are down; it's better to serve old content, than no content.
It's also of value when there are HTTP requests coming in for expired objects: Varnish can reply with a slightly outdated response, while asynchronously fetching a new version from the backend. Your user never needs to wait for that response (it got a slightly outdated response), while your webserver can generate the response.
This is absolutely a lifesaver in cache stampedes scenario's, where Varnish will only send a single request to the backend, even though 100+ users are requesting the same page.
Bonus: CLI tools
Quick demo: some CLI tools.
varnishhist shows a histogram of the requests currently being served by Varnish: the more to the left they are shown, the faster they have been served. These are generally more cache hits, as they're served the fastest.
The more items to the right, the slower each request is.
A pipe sign
| is a cache hit, a hashtag
# is a cache miss (can also be a cache miss on purpose, if the VCL code says
varnishhist is a good quick glance to see if Varnish is caching anything and if so, what the overal ratio is: lots of cache hits or misses. But beyond that, the tool probably won't tell you that much.
varnishlog on the other hand is the real gem: it will show in great detail the request headers of a user and the response headers returned by Varnish. If you really need to debug Varnish, this'll be the tool that tells you the most.
There are filters you can apply to only log requests from a certain IP (ie: your source IP), only log requests matching a certain URL pattern, you can see which cookies are being stripped in the VCL and which aren't, etc.
The only downside:
varnishlog can be a bit daunting at first with lots of output, it takes a while to grok the format.
Another interesting one, especially when just getting started with Varnish, is
varnishncsa: it takes the complicated
varnishlog and shows the output just like the Apache or Nginx access logs.
It won't tell you many details (like which cookies get stripped, which VCL routines got triggered etc.), but it will tell you in a more readable format (or at least: a format you're more used to) which requests Varnish is serving.
You can also run varnishncsa as a daemon/service to log all requests made by Varnish in this format, a bit like the default behaviour in Apache or Nginx. For debugging purposes, this could be useful.
Part Seven: getting started with Varnish
Let's get started.
This is the really short version of getting started with Varnish;
- Download my Varnish templates from Github
- Install varnish, by default it'll run on a custom port (check
- Configure your backend so that it points to your existing Apache/Nginx
- Test your site on the custom port:
- If everything still works, swap the ports: set your Apache/Nginx on port :8080, move Varnish to port :80 and change the backends in Varnish so that it sends its requests to port :8080.
After that, you're done: Varnish now serves all requests on port :80 and will proxy everything it can't serve to port :8080.
Chances are, it'll take longer to implement Varnish than just those couple of steps, but that's basically the gist of it. Run Varnish on an alternative port, so that you can test it safely without interfering with your site, and once it's tested and approved, swap out the ports and let it run on port :80.
A couple of tips, that might help you get started quicker;
- Varnish Agent: an HTML frontend to manage & monitor your varnish installation
- Syntax highlighting of VCL files in Sublime Text
- Troubleshooting Varnish on systemd, if it won't start
- Some useful varnishlog oneliners to help troubleshoot
As with everything: testing, testing & testing is what will make Varnish deployment a success!
For any questions, leave a comment on this blog, poke me on Twitter, send me an e-mail, ...
20 Oct 2016 1:33am GMT
19 Oct 2016
I published the following diary on isc.sans.org: "Spam Delivered via .ICS Files".
Yesterday, I received a few interesting emails in my honeypot. I set up catch-all email addresses for domains that are well known by spammers. I'm capturing emails and extracting MIME attachments for further analysis. Today, my honeypot received three ICS files. iCalendar is a file format used to exchange meeting information between users, mainly via email or a file sharing system… [Read more]
19 Oct 2016 10:47pm GMT
I'm just back from the second day of hack.lu. The day started early with Patrice Auffret about Metabrik! Patrice is a Perl addict and developed lot of CPAN modules like Net::Packet or Net:Frame. That's why he had the idea to write a new UNIX shell working like a Perl interpreter.
So, Metabrik was born. It's not only a shell but a true language that can be empowered with "briks". A brik is a development/prototyping platform used to quickly build your own tool. You can also see it as a wrapper around existing tools.
Patrice's philosophy is: "Everything should be possible from the CLI" but in an automated way and normalized syntax and in a human readable form. And indeed, all external tools are interfaced in the same way and results provided in the same format.
To demonstrate the power of Metabrik, Patrice performed some demonstrations. The first one showed how easy it is to create an array and use CPAN module directly from the shell. So easy! At the moment, Metrabrik has 200+ briks available. I'm sure you will find something interesting. Others demonstrations were performed. Personally, I liked the malware analysis automation. Patrice interfaced VirtualBox with Metabrik to spawn a guest VM, execute the malware and grab results (ex: by performing a memory acquisition with Volatility). If you are interested, have a look at his blog post here. Metabrik is a very nice tool but it will require some time to master it. The only comment that I received while discussing with friends about it was: "Why in Perl and not in Python?". Please, no flame war!
The second slot was assigned to a master speaker: Saumil Shah. I don't need to present him, he's a regular speaker at hack.lu. Every time, I came on stage with a nice presentation. This time, in 2016, it was less technical. The title was "2016:The Infosec crossroads". It was more a reflexion about the defence. Everything started with a fact: "Today's attacks succeed because the defence is reactive".
We have to be honest: attacks evolve way faster than defences. They are shifting regularly between different targets depending on opportunities (servers, workstations, IoT, …) because attackers just follow the money. If there is a change to gain more money, attackers will adapt their techniques. If today's fashion is focusing on breaches, attackers don't follow rules.
Another fact: the defenders tried to buy back their bugs via bug bounty programs. It's not a game anymore but a business. It's a zero acquisition market. Then, Saumil talked about the (d)evolution of users. Advanced technologies are advanced and if you don't know how to use them, it may become dangerous. What is the good reactive approach to defence? For Saumil, compliance and security are not the same. Attackers don't follow rules. Today's infosec defence remains based on rules, signature updates and machine learning. Existing strategies do not match attackers tactics. We have to switch from a reactive to a proactive approach! To achieve this, Saumil presented his own seven axioms:
- Collect everything: build a security data warehouse, retention is cheaper than retention,
- Can't measure? Can't use it!
- Proper Pentesting (are you more scared by pentesters then auditors?)
- User ratings
- Set booby traps
- Analysis decides actions
- Buy-in from the top
It was a very nice presentation but, let's be honest, very difficult to put in practice. From both sides, money is the driver! Attackers want to make more money when defenders are asked to reduce costs!
Today's keynote was presented by Quinn Norton: "A network of sorrows, Small adversaries and small allies" or a status of the Internet… It started with a nice picture: Attackers are everywhere!
Quinn reviewed several issues on the Internet today and how people/organization react to security issues. Example: many schools and hospitals are hit by ransomware in the US. It seems also that teenagers are more concerned about being spied by their parents than by the NSA. The Internet introduced new technologies that are not well understood by users. This is a classic behaviour:
No you won't have my SSH key but I can give you access to my desktop via TeamViewer!
Keep in mind that users must be educated to face the changing world. We ask our users to listen to our advices but we should also listen to the users. They use tools (and the security issues related to them) to solve their problems.
After a good lunch and discussions with peers, there was a second set of lightning talks but I did not attend them, too busy to exchange with peers (that's also the goal of security conferences!). The next talk focused on SAP and how to exploit it via default accounts.
This was presented by ERP-SEC. For most of us, SAP is a boring environment but being using by all of the big players in many business fields and the data stored in SAP systems being highly valuable, such environments are good targets. Of course, patches and security fixes are released by SAP but patching means downtime and downtime means unavailability which means loss of money. So, systems are not often patched. Two big attack vectors for SAP have been identified by ERP-SEC:
- Default accounts
- SAP RFC gateway (then pivoting)
A SAP system may have many default accounts, the first method will be the preferred one. One slide full of default usernames and passwords was displayed. A fact reported by ERP-SEC: 100% of tested SAP instances had at least one default account. For customers, the key question is: "Are those users in my system?" Most of them have! Indeed, the SAP Solution Manager may require new accounts when some features are activated like enabling some monitoring features in the product. An important remark is that a default account alone does not immediately mean a bug security hole. It must be combined with another vulnerability (SQL execution, SMB relay, OS command injection, …) to fully compromise a system. The second part of the presentation was based on demos. How to protect? Nothing new here: Change default passwords, delete / disable unused users if not required. There is nothing new but often not applied.
The next talk was about GPO's (Microsoft Group Policies). The idea of the speakers was to use them to deploy a malware and get persistence. The goal of the talk is to create some awareness or to give more evil ideas to pentesters. How does it start? Microsoft recommends using GPO to solve management issues. Example: to enable WMI on all workstations.
Mainly used in companies, GPO's are also badly managed:
- It tends to be messy
- They are stored everywhere
- There is a lack of naming convention
To achieve their goal, they used a well-known framework called PowerShell Empire. The first demo was about to set persistence by creating a 'RunOnce" registry key which contains the malicious exe file ("calc.exe" in this case for demo purposes). The second demo was about to find a way to quickly search for a specific file across all the workstations. To achieve this, they created a new GPO that enable WMI and open the local firewall to allow the attacker to connect to the victim. It's a nice tool and they are for sure plenty of way to use it for malicious purposes. So, how to protect you? There is no all-in-one solution. To protect yourself, you must review GPO's in place, limit admin privileges, monitor the activity on the network and, of course, keep your environment healthy.
Then, Clarence Chio came to present "Machine Duping - Pwning Deep Learning Systems". Machine learning, or deep learning, or "artificial intelligence" refers to computers which can take decisions by themselves (to briefly resume). The idea of Clarence was to search for a way to pwn them, read: to make them take wrong decisions. As such kind of computer will be more and more used, it is interesting to see how they can be abused.
The presentation explained in details how those computers can be abuse but it was purely theoretical (too much) for me. Clarence looks to really master his topic!
After the second coffee break of the day, the last wave of talks started… Four in a row! That was quite intensive… Matt Weeks came to present "Credential Assessment: Mapping Privilege Escalation at Scale". Matt demonstrated that in major breaches, often the complete network is compromised, data are stolen. He reviewed some nice stories like Target, Home Depot of JP Morgan Chase. Sometimes the attack is more destructive like Sony or TV5-Monde. Usually, a data breach is based on multiple techniques: social engineering, modification of a malware to evade controls in place, etc.
What do they have in common? Credentials were (ab)used to gain access to the infrastructure. In a second part, Matt explained how to protect credentials to not be used in such attacks. But, this must be combined with other security controls. Lessons learned demonstrate that often:
- The corporate network was open to contractors
- There was no response from anti-intrusions or monitoring tools in place
- There is a lack of network segmentation
Matt explained how to track credentials and how to enforce security controls. What must be performed:
- Identification of credentials dissemination
- Identity reused
- Identify impact of credentials
It is mandatory to determine mechanisms to break the chains and to clean up credentials. Data must be collected as much as possible with privileges assignments, group membership, etc.
The next slot was assigned to two guys from Checkpoint (Yaniv Balmas & Ben Herzog). They presented "When crypto fails" or "Finding cryptographic bugs for mere mortals". Checkpoint guys are also regular speakers at hack.lu. Crypto is everywhere and more and more malware implement cryptography today. They are right when they say that cryptographic bugs are not in the crypto algorithm but un the way the developer implements it. Sometimes, a bug can be found in the crypto library. In this case, they recommend to disclose it privately.
After this introduction, the second part of the talk was a review of different badly implemented crypto in pieces of malware. The first one as Zeus, the well-known banking malware. Many malware use RC4. It is very popular, not complex and easy to implement. But… why do the Zeus developers decided to implement their own algorithm, called ZRC4? They called this example: Voodoo programming.
The next example was based on Linux.Encoder which was one of the first ransomware targeting Linux hosts. It uses PolarSSL\mbed-TLS libs for encryption tasks. To provide randomness, it uses the following code to encrypt files: SRAND(time(NULL)). What's bad: encrypted files were created with a new timestamp, so they were easy to decrypt. Keep in ming that when you use time-based seed, the time can be used again to decrypt. The last one was Petya, the ransomware which encrypts also the MBR and finally the Nuclear Exploit Kit.
What's bad: encrypted files were created with a new timestamp, so they were easy to decrypt. Keep in ming that when you use time-based seed, the time can be used again to decrypt. The last one was Petya, the ransomware which encrypts also the MBR and finally the Nuclear Exploit Kit.
The next talk was presented by Jacob Torrey: Bootstrapping an architectural research platform in 60 mins. I bypassed this one…
The day ended with a presented about Hadoop, the tool developed by Apache. Here again, the title of the presentation induces a false idea. The presentation was about abusing Hadoop. After an introduction to Hadoop and its infrastructure (which is quite complex), Thomas & Madhi explained how the framework is badly configured (by default).
Then, they reviewed the Hadoop security model which is very "open". By default, there is no authentication: "simple" means none but there is an alternative based on Kerberos. About authorization and auditing: all components implement their own model. This make it very complex to maintain. Data encryption is available but, here again, disabled by default. If you try to build an attack surface of an Hadoop instance, you will see that many TCP ports are used! The rest of the presentation was based on demonstrations that prove that the security model is by default inexistent! More scary, there are Hadoop instance available publicly available. IMHO, the conclusion to this talk is: Hadoop is a mess!
That's it for today! Same place for the third wrap-up tomorrow…
19 Oct 2016 10:42pm GMT
I had to convert a long PDF to multiple PNG files, and it's pretty easy with ghostscript at the terminal.
First, install Ghostscript using Homebrew.
$ brew install ghostscript
Next up, convert the PDF to PNG files.
$ gs -dNOPAUSE -sDEVICE=png16m -r256 -sOutputFile=page%03d.png input.pdf
A quick rundown:
-sDEVICE=png16m: the convertion to use, in this case convert to a 24-bit RGB color png file. If you value transparancy, use
-r256: the dimensions of your png file, the smaller the number, the smaller the size of your png's. To give you an idea,
-r256comes down to 2816×2176 pixels.
-sOutputFile=page%03d.png: number each page of the PDF as "pageXXXX.png", where XXXX is the number of the page.
input.pdf: the PDF you want to convert.
Good ol' Ghostscript saves the day, yet again.
19 Oct 2016 7:39pm GMT
I wanted to share the exciting news that Nasdaq Corporate Solutions has selected Acquia and Drupal 8 as the basis for its next generation Investor Relations Website Platform. About 3,000 of the largest companies in the world use Nasdaq's Corporate Solutions for their investor relations websites. This includes 78 of the Nasdaq 100 Index companies and 63% of the Fortune 500 companies.
What is an IR website? It's a website where public companies share their most sensitive and critical news and information with their shareholders, institutional investors, the media and analysts. This includes everything from financial results to regulatory filings, press releases, and other company news. Examples of IR websites include http://investor.starbucks.com, http://investor.apple.com and http://ir.exxonmobil.com -- all three companies are listed on Nasdaq.
All IR websites are subject to strict compliance standards, and security and reliability are very important. Nasdaq's use of Drupal 8 is a fantastic testament for Drupal and Open Source. It will raise awareness about Drupal across financial institutions worldwide.
In their announcement, Nasdaq explained that all the publicly listed companies on Nasdaq are eligible to upgrade their sites to the next-gen model "beginning in 2017 using a variety of redesign options, all of which leverage Acquia and the Drupal 8 open source enterprise web content management (WCM) system."
It's exciting that 3,000 of the largest companies in the world, like Starbucks, Apple, Amazon, Google and ExxonMobil, are now eligible to start using Drupal 8 for some of their most critical websites. If you want to learn more, consider attending Acquia Engage in a few weeks, as Nasdaq's CIO, Brad Peterson, will be presenting.
19 Oct 2016 2:59pm GMT
18 Oct 2016
I'm back to Luxembourg for a new edition of hack.lu. In fact, I arrived yesterday afternoon to attend the MISP summit. It was a good opportunity to meet MISP users and to get fresh news about the project. This first official day started later and in a classic way: with a keynote. The slot was assigned to Alice Hutchings and was named: "Stressed out? Denial of service attacks from the providers' perspective".
Alice is working for the Cambridge Cybercrime Center Computer Laboratory. The keynote title did not reflect the exact content of the keynote. It was not covering the DDoS aspect from an "Internet providers" point of view but from the DDoS services providers, read: the bad guys. The idea of the keynote was "know your enemy!".
Alice started with an example of a website providing DDoS services to anybody. A nice look, support, description of the different types of supported attacks, the different membership plans. Looks very professional. Those guys are called "booters". The Alice's research was to study who they are, how they work. She started with cybercrime trajectories: the theory of initiation, maintenance and desistance. First of all, they are techies who know how to attack, how to abuse. It starts always with opportunities. Two pathway:
- The general pathway: the initiation goes from strain to presented opportunity
- The Technical pathway: the initiation goes from associations with others to learn skills and neutralisations
For both, the maintenance is mainly low likelihood detection and benefits relating to offending. Finally, the desistance is when costs outweigh benefits
The biggest part of the study was to organize an online survey. Once a list of booters was identified, she sent invitation request and, first surprise, 25% of them responded (positively as well as negatively). Two of them even accepted an interview. Based on the information gathered, what she learned from the respondents? They are often young and are active from a few months up to a few years. And, what a surprise, their main motivation is to make easy money. Alice release all details of her research in a document called "Exploring the provision of online booter services" (available here). An interesting fact: those guys do not realize that they provide illegal services. For them, it is a normal business.
Then, a first set of lightning talks was organized. Amongst other, I noted:
- A cool demo of Metabrik (they will be the topic of a talk tomorrow)
- A very nice exploitation (RCE, XSS) of a Kerio UTM appliance ("when your firewall turns against you")
After the lunch, the regular talks started again with "Exploiting and Attacking Seismological Networks… Remotely" by James Jara. Why seismological devices? Because it's not a juicy target for most people and it may be an idea for cool scenarios. Indeed such devices are not only used to detect earthquake but also by oil & gas companies to find new sources of combustible.
Like for other devices, they are different vendors, Nanometrics being the biggest one. The devices are Linux based, provide remote management facilities (SSH, Telnet, FTP), a web server, GPS chop, good batteries or solar panels. Of course, they are connected to the Internet to be able to send collected data to seismologic centers. James explained how he found devices. Via netdb.io, a site similar to shodan.io but more focusing on IoT devices. A nice story, to get a copy of the firmware, he just sent a mail to the manufacturer askin politely a copy… and he received one! Everybody know that once you get a copy of the firmware, the most difficult step is completed. It's then easy to analyze it to extract keys, passwords,e etc. James performed lot of demos (some recorded, some live - which is, IMHO, a little bit borderline). Nice research but the audience was not the right one. The vulnerabilities found are the same as any other IoT device. It would be interesting to present the same research to scientific people!
The next talk was "Secrets in Soft Token - A security study of HID global soft token" by Mouad Abouhali.It was a nice research about a specific 2FA implementation based on the HID soft token app. He started with a review of a classic 2FA implementation. Today, many solutions can be installed on smartphones. We don't need hardware tokens which are difficult to manage/deploy and expensive. If the HID solution is available on multiple mobile operating systems, the research focuses on the Android version. The idea was to analyze what may happen when a device is stolen, how the application is protecting itself, how data is stored and how cryptographic features are implemented. If the application looks quite well protected (code is obfuscated, it does not run in a jailbroken environment), the code was successfully analyzed by Mouad's team: they read the code, debugged it and rebuilt it. They also analyzed the behaviour to generate logs and analyze them.
The next step was to understand the encryption tasks. Where is stored the PIN? A first vulnerability was discovered: it is possible to clone the application by copying the HID config files and to gather the Android_id secure attribute. But the attacker still needs the user's PIN code (that can be collected using social engineering techniques). The second vulnerability was to discover the PIN. The develop a tool to do this and it seems to be quite useful. Mouad explained in details how it works. That was a great research performed in one month. They reported the findings to the vendor which is busy with a new design of its application.
Then, "KillTheHashes, 30 million malware DNA profiling exercise" was presented by Luciano Martins, Rodrigo Cetera & Javier Bassi. It started with a fact: Today, the identification of a piece of malware only with its hash is a fail.
Luciano gave several examples of well-known malware (Stuxnet, Zeus, Dino). All of them have specific characteristics like a file name, a system call, a DLL name. The idea behind this talk is to classify the malware based on this information. To achieve this, a tool was developed which provides:
- A DNA profiling search engine that discovers malware patterns and characteristics.
- File type search capabilities
- A framework to search malware
- Large scale malware analytics
Another example, most popular malware code sections are the TSULoader, NullSoft PiMP stub or the 7zip extractor. They can also be used to identify malware samples. Based on this, the malware universe map was created:
The tool is called CodexGigas and is available here. This is definitively something that you have to test if you're dealing with many malware samples.
After a well-deserved coffee break, Wayne Huang & Sun Huang presented their analyze of a malware campaign: "Unveiling the attack chain of Russian-speaking cybercriminals". The botnet is called Asprox and is online since 2007. It targets Windows and Android devices (since 2014). Being operated by Russian speaking guys, it is delivered via classic spam emails. If the delivery is classic, some interesting checks and features are implemented like a delivery testing (they verify the domain reputation via online services like scan4you.org or avdetect.com). Waves of spam are also scheduled by geographic areas. The attack chain is classic:
Victim > Compromised Server > TDS Server > Botnet > NGinx Reverse-Proxy > C2 > Attacker
The explained in details how the botnet works, what are the techniques used to defeat the security researchers' job but also the business around the botnet. Some techniques to make the analyst's life more difficult are really evil. Example with the NGinx reverse proxy: the configuration with C2's IP addresses is generated, the proxy started and the config immediately deleted and replace by a fake one.
The next speaker was Matthias Deeg with a talk called "Of Mice and Keyboards: On the Security of Modern Wireless Desktop Sets".
His talk focussed on remote keyboards and mouses (using RF to communicate with a USB dongle). After a short introduction to the technology used and how it works, Matthias explained the goal of his research. There was already papers and presentation on the same topic but he focused on multiple device vendors (Cherry, Microsoft, Logitech & Fujitsu). The methodology was a classic approach: open the device, identify the chops, read (a lot of) documentation and write some code. The nRF24 looks to be the most popular one. The different exploited scenarios were:
- Physical attack: extract the firmware, change the firmware, extract crypto material and manipulate it
- Use radio signals: exploit open radio comms
- Replay attacks
- Keystroke attacks
- To try to decrypt comms
Matthias made some nice demos: abuse a wireless mouse to launch a virtual keyboard and type some text. The replay attack simply unlocked a Windows desktop by replaying the user's password. The best one was the injection attack: a Powershell payload can be injected and executed! Nice demo but we already know that such devices provide a low level of security.
Paul Rascagnères explained how Microsoft implemented code signing protection but also how to bypass it. A small reminder about Authenticode: it was developed by M$, based on certificates. Kernel drivers must be signed since Win7 64bits. Really?
Paul explained several ways to bypass this protection: low cost and high end. The low cost is based on the bcdedit.exe command that can be used to switch the system to test mode :
bcdedit.exe -set TESTSIGNING ON
If a message is displayed by winlogin.exe, it can be removed by patching the process in memory to NOT display this info on the screen. The other technique (used by Uboruros) is based on CVE-2008-3431. Via VBoxDrv.sys, no message is displayed but it can be detected using bcdedit.exe. Other techniques explained by Paul were exploited by the malware Derusbi, GrayFish and HIDEDRV.
The last talk was presented by Kevin Borgolte about Shellphish and the DARPA Cyber Grand Challenge. The idea behind this project is to organize a CTF game ("Capture the Flag") but instead of human players, computers are playing between and try to exploit each others and to protect themselves in an automated way. Kevin explained in details how it works but it was way too complex to get the point. Just visit the website if you're interested in this topic or watch the video.
That's all for the first day! Note that talks are recorded and videos posted very quickly after the presentation. The Youtube channel is here. Stay tuned for the second wrap-up by tomorrow!
18 Oct 2016 9:43pm GMT
Quick recap; I launched cron.weekly 50 weeks ago, it's a weekly newsletter about Linux and open source and everything that relates to it. And last week, I sent the 50th issue in a row!
It's grown from a handful of subscribers to over 2.500 now -- some are even angry when the mail doesn't arrive on time. I'm extremely happy with the growth and feedback I've received!
If it sounds interesting to you, go to cronweekly.com or scroll to the bottom of this post to sign up.
Convenience is costly
I first signed up for MailChimp to start the cron.weekly newsletter because it was the easiest option. For the newsletter, I wanted an online archive, so the best method to me seemed to be a website to host the archive, and a mailing tool that could read the RSS feed from that and automatically mail the content to everyone who signed up.
I set that up in less than 2 hours, with a couple of clicks;
- WordPress website with a random theme
- The MailChimp plugin for WordPress
- An RSS campaign in MailChimp
Every time I posted something in WordPress, Mailchimp would pick it up over RSS (thanks, Aaron!) and send it to everyone who subscribed, using the simple WordPress plugin.
Convenience for lazy folks hackers.
It's all about the money
Convenience comes at a cost though, and it isn't immediately clear with MailChimp. You can have up to 2.000 subscribers in your mailing list and still send mails for free. Awesome, right?!
Well, things get expensive pretty quickly though.
Signups for cron.weekly are actually going pretty strong, which landed me in the 50$/month plan. That's 600$ per year.
For a hobby that isn't making any money (yet), I'm not going to spend 600$ a year.
In fact, with the current growing rate, I would soon be landing in a 75$/month plan, for almost 900$ per year.
The sysadmin hacker in me disagrees with those pricing. I can do that for less, right?
Enter Sendy and Amazon
There are a couple of alternatives for self-hosting a newsletter, the most popular seem to be Sendy (PHP) and Mailtrain (NodeJS). I chose Sendy as it's the easiest to get running, as all it requires is a webserver and PHP. And I had that already for this blog and the variety of other sites I run.
It isn't free though, but at a one-time fee of 59$ it's good enough.
Once you have the application installed, there are 2 ways to send your newsletter;
- From the local server, using the local MTA (go postfix!)
- Using Amazon's Simple Email Service (SES)
There's this old saying that friends don't let friends run mailservers, so instead of sending mails locally, I decided to use Amazon SES.
After all, I send a little over 12.000 mails per month, nothing a local server can't handle -- but e-mail delivery gets tricky at volume (DKIM, MARC, IP reputation, SPF records, handling bounces, ...). None of those things I wanted to manage.
But the biggest motivation to me was saving money, so here's my Amazon SES bill (forecasted for October):
From 50$ per month to 1.38$ per month. Score!
But, there's a catch.
Responsive mobile e-mail layouts
There's a big plus about MailChimp: for the monthly fee, you get well designed, tested, responsive e-mail layouts.
And as it turns out, creating decent e-mail layouts is like designing for IE6: it's effing frustrating.
I played around with both, but Cerberus seemed to fit my use case best. So I wrote a little e-mail parser to generate the HTML and TXT versions for the newsletter (which is up on GitHub, if you're interested). This probably took most of the time to finetune, test and process reader feedback.
Doing things like this makes you value MailChimps' pricing, but in hindsight I still made a good decision going self-hosted + Amazon SES.
MailChimp is expensive, Sendy is intensive
You spend more time setting up Sendy & Amazon SES (their 'getting started' guide is perfect by the way), but it'll save quite a bit of money in the long run.
My Sendy setup: 59$ one time fee + roughly 2$ a month = around 80$ for the first year and 20$ recurring costs per year.
With MailChimp, I was already at 600$ a year and things were only going to increase.
If you're on MailChimp now: spend some time checking out Sendy (or alternatives) and use SES to send out e-mails. Your wallet will thank me later.
18 Oct 2016 7:21pm GMT
17 Oct 2016
Last week, we launched a new version of Acquia Lift, our personalization tool. Acquia Lift learns about your visitors' interests, preferences and context and uses that information to personalize and contextualize their experience. After more than a year of hard work, Acquia Lift has many new and powerful capabilities. In this post, I want to highlight some of the biggest improvements.
Intuitive user experience
To begin, Acquia Lift's new user interface is based on the outside-in principle. In the case of Acquia Lift, this means that the user interface primarily takes the form of a sidebar that can slide out from the edge of the page when needed. From there, users can drag and drop content into the page and get an instant preview of how the content would look. From the sidebar, you can also switch between different user segments to preview the site for different users. Personalization rules can be configured as A/B tests, and all rules affecting a certain area of a page can easily be visualized and prioritized in context. The new user interface is a lot more intuitive.
Unifying content and customer data
Having a complete view of the customer is one of the core ideas of personalization. This means being able to capture visitor profiles and behavioral data, as well as implicit interests across all channels. Acquia Lift also makes it possible to segment and target audiences in real time based on their behaviors and actions. For example, Acquia Lift can learn that someone is more interested in "tennis" than "soccer" and will use that information to serve more tennis news.
It is equally important to have a complete view of the content and experiences that you can deliver to those customers. The latest version of Acquia Lift can aggregate content from any source. This means that the Acquia Lift tray shows you content from all your sites and not just the site you're on. You can drag content from an ecommerce platform into a Drupal site and vice versa. The rendering of the content can be done inside Drupal or directly from the content's source (in this case the ecommerce platform). A central view of all your organization's content enables marketers to streamline the distribution process and deliver the most relevant content to their customers, regardless of where that content was stored originally.
Content can also be displayed in any number of ways. Just as content in Drupal can have different "display modes" (i.e. short form, long form, hero banner, sidebar image, etc), content in Acquia Lift can also be selected for the right display format in addition to the right audience. In fact, when you connect a Drupal site to Acquia Lift, you can simply configure which "entities" should be indexed inside of Acquia Lift and which "display modes" should be available, allowing you to reuse all of your existing content and configurations. Without this capability, marketers are forced to duplicate the same piece of content in different platforms and in several different formats for each use. Building a consistent experience across all channels in a personalized way then becomes incredibly difficult to manage. The new capabilities of Acquia Lift remedy this pain point.
The best for Drupal, and beyond
In addition, we've also taken an API-first approach. The new version of Acquia Lift comes with an open API, which can be used for tracking events, retrieving user segments in real time, and showing decisions and content inside of any application. Developers can now use this capability to extend beyond the Lift UI and integrate behavioral tracking and personalization with experiences beyond the web, such as mobile applications or email.
I believe personalization and contextualization are becoming critical building blocks in the future of the web. Earlier this year I wrote that personalization is one of the most important trends in how digital experiences are being built today and will be built in the future. Tools like Acquia Lift allow organizations to better understand their customer's context and preferences so they can continue to deliver the best digital experiences. With the latest release of Acquia Lift, we've taken everything we've learned in personalization over the past several years to build a tool that is both flexible and easy to use. I'm excited to see the new Acquia Lift in the hands of our customers and partners.
17 Oct 2016 11:16pm GMT
For the many of you that are following me on Twitter it's probably not news anymore, but I wanted to make sure that all readers of my blog are in the loop (and of course, thanks for visiting my humble spot in the interwebs!). Last Thursday, we announced Flowable, a fork of Activiti. We've written […]
17 Oct 2016 2:34pm GMT
14 Oct 2016
Je viens de terminer un livre qui m'a convaincu qu'Elon Musk, le célèbre CEO de Tesla et SpaceX, est un voyageur du futur perdu dans notre époque. Que ses actions nous révèlent ce qu'il connaît de notre avenir immédiat.
Le résumé du livre
Alors qu'il se ballade tranquillement dans la rue, un homme sans histoire se retrouve brutalement ramené 1400 ans dans le passé.
Grâce à sa culture générale, il arrive à comprendre ce qui lui est arrivé et à se débrouiller dans la langue de l'époque. Préoccupé par sa survie immédiate, il va tout d'abord monter une petite affaire en utilisant ses connaissances du futur. Un produit basique, facile à réaliser qui rencontre immédiatement le succès.
Cette petite réussite permet à notre voyageur temporel de lancer d'autres entreprises. Il sait que, d'ici quelques années, le monde va connaître de grands bouleversements et tomber pour plus d'un millénaire dans un âge sombre de ténèbres, de misère, de famine et de maladie. Un millénaire dont il faudra plusieurs siècles pour s'extraire.
En tentant d'apporter le plus d'innovations possibles, notre héros va chercher à préserver l'humanité de ces ténèbres. Sans répit, sans relâche, il crée des entreprises qu'il doit toutes diriger lui-même, les artisans et les travailleurs de l'époque n'étant souvent tout simplement pas capable de suivre ses instructions à la lettre ou s'évertuant à en tirer un petit profit personnel.
Il tente également d'alerter les intellectuels sur l'imminence de la catastrophe mais elle semble bien lointaine ou irréaliste à toute une population qui trouve bien plus important de se perdre en arguties théologiques ou à comploter pour obtenir un ersatz de pouvoir déliquescent.
Souvent incompris, traité comme un fou mais respecté pour ses succès économiques, notre infatigable voyageur temporel ne prendra même plus le temps de dormir, tentera vainement d'avoir une vie amoureuse mais sera à chaque fois rattrapé par l'urgence absolue de tout tenter pour protéger l'humanité. Il aura à lutter contre l'incrédulité et les croyances absurdes d'un peuple confondant modernité et décadence.
Une coïncidence troublante
Écrit juste avant la seconde guerre mondiale par l'écrivain américain Lyon Sprague de Camp, « De peur que les ténèbres » nous fait suivre Martin Padway, jeune archéologue qui est, dès la seconde page, transporté au VIème siècle dans les derniers jours de l'empire romain.
Fin connaisseur de l'histoire, il va tenter de conjurer l'inéluctable chute de l'empire en tentant des actions à court terme, afin d'éviter les erreurs les plus flagrantes, et des actions à long terme, par l'introduction de technologies comme les chiffres arabes, le sémaphore, l'imprimerie et la presse écrite. À court terme, il cherche à éviter que l'Italie tombe dans le chaos. À long terme, il souhaite éviter l'obscurantisme religieux et l'ignorance qui mènera à un millénaire de misère.
Hormis son caractère de classique de la SF des années 30 et de précurseur de l'uchronie, « De peur que les ténèbres » n'a rien de transcendant. Si ce n'est le particulièrement troublant parallèle que je n'ai pu me retenir de faire avec la vie d'Elon Musk, fondateur de Tesla et de SpaceX.
Tout comme Martin Padway, Elon Musk semble pris par une frénésie d'entreprises, d'innovations. Rien n'est jamais assez bien à son goût et il supervise la plupart des développements importants. Tout comme Martin Padway, Elon Musk ne semble pas attiré par l'argent ou la réussite. À chaque succès, il réinvestit absolument tout dans une nouvelle aventure. Il y'a chez Elon Musk, comme chez Martin Padway, une urgence vitale, obsessionnelle.
Chaque entreprise est considérée comme folle, vouée à l'échec. Pourtant, elles finissent souvent, mais pas toujours, par se révéler des succès même si ce n'est pas immédiat.
Elon Musk serait-il, comme Martin Padway, un voyageur temporel ? Est-il un homme parfaitement banal né en 3400 ? Par sa culture générale, il sait alors que l'humanité s'apprête à vivre une catastrophe.
Cette catastrophe initiale est l'invasion de l'Italie par le général romain Bélissaire pour Martin Padway et la crise du réchauffement climatique pour Elon Musk. Cela expliquerait cette urgence de développer Tesla et Solar City. De faire sortir l'humanité du pétrole en quelques années et non en quelques générations. Car chaque mois compte dans cette course contre la montre afin de prévenir la destruction inéluctable de la planète.
Un période sombre pour l'humanité
Nos voyageurs temporels sont également convaincus que l'humanité va connaître un millénaire de disette et de misère. Pour Martin Padway, la religion et l'ignorance seront les principaux responsables de ce moyen-âge. Il introduit donc l'imprimerie, le télégraphe, les chiffres arabes, la gravitation, l'héliocentrisme. Des innovations qui devraient permettre de conjurer les ténèbres sur le long terme.
Elon Musk, lui, introduit OpenAI dont le but est d'encadrer la recherche sur l'intelligence artificielle afin de s'assurer que celle-ci ne soit pas néfaste pour l'humanité. La crainte d'une intelligence artificielle partagée par de nombreux intellectuels dont le physicien Stephen Hawking.
Si les deux hommes semblent avoir sacrifié leur sommeil et une vie amoureuse normale, ils ont néanmoins leur point faible, leur petit plaisir qui sera banal à leur époque mais est strictement impossible dans le passé où ils ont été projetés. Martin Padway veut lancer la construction de bateaux capables de rejoindre les Amériques afin de ramener du tabac. Elon Musk a lancé SpaceX afin d'aller sur Mars. Mais à quelle fin ?
Les enseignements du futur
Si l'on accepte l'idée qu'Elon Musk soit bel et bien un voyageur du futur égaré dans notre époque, ses actions nous apprennent beaucoup sur notre avenir. Et ce n'est pas particulièrement réjouissant.
Tout d'abord, le réchauffement climatique et la consommation de pétrole vont créer très rapidement une catastrophe importante. La bonne nouvelle c'est qu'il est sans doute encore possible de l'éviter ou de la limiter mais il faut agir tout de suite. Tout comme Martin Padway, Elon Musk fait face à un déni bâti sur l'immobilisme, l'idiotie et la superstition.
En deuxième lieu, les intelligences artificielles vont asservir les humains et leur faire connaître une période terriblement difficile. La bonne nouvelle c'est que les humains vont survivre, au moins assez longtemps pour nous envoyer Elon Musk. Mais il serait sage de prendre au sérieux les avertissements sur le sujet.
Le troisième enseignement c'est qu'en 3400, il est relativement facile d'aller sur Mars et, visiblement, cela en vaut la peine. Enfin une bonne nouvelle !
Finalement, il paraît évident que pour un individu du 35ème siècle nous sommes tous des arriérés superstitieux et obtus, incapables d'avoir une compréhension globale du monde. Malgré tous les efforts que le voyageur temporel fait pour nous sauver de nous mêmes et de nos ridicules guerres religieuses ou nationalistes, nous nous évertuons à nous croire invincibles et à ne pas voir plus loin que le bout de notre nez.
Au fond, l'enseignement immédiat que nous pouvons tirer de la connaissance du futur d'Elon Musk c'est que le réchauffement climatique et les intelligences artificielles non-contrôlées sont des problèmes graves à régler bien plus rapidement que de savoir qui fait partie de quel pays et quel livre sacré est le meilleur.
Peut-être que tout ceci n'est qu'une coïncidence et qu'Elon Musk ne vient pas du futur.
Mais voulons-nous vraiment prendre ce risque ? Que pensez-vous que penseront les humains de l'an 3400 de nos actions, de nos comportements quotidiens et de notre immobilisme ?
Photo par Thierry Ehrmann.
Ce texte est publié sous la licence CC-By BE.
14 Oct 2016 12:56pm GMT