fidzu : Apache

This bit-o-news is wandering around the inter-tubes…

The fire- and natural-gas alarm system aboard the Deepwater Horizon was partly disabled on the night the drilling rig caught fire, the chief electrician aboard testified Friday at a hearing outside New Orleans. "The general alarm was inhibited," said Michael Williams, an employee of Transocean Ltd., which owned the rig. He explained that the system that automatically sounded a general alarm had been disabled because rig managers "did not want people woken up at 3 a.m. with false alarms."

My reaction initial reaction was; "But of course." If you read the books about a catastrophes this is a standard plot element, happens every time.

But there is more to this. We all make conscious choices about which alarms to turn on and off. The term alarm isn't quite right though, again this is the question of how we manage our attention. Attention is scarce, you have to manage it. The universe of things we ought to pay attention to is very large. It has a long tail, and the black swans live out on that tail.

It's one thing when the daemons raising alarms are all inside your head. But in larger organisms the alarms are raised by real people, or the systems they built. Managers have to make choices about, and most managers assume those around them will advocate for various classes of alarms as necessary. Those choices are fraught with risk, but that insight isn't much help; life is full risk.

There is some mileage to be gotten framing management as alarm tuning. That's probably a variant of metrics management thinking. Trading off the risk of an entire crew who haven't gotten enough sleep against the risks obscured by disabled alarms. Well, that's what managers do. When it calls to hell in a hand basket, as it so often does, they get the blame. They deserve it.

31 Jul 2010 3:44pm GMT

Es ist ja doch interessant, was Gerüstbauer sich ausdenken, um die Kunden zu verar***

Wir haben unsere strassenseitige Hausfront gedämmt und uns vom Gerüstbauer Schürzeberg aus Viersen ein Gerüst aufstellen lassen. Es fehlten:

• die abgemachte Aussockelung, damit die Verlängerung des Dachüberstands gemacht werden konnte
• eine Leiter für den Einstieg
• die Beleuchtung des Gerüsts
• laut Putzer eigentlich sogar die "Boards" - eine Absicherung gegen herunterfallene Kleinteile

Das feinmaschige Netz, was wegen der Putzarbeiten benötigt wird, kam auch erst nach telefonischer Nachfrage zwei Tage später - nachdem der Putzer zum ersten mal da war …

Aber berechnet wurde natürlich alles - inklusive einer Genehmigung der Stadt.

Mit dieser Genehmigung, die im Amtsdeutsch "Erlaubnis zur Sondernutzung des öffentlichen Strassenraumes" heißt, hat es folgende Bewandnis: der Bauherr - also ich - muss diese bei einer Kontrolle des Ordnungsamtes vorweisen können. Hätte der Gerüstbauer diese also besorgt, hätte er sie mir aushändigen müssen. Ob er sie überhaupt eingeholt hatte …. wurde mir zumindest nicht nachgewiesen.

Glücklicherweise verfügte ich über ein zweites Angebot von Schürzeberg und konnte eine Differenz ermitteln, die ich dann zurückgehalten hatte. Dafür revangierte dieser sich, indem er ohne Rücksprache das Gerüst früher abbaute.

Aber da unser Putzer früh genug fertig war, kam uns das schon fast gelegen - hatten wir doch so früher das Teil vom Hals

31 Jul 2010 8:35am GMT

Just in case you hit a problem if, you are usin 10.03, the router shutdown by an upgrade, power off, whatever… your partition is read only, and you are not able to modify anything on the router. Login on the console and type dmesg. If complains about

Write of 186 bytes at 0x0015a680 failed. returned -30, retlen 0

Just type:
mtd unlock rootfs_data

#7298 (Cannot edit files in /etc/config due to flash write error) - OpenWrt.

31 Jul 2010 7:17am GMT

Today I patched www.reppep.com, and it broke email once again. As on several previous occasions, perl modules were broken, amavisd-new was throwing misleading errors on startup, and I had to reinstall Scalar-List-Utils to get rid of complaints about Compress::Zlib.

This time, however, I decided to upgrade amavisd-new in hopes the new version would be smarter about the (bogus) perl module complaints at startup. I also tried using yum to install some of the perl module dependencies, which entailed reinstalling spamassassin. Alas, amavisd-new-2.6.4 is no smarter, but either amavisd-new or spamassassin introduced a new dependency on Mail::DKIM, which requires the Crypt-OpenSSL-Random perl module. I tried getting them through cpan, but it kept choking -- apparently Crypt-OpenSSL-Random requires the openssl-devel RPM on CentOS, but isn't smart enough to throw a clear error demanding it.

I never did figure out where Mail::DKIM was enabled, or how to disable it, but I seem to have found a much better solution.

amavisd-new is not in the base RHEL (or CentOS) repositories, so the CentOS wiki recommends getting it from RPMForge. This turned out to be pleasantly simple, and should prevent yum from breaking it in the future. Here's hoping, anyway!

31 Jul 2010 1:09am GMT

• Keyboard shortcuts for positioning windows in Mac OS X : from Tony Finch. great stuff, I used to use shortcuts like this all the time on my Linux desktops to avoid rodentage
  (tags: mouse keyboard navigation windows shortcuts scripting ui automator)

• Draft Functional Spec of Hadopi "securisation" software : Crazy suggestions leaked from the French anti-piracy authority. Mandatory host-based and router-based anti-piracy software and firmware with blocklists of suspect keywords, suspicious applications, TCP ports, protocols; detect suspicious apps installed; detect use of open wifi; detect use of anti-filtering/anti-blocking "workarounds" (ie. VPNs and Tor). Log all this to a dual journal, one of which will be encrypted using key escrow (presumably for use in prosecutions), retaining data for a year. Basically, a mandatory snooping infrastructure. Where would this leave Macs and Linux for French users?
  (tags: hadopi piracy filtering snooping big-brother 1984 via:adulau vpn tor blocklists)

30 Jul 2010 10:05pm GMT

A brief listing of some of the news around the ASF this past month.

• Adobe is in talks to purchase Day Software. You can read the Redmonk'er Coté's analyst view of how the products compliment each other, a good ComputerWorld overview article, or can read Bertrand's or Jukka's great community analysis, or just see Bertrand's delicious.
• Greg Stein was awarded an O'Reilly Open Source Award. While Greg has his own large pitcher of accomplishments elsewhere, he's also a significant contributor at the ASF, currently serving as a Director and VP, and also partly to credit for Subversion becoming Apache Subversion.
• Apache FOP has released 1.0! FOP has been around a long time, one of the major XSL Formatting Objects processors, and deserve their moment in the sun for organizing a complete 1.0 release.
• Apache Cayenne released 3.0, and include a fact sheet of the newest features in their ORM product.
• Apache Tomcat released 7.0, implementing the latest Servlet 3.0, JSP 2.2, and Expression Language (EL) 2.2 specs, and getting lots of coverage.

Oh, and the ASF elected a new board of directors as well - there are some different (and one new) faces, but overall, we expect steady sailing into better waters.

Want to get your own news about Apache projects? Read or feed from the announce list, official Foundation and project blogs, or get the Planet Apache community perspective.

30 Jul 2010 6:10pm GMT

Thank you Google Alerts, for pointing out this article on choosing a Java web framework. It's over a year old, but I think the things that make Tapestry special have only gotten stronger in the intervening time.

30 Jul 2010 5:24pm GMT

Several years ago (it feels like very many years ago) I built Simpy, a social bookmarking service. This was back in the Web 2.0 when tags were quasi-new, when the term folksonomy was invented, and when anything social was cool (this may still hold true now in 2010?). A few years ago, Simpy was sold to a large, international, household-name news agency. Without getting into any details, a few months ago I decided to move my bookmarks from Simpy to Delicious, of all places. Here is the Java code I used for that:

import com.simpy.api.rest.client.Simpy;
import com.simpy.api.rest.client.beans.Link;

import del.icio.us.Delicious;

import java.lang.StringBuilder;
import java.util.Date;
import java.util.List;
import java.util.Iterator;
import java.text.SimpleDateFormat;

/**
 * API Key (consumer key):
 * XXXXXXXXXXXXXXXXX
 *
 * Shared secret
 * YYYYYYYYYYYYYYYYY
 *
 * Application id
 * ZZZZZZZZZZZZZZZZZ
 */
class sy2x {

  sy2x() {}

  public static final void main(String[] args) throws Exception {

    Simpy sy = new Simpy("username", "password");
    Delicious del = new Delicious("username", "password");
    List links = sy.getAllLinks(null, null, null, null);
    //System.out.println("Count: " + links.size());
    int counter = 0;
    int failCounter = 0;
    Iterator it = links.iterator();

    while (it.hasNext()) {
      //System.out.println(it.next());
      Link link = (Link) it.next();
      List tags = link.getTags();
      Iterator tagIt = tags.iterator();
      StringBuilder delTags = new StringBuilder();
      while (tagIt.hasNext()) {
        String tagVal = (String) tagIt.next();
        tagVal = tagVal.replace(" ", "_");
        delTags.append(tagVal + " ");
      }

      boolean shared = link.getAccessType().equals("private") ? false : true;
      SimpleDateFormat sdf = new SimpleDateFormat("y-M-d H:m");
      Date addDate = sdf.parse(link.getAddDate());

      boolean ret = del.addPost(link.getUrl(),link.getTitle(),link.getNote(),delTags.toString(),addDate,true,shared);

      if (ret == false) {
        failCounter++;
        System.err.println("FAILED:");
        System.err.println("URL\t" + link.getUrl());
        System.err.println("Note\t" + link.getNote());
        System.err.println("\t\textended");
        System.err.println("\t\thash");
        System.err.println("Tags\t" + delTags);
        System.err.println("Date\t" + link.getAddDate() + " (" + addDate + ")");
        System.err.println("Access\t" + link.getAccessType());
        System.err.println("----------------------------------------");
      }
      counter++;
      System.out.println(counter + " (failures: " + failCounter + ")");
      Thread.sleep(5000);
    }
  }
} 

You'll need some jars (which you can find online):

$ export CLASSPATH=$CLASSPATH:commons-codec-1.3.jar:commons-httpclient-3.0.jar:commons-logging-1.0.4.jar:delicious-1.14.jar:simpy-java-1.2.jar:simpy-java-1.2-with-dependencies.zip

Compile the above class:
 

$ javac sy2x.java

Run it:

 

$ java sy2x &> out.log

30 Jul 2010 4:12pm GMT

I don't spend a lot of time on The Apache HTTP Server Users mailing list, but a discussion sprang up there this week on which I think I should share my response. The issue was why the server in question did not have permission to show a particular file. The initial response was "just chown your document root to the Apache user" and, when pointed out that this introduced security issues,

Oh man an experienced sys admin told me to do it that way.
Please tell me what is wrong in this and where is this documented on Apache
docs.
I want to read.

Here is my response reproduced: read on.

The Apache HTTP Server needs read access to its configuration files and the files it serves. In and of itself, the server does not need write access anywhere on the system: even its log files are opened for write when the server is still root, and the open file descriptors passed to the child processes which change their user id to the lesser privileged user.

Read access only. The web server user should not own, or be able to write to, its configuration files or content.

Content, other than CGI scripts, generally does not need Execute permissions. Even PHP files that are interpreted by the server do not need to be Executable.

Certain applications, especially publishing platforms and Content Management Systems that you manage and populate through the web server itself using a browser, require that certain directories on the system be made writable by the web server user. You can do this by changing the owner of the directory to that user (usually www but ymmv), or by making the directory group-writable and changing the group to the group as which Apache runs.

Making directories writable by the web server should be done only with care and consideration. The usual threat model is that someone manages to upload (for instance) a PHP script of their own making into the document root, and simply executes that by accessing it through a browser. Now someone is executing code on your machine. Google for 'r57′ for an example of what such code can do.

If a web app needs writable directories, it's often better to have those outside the DocumentRoot: that way the uploads can't be accessed from the outside through a direct URL. Some applications (Wordpress for instance) support this, others do not.

In many cases, writable directories are not strictly necessary even though the web app might like them: rather than upload plugins (which contain code that gets executed or interpreted, yech!) through the web browser, upload them through ssh and manually unpack them on the server. The CMS Joomla! likes to write its configuration file to the Document Root on initial install (which promptly becomes a popular attack target) but if it can't write to the Document Root, it will output the config to the browser to the user can manually upload it.

The Apache Documentation will merely tell you to make the server installation root-owned. The HTTP Server Documentation does not cover third party applications like Wordpress or Joomla!, so it will not discuss their need to have some directories writable. I hope the above makes the picture a little more complete.

Share this post:

30 Jul 2010 4:00pm GMT

A new bugfix release of ActiveMQ-CPP is out v3.2.2. For this who have experienced segfaults from APR Atomics on startup this release should address all those issues. There is also a fix for some bugs related to sending and receiving empty MapMessage objects.

You can download the source bundle from the ActiveMQ-CPP release page:

30 Jul 2010 2:01pm GMT

30 Jul 2010 1:52pm GMT

What does the acquisition of Day by Adobe mean for Day's open source activities? Some people are disappointed by the lack of comments about this in the official announcements to date.

Thankfully, Erik Larson, senior director of product management and strategy at Adobe, commented on Glyn Moody's blog post quite early in the frenzy of tweets and blog posts that followed yesterday's announcement.

Quoting him:

…we are very excited for Day's considerable "open source savvy" to expand Adobe's already significant open source efforts and expertise. That is a strategic benefit of the combination of the two companies. I have personally learned a lot from David Nuscheler and his team in the past few months as we put the deal together.

Not bad for a start, but we're engineers right? Used to consider the worst case, to make sure we're prepared for it.

Me, I'm an engineer but also an optimistic, and I'm used to start with the ideal, happy case when analyzing situations. It helps focus my efforts on a worthy goal.

So let's do this and dream about the best and worst cases. This is absolutely 100% totally my own dreams, I'm not speaking for anyone here, not wearing any hat. Just dreamin', y'know?

The Dream

This is late 2011.

The last few months have more than confirmed that Day's acquisition by Adobe, one year ago, happened for strategic reasons: a big part of the deal was filling up gaps in Adobe's enterprise offering, but Day's open source know-how and network have brought a lot of value as well.

Day folks have played an important role in expanding the open development culture inside Adobe; Photoshop will probably never be fully open source, but moving more key components of the Adobe technology stack to open source, and most importantly open development, has paid off nicely. In terms of reaching out to developers and customers, in getting much better feedback at all levels, and in terms of software quality of course. It's those eyeballs.

The Apache Software Foundation's Incubator has been quite busy in the last few months. The new platinum sponsor enjoys a fruitful relationship with the foundation.

With JCR moving to their core, Adobe's enterprise applications are starting to reach a new level of flexibility. Customers are enthusiastic about being able to access their data via simple and standards-based interfaces. Enterprise-level mashups, anyone?

JCR is not just that minor content repository API pushed by that small swiss software vendor anymore: being adopted by a major player has made a huge difference in terms of market recognition (I'm sure my friends at Hippo, Jahia and Sakai, among others, will love that one). The added resources have also helped improve the implementations, and people love the book!

With this, Apache Jackrabbit and Apache Sling have reached new levels of community participation and quality. Although quite a few new committers are from Adobe, a number of other companies have also pushed their developers to participate more, due to the increased market visibility of JCR.

Adobe's additional resources, used wisely to take advantage of the Day team's strengths, have enabled them to fully realize the CQ5 vision. Everything is content, really.

As in all fairy tales, the former Day team and Adobe live happily ever after. (Editor's note: this is not Disney, can we strike that one please?)

The Nightmare

This is late 2011, and I can hear the programmers complaining in their bland cubicles.

Aaarrggghhhhh.

The few Day folks who still work at Adobe did try to convince their management to continue on the open source and open development track. No luck - you can't argue with an US company making 4 billion a year, can you?

CQ5 customers are too busy converting their websites to native PDF (this is about documents, right?) to realize what's going on. The most desperate just switched to DrooplaPress, the newest kid on the LISP-based CMSes block. That won't help business much but at least it's fun to work with. If you love parentheses, that is.

Adobe's competitors who really jumped on the open source and open development train are gone for good, it is too late to catch up. You should have sold you shares a year ago.

Luckily, Apache Jackrabbit and Apache Sling are still alive, and increased involvement of the "Benelux Gang" (ex-Day folks spread over a few Benelux content management companies) in those projects means there's still hope.

You wake up wondering why you didn't accept that job at the local fast food. Computers are so boring.

Coda

I know life is more complicated than dreams sometimes, but I like dreams much better than nightmares, and I'm a chronic optimistic. So you can easily guess which scenario I'm going to work towards!

I'll keep you posted about what really happens next. Once I wake up, that is.

Just dreamin', y'know?

Related reading

Open Source at Adobe by my colleague and fellow Apache Member Jukka Zitting.

Open innovation in software means Open Source, a recent post of mine.

See also my collected links related to the announcement at http://delicious.com/bdelacretaz/adobeday.

30 Jul 2010 1:49pm GMT

Joe, you are an unashamed Jesus phone user
You dont take any grief from any luser
Your perl fu stands head and shoulders above,
But somedays that is just not enough. 

You were briefly courted by Ning,
Just so you know we wouldn't trade you for anything
You are well known for speaking your mind
But sometimes it feels so unkind.
Do you think you could be nice, if even for a day
But be sure that we love you no matter what you do

We are on the hunt for a new president and you know who would be best,
For once you and I agree, and in this I do not jest. 

So, what are we going to do now.  You have started working for slayer
of hopes and dreams and the betrayer of sysadmins.
Next you'll be coming back and telling us that Java is not all that bad,
and we should all extoll the virtues of git.

Oh Joe, oh Joe, oh Joe
Whatever would we do without you?
Some days you get out of bed on the wrong side,
Others you're just like a grizzly bear with the flu.

You like to tease, poke and prod,
You set your standards really high,
But whatever happens to those who get it wrong?
You're always there to say I told you so,
Or just call someone a twit.
Now you have Gavin to teach and guide
I'm sure he'll turn out just fine

Your choice of shell is ZSH,
That speaks volumes about your age.
Don't forget to remind us that git is the VCS that Jesus would use.

This past fall we finally got to meet,
Oh boy it was such a treat.
Today you're off too Disneyland,
Even at your age I'm sure it'll be grand.

30 Jul 2010 1:29pm GMT

Happy Sysadmin Appreciation Day to the Apache Infra team! Thanks so much for all you do to keep us safely and securely computing. We enjoy the fruits of your ridiculous talent and tireless labor, you guys rock!

30 Jul 2010 11:29am GMT

Probably not too unusual for people working on free software mostly (though no longer exclusively) in their spare time, the number of items that appear in my private calendar have increased steadily in the past months and years:

• Every three months I am organising the Apache Hadoop Get Together in Berlin.
• I have been asked (and accepted the offer) to publish articles on Hadoop and Lucene in magazines.
• There are various conferences I attend - either as speaker or simply as participant: FOSDEM, Froscon, Apache Con NA, Devoxx, Chemnitzer Linuxtag - to name just a few.
• For Berlin Buzzwords I did get quite a bit of time for organisation, still some issues leaked over to what others would call free time.
• I am mentoring one of Mahout's GSoC students which is a lot of fun.
• At least I try to spend as much time as possible on the Mahout mailing lists keeping up with what is developed and discussed there.

There are various techniques to cope with increased work load and still find enough time to relax. Some of them involve simply remembering what to do at the right time, some involve prioritization, others deal with measuring and planning what to do. In this tiny series I'll explain the techniques I employ - or at least try to - in the hope of getting your feedback, and comments on how to improve the system. After all, the most important task is to constantly improve ones own processes.

30 Jul 2010 7:07am GMT

If you're a long-time reader of this blog, you'll know I've been to some great weddings in the last couple years. This past weekend, I had the pleasure of experiencing yet another fantastic celebration with two old and close friends, Clint and Jess. You might remember Clint from his wedding in Costa Rica or when we almost slept in a snow cave. I'm happy to report we didn't get in any trouble and everyone survived the weekend without a scratch.

My trip to Jess's wedding (on the Lost Coast of Northern California) started with a flight to Portland, Oregon. After arriving, I drove to Clint and Autumn's house in Eugene where we enjoyed some sweet Oregon micros and reminisced about Costa Rica. The next morning, we headed for the wedding; an 8-hour drive. Our road trip was awesome, especially when we started driving through the Redwood Groves on 101.

We stayed in a sweet beach house for the weekend. While it was foggy most of the time, the sun did come out on Saturday. We quickly became surrounded by beautiful views and headed to the beach to relax with Jess.

The wedding was on Sunday, a mere block from where we were staying. The ceremony was one of the most heartfelt I've ever heard, especially since the Wedding Official was a friend of the bride's since she was born.

The reception afterwards was a truly spectacular party that lasted well into the evening. Clint and I vowed to go to bed early, but we ended up having so much fun we closed the place down. Jess and Lili were an instrumental part in creating a spectacular night, especially with their wedding dance and infectious happiness.

The next day, we woke up on time, embarked on the 10-hour road trip back to Oregon and enjoyed a quick detour through the Avenue of the Giants. I did end up missing my flight home, but it was worth it. Thanks to Lili and Jess (and their families) for showing us such a great time. It was truly spectacular.

For more pictures, see albums on Flickr, Facebook or the slideshow below.

30 Jul 2010 5:54am GMT