fidzu : Apache

A common question I get during Tapestry training sessions is: Why can't Tapestry reload my services as well as my pages and components?. It does seem odd that I talk about how agile Tapestry is, with the live class reloading, and how nicely OO it is, what with services ... but when you move common logic to a service, you lose the agility because services do not live reload.

This came up yet again, during my latest training session, in London.

I've considered this before, and I've been opposed to live service reloading for a couple of reasons. First, live reloading requires creating new class loaders, and that causes conflicts with other frameworks and libraries. You get those crazy ClassCastExceptions even though the class name is correct (same name, different class loader, different class). Further, in Tapestry IoC, services can be utilized to make contributions to other services ... changing one service implementation, or one module, can cause a ripple effect across an untraceable number of other services. How do you know what needs to be reloaded or re-initialized?

When I last really considered this, back in the HiveMind days, my conclusion was that it was not possible to create a perfect reloading process: one that would ensure that the live-reloaded Registry (and all of its services with all their internal state) would be an exact match for what configuration you'd get by doing a cold restart.

So I shelved the idea, thinking that simply redeploying the WAR containing the application (and the services and modules) would accomplish the desired effect.

But as they say, The Perfect Is The Enemy Of The Good. One very sharp student, Andreas Pardeike, asked: Why not just reload the service implementations?.

Why not indeed? Why not limit the behavior to something understandable, trackable, and not very expensive. Most of the infrastructure was already present, used for reloading of component classes. What about ClassCastExceptions? In Tapestry, service implementations are already buried under multiple layers of dynamically generated proxies that implement the service interface. The underlying service implementation is never automatically exposed.

A few hours of work later ... and we have live service reloading. Each reloadable service gets its own class loader, just to load the service interface class. When Tapestry is periodically checking for updated files, it checks each reloadable service. If necessary, the existing instance, class and class loader is discarded and a new class loader created for the updated .class file.

This is going to make a big difference for me, and for most Tapestry developers. Both applications I'm working on have enough Hibernate entities and other clutter to take some time (20 - 30 seconds) to restart, and most functionality is hidden past a login page. Being able to change a service, for example to tweak a Hibernate query, with the same speed with which I can tweak a template or component class, is just one more thing to keep me in the flow and super productive.

Give it a try ... it's one more step towards making Tapestry so compelling, you wouldn't think of using anything else!

12 Mar 2010 6:00pm GMT

In cloud computing, one of the big issues is how to transfer your data to the cloud. Various techniques are used, but here's, apparently, the current state of the art:

• 
  
• Get a SATA drive
• 
  
• Fill it with your fine data
• 
  
• Put it in a mailing envelope and mail it to Amazon
• 
  
• They'll load the data into your cloud for you!
  

The more things change, the more they stay the same.
As Andrew Tanenbaum wrote in his textbook on networking:

Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

(Though I remember the quote more vividly from Jim Gray's work on data transfer for astronomers):

I've been working with a bunch of astronomers lately and we need to send around huge databases. I started writing my databases to disk and mailing the disks. At first, I was extremely cautious because everybody said I couldn't do that-that the disks are too fragile. I started out by putting the disks in foam. After mailing about 20 of them, I tried just putting them in bubble wrap in a FedEx envelope. Well, so far so good.

12 Mar 2010 5:05pm GMT

The last time I looked into web server architecture was six or seven years ago, when someone pointed me at Dan Kegel's C10K paper. I'm pleased to see that he tried for a while to keep that paper up to date, although it looks like it's been a few years since it was updated.

Recently, I followed some links and ended up at Matt Welsh's SEDA work. Welsh appears to have moved on to other projects, but the SEDA work remains live at SourceForge.

I read through the SOSP 2001 paper and enjoyed it; it's quite readable and well-written, and explains the basic principles clearly.

The primary focus of SEDA is on helping software developers build what Welsh calls "well-conditioned services":

The key property of a well-conditioned service is graceful degradation: as offered load exceeds capacity, the service maintains high throughput with a linear response-time penaly that impacts all clients equally, or at least predictably according to some service-specific policy. Note that this is not the typical Web experience; rather, as load increases, throughput decreases and response time increases dramatically, creating the impression that the service has crashed.

Modern network servers tend to use one of two basic programming techniques: thread-based concurrency, or event-driven programming. Welsh compares and contrasts the two models:

The most commonly used design for server applications is the thread-per-request model. ... In this model, each accepted request consumes a thread to process it, with synchronization operations protecting shared resources. The operaitng system overlaps computation and I/O by transparently switching among threads. ...
When each request is handled by a single thread, it is difficult to identify internal performance bottlenecks in order to perform tuning and load conditioning.

...

The scalability limits of threads have led many developers to eschew them almost entirely and employ an event-driven approach to managing concurrency. In this approach, a server consists of a small number of threads (typically one per CPU) that loop continuously, processing events of different types from a queue. ... Event-driven systems tend to be robust to load, with little degradation in throughput as offered load increases beyond saturation. ... The choice of an event scheduling algorithm is often tailored to the specific application, and introduction of new functionality may require the algorithm to be redesigned. Also, modularity is difficult to achieve, as the code implementing each state must be trusted not to block or consume a large number of resources that can stall the event-handling thread.

To overcome the problems of the two basic approaches, the SEDA work proposes a synthesis and unification:

The staged event-driven architecture (SEDA) ... decomposes an application into a network of stages separated by event queues and introduces the notion of dynamic resource controllers to allow applications to adjust dynamically to changing load.

...

A stage is a self-contained application component consisting of an event handler, an incoming event queue, and a thread pool. ... Each stage is managed by a controller that affects scheduling and thread allocation. ... Stage threads operate by pulling a batch of events off of the incoming event queue and invoking the application-supplied event handler. The event handler processes each batch of events, and dispatches zero or more events by enqueuing them on the event queues of other stages.

The paper goes on to work through many of the details, and to demonstrate a variety of different resource controller implementations, showing how the alternate controllers can provide elegant implementations of adaptive load shedding without forcing complexity into the system developer's core logic.

It's quite a nice presentation, and I liked the way it was all pulled together. I was intimately familiar with the basic techniques and approaches, but it was very helpful to see them pulled together and structured in this fashion.

If you're at all interested in how to build scalable and robust network services that perform reliably under extreme conditions, it's worth spending some time with this work.

12 Mar 2010 3:24pm GMT

From the job description:

We are seeking a Software Developer who is experienced in creating sophisticated, highly interactive, JavaScript applications. Ideally we desire someone that has experience in TinyMCE or has experience working as part of an open source project. The right person will have the ability to work remotely in a highly collaborative manner with virtual teams.

I'm pretty excited about this new opening within Ephox. Lots of great stuff to come out of it hopefully, but in particular helping Ephox to start working better with Open Source communities and developing some awesome stuff with JavaScript. While TinyMCE experience is something we're particularly keen to have "ready to go" if possible, whoever fills this role is going to become a web content editor expert in general from Tiny to CK, Dojo and of course our personal favourite EditLive!

The position is open regardless of your location in the world, though if you happen to be near Brisbane, Maidenhead or Palo Alto we have nice offices you can come in and work from if you like.

12 Mar 2010 2:34pm GMT

For the last couple of years, I've done a ski trip with college buddies to an out-of-state destination. Two years ago, we went to Tahoe and had a great time. Last year, we did it again and I (amazingly) flew and rented a car without a driver's license. This year, we decided to switch things up a bit and head for Jackson Hole.

In previous years, only a couple of us went, but this year was organized by my good friend from Boston, Chris Morse. He managed to take it up a notch and invited a great group of guys, 9 of us in all. I knew about half the group, and met everyone else upon arrival.

The only unfortunate part about the trip was that no new snow fell. However, the Spring Skiing was warm and beautiful, somewhat making up for the lack of snow. The thing I enjoyed the most about this trip was how well the group jelled. Kudos to Chris for assembling such an awesome group and putting such a spectacular trip together. Can't wait for next year.

For further action of what the conditions where like, checkout the YouTube video I made. If only I'd recorded it for another 5 seconds to catch the digger that Corey takes at the end.

To see all the pictures we took on this adventure, checkout my Jackson Hole Set on Flickr.

12 Mar 2010 2:09pm GMT

One of the most frustrating things about trying to find out what is broken in a running application is that the information in the logs is not informative enough, and if it is, there is too much of it to be useful. This is often because the logging configuration has to be set prior to the server starting. So if you have 2 options. Configure the logging at debug level just in case there is a problem, or restart the server when there is a problem. Nether are really practical in production. Debug an everything kills the server, and your definitely in the wrong profession if you can predict where to put debug on (ie the future), you should be a banker.

Fortunately Sling/Felix have a nifty feature, exposed in Sakai Nakamura that allows a sysadmin to reconfigure logging at runtime. If you go to the Configuration console plugin http://localhost:8080/system/console/configMgr and find the Configuration factory called "org.apache.sling.commons.log.LogManager.factory.config" you can create a new configuation. In this configuration you can specify a level, target file, and best of all, a package sub path. For instance I just needed to look at the way ACLs were being resolved in Nakamura so I created a configuration logging to logs/acl.log at log level Debug and used a package name org.apache.jackrabbit.core.security.authorization.acl. Now all the ACL activity happening in a class DynamicACLProvider appears in the log file logs/acl.log. When I am done, I can delete the configuration. Neat. No server restart, and highly targeted logging with no need to go round to each node in a cluster and configure each one differently. I could probably make it all stream out to syslogd on demand and collect the results from a cluster centrally. So if you are sinking under mountains of log files with useless debug statements that you cant get rid of till next time downtime is scheduled, there is another way.

12 Mar 2010 1:30pm GMT

Sometimes I am surprised by the power of Wordpress. To be honest, I don't like the style of coding done in Wordpress. All that global functions… bwah. But on the other hand, people managed it somehow to create a flexible and extensible product. I don't know how it is to make huge projects with Wordpress, but for my little own blog it gave me everything I need so far. This time I wanted to create localized pages, which have been served by PIWI before. In order to reduce the overhead for my website, I decided to use Wordpress as one and only the system. It is updated in regular periods and PIWI is not. It has tons of plugins and PIWI not. However, PIWI is more to my taste of professional software and I will continue to use it on more enterprise developments.

Localization does not work out of the box in Wordpress. But the concepts in Wordpress allow to use a combination of custom fields and themes to make this work. I refused to translate all my blogposts and for only pages it is very straightforward.

First I created my pages in the wordpress pages section. In the Custom fields I used a new meta key named "language" and put as value "de" or "en" for german or english.

I couldn't use the Pages Widget, because I need to generate my navigation out of the selected language. Not to conflict with my current implementation I duplicated my current theme and used the preview function to work on it (on a live system, which is a really great feature for prime time programming). I should mention that I created two pages, one for each language, only stating that the language has been changed.

Next step was to include the language flags in my theme (header.php in my case). I linked the flags to my newly created "language switched" pages. Additionally I have added the param language=de (or en for english) to the link. My complete link for the german language is as follows:

http://www.grobmeier.de/deine-sprache-wurde-umgestellt?language=de

Ok, I am now able to link between the languages. In my header.php I wrote this:

session_start();

include(TEMPLATEPATH . '/grobmeier-session.functions.php');

I started the session myself - it seems Wordpress frontend doesn't use session at all. In the included new php grobmeier-session.functions.php I create a function for storing my language key. This function has been stolen and modified from PIWI:

function grobmeier_getUserLanguage() {
    $supported[] = 'de';
    $supported[] = 'en';

   if (isset ($_GET['language'])) {
       if (in_array($_GET['language'], $supported)) {
            $_SESSION['language'] = $_GET['language'];
       }
    }

    if (!isset ($_SESSION['language'])) {
        $_SESSION['language'] = 'en';
    }
    return $_SESSION['language'];
}

Now I just need to generate the navigation. I'll do this with a prepared function of wordpress. On the correct place in my header.php I wrote:

As you might see, the navigation is build on the user language setting. And that's all - you'll deliver localized versions of your pages from now on. I should mention this works with 2.9.2 and I think earlier versions too. You just need to create pages with the correct custom field setting.

12 Mar 2010 10:26am GMT

For years I've maintained a page of Macintosh Tips and Tricks. It's one of the most referenced pages on my blog, so someone must be using it, despite the fact that it was only up to date for Mac OS 10.5. I've finally gotten around to updating it for my current world. I hope it continues to be useful.

12 Mar 2010 1:10am GMT

I think I've come to understand why Eclipse leaves me always feeling a bit frustrated. Yes, it is more stable than IDEA, uses less memory, has some documentation, and a lot of acceptance ... but even so, it just leaves me cold (and I was an early adopter, signed up for the beta way back in 2000!).

Keystrokes are not modal

The fact that I can type a common keystroke into an Eclipse window and not know what it will do is painful. How a keystroke is interpreted depends on what perspective is active, what view or editor has focus, and what kind of data is being edited in the editor. That's dead wrong; keystrokes are about muscle memory, and muscle memory remembers motion, not context. The end result is that I get frustrated hitting keystrokes and seeing nothing happening. It doesn't help that I cycle between Mac and a PC on most days.

You can't have it your way

A tool as powerful and extensible as Eclipse walks the tightrope of offering lots of features and customizations without overwhelming the user. Alas, Eclipse is lying in a broken heap fifty feet below that tightrope. Eclipse has an unending set of options and defaults for things I don't care about, but anything I do care about seems to never be presents. Here's a few ideas of the top of my head:

• Stop running launches when I close the project (I often have to kill them from the command line)
• Give me a quick way to stop all running launches
• Why so many steps to implement an interface? It's the second most common thing I do!
• How about a button to quickly relaunch the current running launch?
• Why are the available refactorings so paltry and where are the 3rd party ones?

Who's eating their own dog food?

When I used IDEA, I was constantly struck by little details that showed that the IDE developers were also its prime users. For example, it has open-type and open-resource dialogs much like Eclipse ... but each recognizes the keystroke for the other, so that if you mistakenly activate the open-type dialog, you just hit the normal keystroke to switch over to open-resource. Eclipse makes you cancel the dialog first.

Another example: in IDEA if you rename a field, it notices the getter and setter and will offer to rename those as well.

IDEA also has lots of quick fixes everywhere, such as "implement this interface" and lots of other tiny, cool things I miss every single day I use Eclipse. It's been about a year since I gave up on IDEA and I still miss it.

Is it cultural or organizational? Eclipse gives me the impression that day-to-day developers either have no concept of how the IDE gets used (and what rough spots are causing some serious chafing) OR they are somehow prohibited from fixing things that are obviously wrong.

If you love IDEA so much why don't you marry it?

So why don't I use IDEA anymore? Two main reasons: first, it's become very bloated, to the point that unless you go in and shut off a ton of features, it's unusable on my hardware. Merlyn has the same problem doing GWT work on his MacBook Pro ... all the help it gives you comes at a cost in terms of CPU and memory utilization and some instability.

Secondly, I tried (even before IDEA went open source) to use IDEA in my training labs and I hit a stone wall of non-acceptance. Switching to Eclipse was a benefit to my students since, even running in Ubuntu instead of Windows, it was familiar and easy to navigate. It also out-performs IDEA inside my Ubuntu Virtual Machine. I simply lack the ability to switch between the two on a constant basis without getting completely confused and frustrated. I had to choose one, and I chose Eclipse: stable and accepted, even if it is brain dead.

Why call it Ugly?

One thing I don't get is how many people claim Eclipse is "ugly" and IDEA "beautiful". I found IDEA to be overly chock-full of modal dialogs and a number of improperly resized (or non-resizable) dialogs and windows. It's a real dog's breakfast in terms of UI, and has the classically ugly Swing look and feel.

I've always found Eclipse to look sharp and somewhat elegant. You can have a debate about the technical merits of SWT vs. AWT and Swing, or the ability to tune Swing to look like SWT ... but SWT out of the box is simply a better L&F visually.

On a Mac they both suck at keyboard navigation, though.

There, I've vented. See what going cold-turkey from Twitter can do?

12 Mar 2010 12:41am GMT

• NexPod, Freedom of Espresso : Nespresso-compatible capsules - fill up with your own freshly-ground coffee and use in any Nespresso machine
  (tags: nespresso coffee capsules espresso)

• Wall anchors & plasterboard/dry lining walls : Boards thread with good advice regarding wall fixings for drylined walls
  (tags: diy boards walls howto)

11 Mar 2010 11:05pm GMT

Good news! The Apache Incubator Log4PHP project graduated as subproject to the Apache Logging project. This is really a big step forward! The next days I will try to push the real work behind a graduation forward, moving webpages and svn and such. After that move we'll continue with a new small release. Cheers all!

11 Mar 2010 6:13pm GMT

Apparently this is the first hurricane, that we know of, to ever form in the South Atlantic. Roughly it's due east of Uruguay and a bit North while Rio de Janeiro is a bit off the top of this image.

11 Mar 2010 5:47pm GMT

this is hilarious!

http://qntm.org/gay

11 Mar 2010 5:03pm GMT

A good friend of mine is embarking on a new career at Google. Bravo! I think this will be a really exciting move for him; he's a special individual and I think he will do quite well. He's spent too much time working in organizations that wasted his talents, and it's past time for him to have an opportunity like this.

We happened to be talking the other night, and he was describing the process known as Allocation. As I understand it, it goes something like this:

• 
  
• You interview at Google. You do well, you like Google, they like you, you have the right stuff, and they make you an offer.
• 
  
• You accept.
• 
  
• This process is being repeated, simultaneously, with dozens of other individuals (Google are now quite a large organization). So you become part of a pool of talent who are all entering Google at the same time.
• 
  
• Every so often (monthly? bi-weekly?), Google convene the Allocation Review Board, comprised of some set of HR personnel, together with the hiring managers of various projects who have budget ("open reqs").
• 
  
• The Allocation Review Board sit and study the available talent in this pool. Each project has a certain amount of "points" to spend; they discuss things somewhat, then they each "bid" on the incoming fresh meat.
• 
  
• This information is then entered into a computer, possibly along with the results of a questionnaire you've completed about yourself. An algorithm then crunches the data, and performs the allocations.
• 
  
• Your precise position, including your title, job description, chosen team/manager, and initial project(s), are now known, and you are informed.
• 
  
• You show up for your new job!
• 
  

When my friend described this process to me, I was aghast, horrified. It sounded paternalistic and demeaning; I am not a number! Is it just my age? Do all companies do this nowadays? Am I misunderstanding the process?

Has anyone been through this, and willing to share their experience? I'd love to learn more...

11 Mar 2010 4:57pm GMT

Recently, I'm on vacation. The video was recorded when I went to a mart and bought some computer stuff.

The scene looked like a road of cherry blossoms. The computer was in my trunk but, I couldn't control my speeding instinct. :)

11 Mar 2010 2:24pm GMT

If you download the latest source from SVN you will find that you can now set the useCompression option on the Connection URI and your Message bodies will be compressed using a Java compatible ZLib deflater. This allows you to reduce larger payloads for faster transmission on the wire and should be fully compatible with the AMQ Java client and with the NMS.ActiveMQ client if you use the DotNetZip library in a compression policy, more about that later.

11 Mar 2010 1:46pm GMT