09 Nov 2011

feedPlanet Zope.org

Updated MiniPlanet, now with meta-feed

My MiniPlanet Zope product has been working steady and stable for some years, when suddenly a user request came along. Would it be possible to get a feed of all the items in a miniplanet? With this update it became possible. MiniPlanet is an old-styl...

09 Nov 2011 9:41am GMT

07 Nov 2011

feedPlanet Zope.org

Welcome to Betabug Sirius

It has been quite some time that I announced_ that I'd be working as a freelancer. Lots of stuff had to be done in that time, but finally things are ready. I've founded my own little company and set up a small website: Welcome to Betabug Sirius!

07 Nov 2011 9:26am GMT

03 Nov 2011

feedPlanet Zope.org

Assertion helper for zope.testbrowser and unittest

zope.testbrowser is a valuable tool for integration tests. Historically, the Zope community used to write quite a lot of doctests, but we at gocept have found them to be rather clumsy and too often yielding neither good tests nor good documentation. That's why we don't use doctest much anymore, and prefer plain unittest.TestCases instead. However, doctest has one very nice feature, ellipsis matching, that is really helpful for checking HTML output, since you can only make assertions about the parts that interest you. For example, given this kind of page:

>>> print browser.contents
<html>
  <head>
    <title>Simple Page</title>
  </head>
  <body>
    <h1>Simple Page</h1>
  </body>
</html>

If all you're interested in is that the <h1> is rendered properly, you can simply say:

>>> print browser.contents
<...<h1>Simple Page</h1>...

We've now ported this functionality to unittest, as assertEllipsis, in gocept.testing. Some examples:

self.assertEllipsis('...bar...', 'foo bar qux')
# -> nothing happens

self.assertEllipsis('foo', 'bar')
# -> AssertionError: Differences (ndiff with -expected +actual):
     - foo
     + bar

self.assertNotEllipsis('foo', 'foo')
# -> AssertionError: "Value unexpectedly matches expression 'foo'."

To use it, inherit from gocept.testing.assertion.Ellipsis in addition to unittest.TestCase.


03 Nov 2011 7:19am GMT

02 Nov 2011

feedPlanet Zope.org

Obviel

Introduction

This document is an introduction to Obviel from the perspective of how it was developed. For those impatient with history, you can just go to http://www.obviel.org and read the documentation there, however.

Genesis

A few years ago I worked on a dynamic web user interface on a project for Gocept, using JavaScript of course. There I invented a somewhat unusual approach to do:

  • use JSON Template to do client-side templating
  • the server would return a JSON object and the client side template would then be used to render it.

Callback Complexity

In the fall of 2010, I working on a few related customer projects to create complex web applications that needed a good UI. As they were applications, not web sites, I decided to create those UIs using a single-page approach. The server would just present a single web page with JavaScript to build and update the UI.

We decided to use jQuery for those projects. jQuery is very popular, with a vibrant community with a lot of components choose from. It would mean we would have to do more integration than we would need if we chose a more integrated framework such as YUI, which I also have experience with, but I was a bit tired of big frameworks. And jQuery's terse and powerful way to express things is attractive. And you just have to make a choice to see what happens at some point, right?

The JavaScript code in those applications was quickly growing out of control. We were trying to do proper REST: the client would just start with a single URL, which would resolve to a JSON object that would have links to other JSON objects on the server, which would in turn have links to other JSON objects. Proper REST is nice: you can decouple your client from the server more, because your client code doesn't depend on the specific URL structure of your server -- instead it just follows links.

But doing proper REST with JavaScript without taking special measures tends to result in endlessly nested callbacks in callbacks in callbacks. The code was too tightly coupled and hard to follow.

This made me think about the stuff I did at Gocept again. I felt there was quite a bit of potential in those patterns, potential I hadn't explored fully yet. So I added it to one of our projects and started to rewrite the UI to use it, to try to clean things up.

People

At this stage more people started to be involved. Ideas tend to get better with good input, and with usage by a larger group of people. I will forget some of the people involved in this project, for which I apologize, but I'm going to drop some names now.

The original idea was born at Gocept. I was doing the work for Christian Zagrodnick there, and I had some good conversations with him about it then. He also used the early code in a few of his own projects later on. Just as I had checked in the view stuff into the new project I was doing I found myself in Germany at Gocept again, and had further productive conversations with him about it. Thanks Zagy!

Going to Germany, where Gocept is based, from where I live, the Netherlands, means a train ride, and in the train I had the opportunity to bounce ideas off Jan-Wijbrand Kolman (and him ideas off me), something I always enjoy tremendously. The biggest thing that came out of that was Fanstatic, but that's the topic for another discussion. It also helped me think through what later became Obviel. Thanks JW!

The idea developed around this time was that the JSON returned by the server to JavaScript should have a type marker to describe what kind of JSON we are dealing with then. In Obviel we call this type marker an iface, short for interface. On the client, we could then hook up views to ifaces. This way if the client is confronted with some unknown JSON, it would know what to do with it by looking up the appropriate view for that iface. This helps with REST and therefore with loose coupling: the client code not only doesn't need to know what resources the server is presenting, but also would be able to know what resources are on what URLs; the resources would disclose this themselves. In this, "JavaScript views" (as we were calling it at the time) resemble dynamic view lookup like practiced in the Zope Component Architecture (don't you all run away now, though!).

Practice

New approaches get better when you try to actually use them in practice. You learn a lot. The customer projects I was doing were too big to do on my own, I was working with Izhar Firdaus on them. Izhar gave me a lot of invaluable feedback and improvements and came up with new patterns of using these "JavaScript views" (as we were calling them) throughout the projects that use Obviel; I had great pleasure working with him for over a year.

I think I already rewrote the system halfway back then, and I remember Izhar doing something similar. Eventually after some experience I sat down and started to write a document describing how I wanted this JavaScript views thing to work. So I got Guido Wesdorp involved. This meant Izhar and I were freed up to build more application specific code and gain experience with it.

Working with Guido in this way was a new experience: I had designed a framework but Guido was implementing it. Guido focused on making the code solid. I, Guido and Izhar together came up with various new approaches.

Forms

The projects we were working on needed a number of complex web forms. I have a lot of experience with server-side form libraries. Some time before I had been in a rich-client project with Jasper Op de Coul and he suggested creating a JavaScript form library instead. This would offer some benefits, such as inline validation for everything by default, and web-framework neutrality. While we didn't implement the idea at the time, it did stick in my mind.

Here we needed lots of complex forms again, and server-side validation with errors didn't really fit well in a rich client UI. So I sketched up a way to do JavaScript forms using a new technology jQuery datalink. The concept of Obviel forms is to maintain a JavaScript object with the form contents in parallel to the HTML form (an earlier conversion with Antonin Amand helped me develop this concept). When the user modifies the form, the JavaScript object is automatically updated. A form submission then simply submits that JavaScript data object as JSON to the server.

I had Guido go ahead and start building the form library for us.

Rewrites

"JavaScript views" had already gone through a few incarnations. I'd rewritten it at least once, Izhar had given it another shot, and Guido had rewritten it again.

Meanwhile Sylvain Viollon at Infrae had started using Obviel as well. He had decided to do its own rewrite, to my chagrin at the time, as now we had two parallel versions. But it turned out for the good: he had introduced some new ideas into his version that were very useful: when a view was rendered, it would be bound to the object it had just rendered, and the element it was rendered on. These bound views would allow a better structuring of view code than passing this information along as parameters all the time.

We kept learning however, and after a while, when Guido had already moved on from the project, several things in Obviel forms in particular started to be frustrating.

So I decided to rewrite Obviel forms cleaning it up. Among new things introduced were composite sub-forms and a generic repeat widget to allow the user to enter repeating sub-forms as well, and a way to maintain the inline error messages as a datalinked object as well. Govert Buijs, who had started to work on the projects with us, also helped work out advanced form validation scenarios.

On a roll, I decided to rewrite the Obviel core as well, introducing Sylvain's ideas about bound views. Along the way I also started using the new jQuery deferreds in its implementation (Sylvain then turned out to have done so in his version as well, in parallel).

Documentation

Now it is time to attract more people to Obviel. It's been battle tested in a number of big projects.

To attract people to a project you not only need good code. We do have good code I think: it's unit tested and coverage tested. But you also need great documentation. Along the way I and Guido had written bits of documentation about Obviel. It took me months of stops and start to get this documentation in the state it is now, so that newcomers to Obviel aren't lost.

So, if you are curious about Obviel and want to find out more, go to http://www.obviel.org

And give us feedback, please!

02 Nov 2011 4:14pm GMT

gocept talks at PyCon DE

No blog post for quite a while… part of the reason is that we gocept developers were busy preparing talks for PyCon DE 2011. As result, we presented an impressive number of 7 talks/tutorials at this lovely conference.

Curious? Here is a list of all sessions (most with video recordings). Please be aware that nearly all of this stuff is in German.

Tutorials

Talks


02 Nov 2011 1:34pm GMT

27 Oct 2011

feedPlanet Zope.org

Security announcement: Zope Hotfix 20111024

The Zope Security Team has announced a hotfix, Products.Zope_Hotfix_20111024, for a vulnerability in the Zope Application Server, versions 2.12.x and Zope 2.13.x.

Most Plone installations are not vulnerable and do not need the hotfix. Please read this announcement carefully for instructions on how to determine whether or not you need to apply the hotfix.

The announced vulnerability is in Zope's default authentication system. When a Plone site is installed in a Zope database, the Plone installation usually replaces the basic Zope authentication system with the Pluggable Authentication System (PAS). PAS is not vulnerable to this problem.

You may verify that you are using PAS by using the Zope Management Interface to examine the acl_users object in the root of your Zope database.

If the title of the object reads "User Folder at /acl_users", your system is vulnerable and you should apply the hotfix.

If the title of the object reads "Pluggable Auth Service at /acl_users", your system is not vulnerable.

Versions Affected: Zope 2.12.x <= 2.12.20 and Zope 2.13.x <= 2.13.10 that do not have the Pluggable Authentication System installed.

Versions Not Affected: Zope installations where Plone installation has replaced the Zope baseline authentication system.

See the Zope Hotfix Announcement for details on installing the hotfix.

General questions about this announcement, Plone patching procedures, and availability of support may be addressed to the Plone support forums. If you have specific questions about this vulnerability or its handling, contact the Plone Security Team.

To report potentially security-related issues, please send a mail to the Plone Security Team at security@plone.org. The security team is always happy to credit individuals and companies who make responsible disclosures.

27 Oct 2011 9:27pm GMT

#planetzope Just changed the filter at planetzope.org to ['zope', 'zope2',

#planetzope

Just changed the filter at planetzope.org to ['zope', 'zope2', 'zope3', 'ztk', 'zodb', 'bluebream', 'planetzope']. That is you will miss most of the 'plone' related news.

27 Oct 2011 10:09am GMT

25 Oct 2011

feedPlanet Zope.org

#zope #planetzope I think it is about time to close planetzope.org anytime

#zope #planetzope

I think it is about time to close planetzope.org anytime soon - the service has been online for almost 6 years. There is no zope related news for some time now - except for security releases. Almost all of the news can be found on planet.plone.org.

If i get around, i'll publish the planetzope source to github. If some of you want to take over, the domain is reserved for the next 18 months.

Anyway, just contact me on the website.

PlanetZope.org - Zope related news

25 Oct 2011 11:18am GMT

Hotfix for security vulnerability

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Zope security response team, I would like to announce
the availability of a hotfix for a vulnerability inadvertently
published earlier today.

'Products.Zope_Hotfix_20111024' README
======================================

Overview
- --------

This hotfix addresses a serious vulnerability in the Zope2
application server.  Affected versions of Zope2 include:

- - 2.12.x <= 2.12.20

- - 2.13.x <= 2.13.6

Older releases (2.11.x, 2.10.x, etc.) are not vulnerable.

The Zope2 security response team recommends that all users of
these releases upgrade to an unaffected release (2.12.21 or
2.13.11) as soon as they become available.

Until that upgrade is feasible, deploying this hotfix also
mitigates the vulnerability.


Installing the Hotfix:  Via 'easy_install'
- -------------------------------------------

If the Python which runs your Zope instance has 'setuptools'
installed (or is a 'virtualenv'), you can install the hotfix
directly from PyPI::

  $ /

25 Oct 2011 4:54am GMT

24 Oct 2011

feedPlanet Zope.org

Self-publishing a book, Part 1: Why and How

Nobody that reads this blog can have missed that I've written a book and self-published it. It's been a long and interesting journey, and I think I should share some of the experience.

The book started out of necessity. Not mine, but the communities. There was no good documentation on how to port to Python 3, and I realized that if it was going to be made, I had to do it, and that it would be a crazy amount of work. So I wondered if there was any way I could recuperate some of that in money. The idea I had was to write a series of 3-5 articles in Python Magazine, and then merge those articles with some appendices and forewords and things into a small book. I shipped this idea to Brandon Rhodes, the editor of the magazine, and since Python Magazine had a limited time of exclusivity on their articles, it turned out to work fine. I could write the articles, and after their exclusivity ran out, also publish them as a book.

So I started writing, around December 2009 if I remember correctly. During spring 2010 I had finished and delivered three articles, but by that the reorganizing Python Magazine was doing never got anywhere, and the articles were never published.

Publishing

At this time I started looking for publishers, as I now believed I could actually deliver a book. It's a short book, but nevertheless, it's a book. Most publishers, like O'Reilly, will pay out advances, and as such they will only accept your book if they believe in it. So they require you to fill out these long applications full of questions. As it would probably take me days just to fill out these applications and come up with answers to most of their questions (that I felt often didn't apply) I decided that major publishers probably weren't the right thing for me.

I also talked to Packt Publishing. I've reviewed several of their books here, and they have several times mailed me if I want to write a book, often on weird topics that they think have a market, like "Zope Configuration". I said that I've worked with Zope for 10 years and have no idea what "Zope Configuration" means. Packt first said no to my book, as they didn't want to publish a book that was likely to be less than 200 pages long. Then they said maybe, can you please write it in Microsoft Word using our templates? I said that I couldn't. I wanted the examples to be tested, meaning that using Word was out. I needed a format that was basically text.

The option that lay ahead then was self-publishing. This also had the benefit of making it possible to use Creative Commons, something I wanted. After talking to Chris McDonough, who self-published his BFG (now Pyramid) book via CreateSpace, I decided to use CreateSpace as well. CreateSpace has the benefit of being owned by Amazon, and hence you get a much better deal when selling books via Amazon.com from them, then from other self-publishing print houses, and quite a lot of sales will come from Amazon.com.

So I went with Create Space, and I wrote the book in Restructured Text, and used Sphinx and LaTeX to create a PDF output of the book, and I used GIMP to make a PDF cover, using an image of an albino Burmese Python I found on flickr.com by using their creative commons search. Brett Cannon was nice enough to write a foreword which added a nice touch of authority to the book and makes it sound like I know what I'm talking about.

But although I had the text, there was a long path to having a book. In Part 2 I'll write a bit about the toolchain I used and the technical problems I had.


Filed under: python

24 Oct 2011 1:49pm GMT

21 Oct 2011

feedPlanet Zope.org

Carol Ganz, Alan Hoey and Héctor Velarde chosen as Plone Foundation members

Carol Ganz works for Indianapolis-based Six Feet Up. She has been overseeing Plone Tune-ups since 2010 and has frequently represented Plone at events as an evangelist - including NTEN and OSCON in 2011. She's been heavily involved in Plone Marketing for the last 3 years, and was involved in creating many of the reference guides, giveaways. and documents which are used to evangelize Plone.

Alan Hoey works for Team Rubber in Bristol, UK. He is the Plone 3.3 Release Manager, and has been a frequent attendee of Plone conferences, including Naples, Washington DC, Budapest and Bristol - as well as many sprints. He's an active contributor to many add-on packages such as plone.app.discussion, ZopeSkel, collective.recipe.funkload, mr.monster, borg.localrole, and teamrubber.theoracle. Alan is also a member of the Plone Security Team and the discoverer of several Plone vulnerabilities which he helped fix.

Héctor Velarde is a freelance professional located in Mexico. He has contributed code and translations since 2005 and has coordinated the activities of the Mexican Plone Users Group (which he co-founded in 2008) since 2010. He helped organize the 2011 Plone Symposium South America. Projects he has contributed to include TagCloud, ATGoogleVideo, nitf4plone, collective.nitf, collective.atomsyndication, collective.newsticker and collective.upload. His package translations into Spanish include collective.disqus, cioppino.twothumbs and plone.app.dexterity.

The Plone Foundation is the organization formed in 2004 to serve as a supporting organization for Plone and its community. Members of the Foundation are chosen by the Foundation's Membership Committee on the basis of merit - specifically that they have made significant, enduring contributions which benefit the general Plone community.

The Foundation is happy to welcome Carol, Alan and Héctor as our newest members.

21 Oct 2011 5:28am GMT

16 Oct 2011

feedPlanet Zope.org

(Dutch) Python/Django vacature in Utrecht

English note: this is about a job opening, but we really want someone who can speak Dutch, so I'm keeping the text in Dutch, too.

Na twee eerdere vacatures hebben we er weer eentje! We doen het goed, want onze klanten geven ons veel werk :-) Zie de officiële vacature.

Ik zou die twee eerdere vacatures even doorlezen, dat geeft je een aardig idee. Een paar zaken die ik eraan toe wil voegen:

  • Het gros van onze code is open source. We zijn net bezig onze spullen van een intere subversion naar github over te brengen.
  • Iets wat ik me afgelopen woensdag op de Django meetup pas realiseerde toen ik er veel positieve feedback op kreeg: we hebben al best een lekker Django team. 10 personen groot! 10 van de 40 in het hele bedrijf. Lekkere club met veel kennisuitwisseling waarbij je veel nieuwe zaken leert.
  • Hartje Utrecht, dus het zou voor jou wel eens lekker bereikbaar kunnen zijn. 10 minuten lopen vanaf het station. Ken je dat grote kruispunt met de Bijenkorf/We/C&A? We zitten in de achtertuin van de We.
  • Beginners prima! Je moet wel kunnen programmeren, maar je hoeft geen Django core committer te zijn. Ook als je alleen in de avonduren wat met Python "speelt" en overdag met Java werkt: prima.

Voor verdere vragen kan je bij mij terecht, maar voor een sollicitatiebrief is het makkelijker rechtstreeks te mailen naar Helga, zie onderaan de officiële vacature.

Fietsen in de buurt van Utrecht (Langbroekerwetering)

16 Oct 2011 11:00pm GMT

12 Oct 2011

feedPlanet Zope.org

PyCon DE 2011 - The First

What a conference. 200 Pythonistas met for the better part of the last week in Leipzig.

We started out with a tutorial day. More than 80 people took advantage of the opportunity to attend 13 tutorials covering diverse topics including algorithms, database programming, web frameworks, scientific data analysis and Python introduction.

In parallel to the tutorials we had a barcamp with about 30 people that discussed different topics and had lightning talks. Topics included web frameworks and SQL/NoSQL compassions.

The core conference had three parallel tracks with 30-minute and 60-minute slots. Our rather rigorous time management worked out: all presenters stayed within their allocated slots. It is a German conference after all. ;) Over three days we had 55 talks. The topics covered a wide range. Web development and scientific applications were the two largest themes but many other topics such as teaching Python, migration to Python 3 or Python compiler were covered.

All talks are on video and will be posted as soon as the upload bandwidth allows. We had a multinational team of video filmers form the US, UK, Norway and Germany. Thanks for your effort and the great job you did.

We had three keynotes with very different focuses, one at each conference day. Jan Lehnhardt of CouchDB fame provided a perspective on the Python community from the outside. Paul Everitt, early-day Zope activist, draw from his large experience of doing things well ahead of its time. He gave some insight into the past that provided lots food for thought how do things right and how to be successful. Andreas Schreiber, scientists at the German Aerospace Center, showed the impressive use of Python in high-tech. The only conclusion can be: "The future
is all Python!".

The weekend was reserved for sprints. About 30 people gathered at the first sprint day and worked on Python, Cython, Zope and other software packages as well as on the Germany Python website. The second day saw a few less people continuing their work.

All technical and organizational things worked out smoothly. The Wi-Fi was stable over the whole conference, not a single complain. People liked the food (part of it was vegan), projectors and microphones worked. Minor problems were solved in record time by the house technicians. The city tour and the social event were well received judging by the feedback from the
participants.

But the most important thing, you could feel the community. Quite a few people knew each other before. But many saw others the first time in real live. It was the biggest gathering of German speaking Python folks after all. The hall way track was always well attended. People were hanging out late to talk, discuss and deepen connections. The atmosphere was positive all over the place. Everybody could feel it. No email, twitter, irc or skype comes even close to live human interaction. After all, this is what made us human over the last few hundred thousand years.

PyCon DE 2011 is also the birthplace of the Python Software Verband e.V. an organization comparable in scope to the PSF but for the German speaking Python community. It's a good start because we can build on many years of organizational experience of DZUG e.V., the German Zope User Group organization that widened its scope to become the Python Software Verband e.V. This should give the organized German speaking Python community a head start.

Everybody left high spirited looking forward to PyCon DE 2012 that will happen. Details coming soon.

12 Oct 2011 12:24am GMT