fidzu : Plone

Rich Javascript frontends, mobile applications and third party software need an API to interact with your CMS. Let's give Plone one!

01 Aug 2014 11:21am GMT

Some notes on EuroPython 2014 that ended last Sunday with over 1200 attendees.

31 Jul 2014 4:12pm GMT

On one of our sites we wanted to provide a search portlet that would search only in a particular section of the site, instead of everywhere in the site.

Warning

The following instructions will cause all search portlets on a site to behave the same way, so it's not an ideal solution if the same site needs to provide the default search portlet behaviour.

Example

In our example, we want to limit search results to those that are in the section of the site representing the faculty/staff handbook. The ID of the site is "Provost"; this is distinct from and not necessarily the same as the URL of the site, which in our case happens to be http://www.uwosh.edu/provost. You will need to know the ID of the site, because it appears in the portal_catalog's path index.

Instructions

• Add a search portlet
• Go to ZMI -> portal_view_customizations
• Customize search.pt
• Just after the line that says <div class="LSBox"> add this tag <input type="hidden" name="path" value="/Provost/Main Highlight/handbooks/online-faculty-staff-handbook" />
• Press the Save Changes button

Explanation

What the new <input> tag does is append a search parameter limiting results to those that are in the specified section of the site.

Testing

To test this, use the search portlet to search for a term, and then use the standard Plone search box (top right corner of each page) to search for the same term. You should see fewer results from using the search box.

30 Jul 2014 6:29pm GMT

Overview with links to more information about a recent talk Tim Stollenwerk gave about the Jenkins Continuous Integration Server at the Jenkins User Conference/Europe.

30 Jul 2014 5:40pm GMT

30 Jul 2014 10:45am GMT

TL;DR: Production-level Plone on free-tier Heroku: https://github.com/niteoweb/heroku-buildpack-plone

First, a bit of history: it was year 2006 and I was realizing that I was not made to be an academic. I made my first strides into entrepreneurship and being in IT, the first logical step was to create a few websites and try to get paid for my work. I did have some programming experience but haven't done any web development yet. I heard PHP was not the ideal solution so I started looking elsewhere. In my student association I heard about this Plone thingy and gave it a go: I purchased a 20€ per month shared Plone hosting account at Nidelven IT and started hacking. It was the Plone 2.x era and boy was I productive! I threw in content, installed a ready-made theme and did some TTW tweaks. Done! First customer happy! Rinse & repeat, upgrade to a beefier server, rinse & repeat. NiteoWeb Ltd. was born.

Fast forward to a couple of months ago: we used GeckoBoard to drive a wall-mounted dashboard display. GeckoBoard works fine, but they want almost $60 per month if you want to drive more than one display. Sounds quite expensive for a bit of HTML and JavaScript, doesn't it? So I looked around for alternatives and one of them was the Dashing dashboard from Spotify. I was reluctant to even give it a try as it was a self-hosted Ruby app. And I didn't know *any* Ruby. But there, in their documentation, I found a short guide on how to deploy your very own version of a sample dashbord to your personal Heroku account. And when I say short, I mean short! I copy/pasted 6 simple commands into my console and that was it! My very own dashboard! After it was running I was motivated enough to read through Dashing's docs and did a few minor tweaks. One "git push heroku master" later, my changes were again deployed to Heroku and showing up on my dashboard display. Wow, is this developer-friendly or what!

My mind drifted and I thought: Boy wouldn't it be nice if a non-Plone person could come to a Plone add-on page and create their very own Plone instance with the add-on installed, and they could immediately start using it. Uhm ... why not? Why don't we, as a community, provide "private" demos that people can use? Is there something that prevents us from using Heroku for demos, the same was as Dashing, and many other Ruby products, use it?

Turns out, there isn't. During the Plone dinner at EuroPython 2014 conference last week, I ordered a few beers and got hacking. Goal: get Plone to run on a free-tier Heroku account.

There have been attempts to run Plone on Heroku before, but they failed because they took the wrong approach. Look, Heroku, by default, supports Python apps that are installed with ``pip``. Previous attempts were all focused on fixing Plone so it could be installed with pip. And they failed, Plone's ecosystem is just too complex.

I decided to take a different approach: buildpacks. Heroku allows you to build *anything* on it. So I created a buildpack that supports zc.buildout. Once I got that done, it was not far from getting Plone installed on Heroku.

The next roadblock came in the form of Heroku's ephemeral filesystem. Everytime your Heroku "dyno" is restarted, the filesystem is recreated. And you lose your Data.fs. Humpff. Wait, but what about the PostgreSQL that Heroku offers? A production-quality PostgreSQL, for free, with a 10k row limit. That could work! So, add in support for RelStorage and you have a production-ready Plone site running on Heroku. For free. And you are not limited to one, you can have as many as you wish, on the same account. Heroku really is awesome!

So, Plone is suddenly again a viable option for college dropouts starting their businesses! No need for system administration knowledge, how to setup Nginx in front of Plone, how to do proper backups, just a few command-line commands and your site is online!

And, our add-ons can now have demos. If you use Data.fs instead of PostgreSQL, the demo instance's data will be recreated at least once per day, giving visitors an up-to-date demo instance, displaying what your Plone add-on does and how it looks.

Does this really works? Hell yeah it does! This blog has been running on Heroku since last week! And here's a Plone 4.3 demo, a Plone 5 demo and a collective.cover demo. Wanna see plone.app.mosaic in action?

Why doesn't your add-on have a demo yet? Follow instructions on https://github.com/niteoweb/heroku-buildpack-plone and showcase your add-on to the world!

30 Jul 2014 7:05am GMT

Here are some stories on how you can go wrong with naming, especially in open source software.

29 Jul 2014 2:33pm GMT

I had a fun time at EuroPython 2014 in Berlin last week. It was a very well organized conference and I enjoyed meeting old friends again as well as meeting new people. Before I went I was a bit worried with the amount of attendees it'd feel too massive; I had that experience at a PyCon in the US a few years ago. But I was pleasantly surprised it didn't -- it felt like a smaller conference, and I liked it.

Another positive thing that stood out was a larger diversity; there seemed to be more people from central and eastern Europe there than before, and most of all, there were more women. It was underscored by a 13 year old girl giving a lightning talk -- that was just not going to happen at EuroPython 5 years ago.

This is a very positive trend and I hope it continues. I know it takes a lot of work on the part of the organizers to get this far.

I gave a talk at EuroPython myself this year, and I think it went well:

28 Jul 2014 12:00pm GMT

As I described in my previous blog post traceview can run on a stand-alone Plone site, i.e. without no front-end webserver. In this blog post I will described the required steps to get started on using traceview for Plone.

Step 1 - login to traceview

The first step is of course to login to the traceview interface, if you don't have an account, sign up at AppNeta's homepage. Apparently, they got a free plan if you just have one application, i.e. one Zope instance . After you have logged in go to "Get started" and select "Install host agent". Then you will see the appropriate command you should run on the server. This will install the daemon that sends data to AppNeta, and also various C libraries.

Step 2 - set up your Python environment

You need the oboe module to use collective.traceview. It can be installed either via pip/easy-install. I recommend to use a virtual environment if you choose this approach. It can also be installed via buildout. The oboe egg is on pypi so it should not be any problem just adding it to the egg list. Also, remember to add collective.traceview to the egg list.

Step 3 - setting up environment variables

You need a series of environment variables to inform Plone that it should do tracing and also how it should do the actual tracing. Ideally, you should set these environment variables via buildout since you may have different profiles, e.g. test, production, etc. You should have a buildout configuration similar to this:

[instance]
...
environment-vars +=
    TRACEVIEW_PLONE_TRACING 1
    TRACEVIEW_IGNORE_EXTENSIONS js;css;png;jpeg;jpg;gif;pjpeg;x-png;pdf
    TRACEVIEW_IGNORE_FOUR_OH_FOUR 1
    TRACEVIEW_SAMPLE_RATE 0.3
    TRACEVIEW_TRACING_MODE always

Most of these variables are quite obvious, and each of them is explained in the README of collective.traceview. Basically, what the configuration says is, that Plone should trace (TRACEVIEW_PLONE_TRACING 1) and there should be done no tracing for js, css, etc files (TRACEVIEW_IGNORE_EXTENSIONS js;css;png;jpeg;jpg;gif;pjpeg;x-png;pdf). Furthermore 404 pages should not be traced (TRACEVIEW_IGNORE_FOUR_OH_FOUR 1), and we trace just 3 out of 10 requests (TRACEVIEW_SAMPLE_RATE 0.3).

Just run buildout now, and new traces should occur in the "Default app" on the traceview interface. Just so you can see how an example trace looks like, I have attached a screenshot below (each layer can be viewed and further information is provided, for example, for catalog queries the entire query is logged).

Happy tracing!

25 Jul 2014 9:36pm GMT

Finally back for summer vacation, three weeks without a single line of code - how refreshing! It has been a long time since we did any work on collective.traceview but we have finally implemented a feature that has been wanted for long.

The current state of the module relies on a front-end web server (for example, apache or nginx) to kick the trace started. What happens if that the oboe module in apache will generate a unique trace id, referred to be the X-Trace, which will be sent to Plone and it will be used as reference to do the actual full-stack tracing. This is not always a good idea to do it in this way, consider the following scenarios:

• You have no apache (or nginx), maybe just varnish in the front and it distributes the requests directly to the Plone instances, why wasting CPU power on running an additional web server.
• You have apache, varnish and then Plone. You will get quite bogus traces for cached pages served directly by varnish, just showing apache.

For these scenarios it makes much more sense to start the tracing in the ZServer HTTP server. So we added an additional patch to the product patching the Medusa based implementation (I was not turned into stone for looking at it). This patch can start the tracing, giving us additional benefits than leaving out the front-end web server. It is now also possible to see the actual waiting time from the request hit ZServer to it is being served by the publisher. This may give you a hint if you need more ZServer threads or more Zope instances. For example, on the screenshot below, you can see a little waiting time around 9.00 AM between the ZServer HTTP server and the Zope publisher.

If you want to play with it just follow the instructions given in the README of collective.traceview to set the proper environment variables. These can preferably be set using buildout. The feature is provided by collective.traceview 1.3.

Please let me know if you have any feedback on the feature, or questions on how using it.

UPDATE: In github master is experimental Chameleon support, so you can see how much time that is spend on rendering individual templates; the file name of the template is logged, so it is easy debugging template renderings that seems to be a bottleneck.

24 Jul 2014 9:07pm GMT

jQuery is a JavaScript library that allows you to do many of the same things as regular JavaScript, but in a much easier way that also results in less code. It is not a separate language, so regular JavaScript (variable assignments, loops, etc) can still be used alongside jQuery code. The library contains dozens of functions that allow you to manipulate DOM elements, retrieve information, and show animation. It is extremely popular around the web, as it is used on over 60% of the top 100,000 websites.

This example shows how the code is easier and shorter. The first line is JavaScript, the second jQuery, and both lines are hiding an element with the id element:

document.getElementById('element').style.display = "none";
$('#element').hide();

jQuery uses CSS-style selectors, which makes the language super easy for front end developers to pick up. This allows a lot of flexibility when narrowing down the specific element(s) you want to manipulate.

Tips

Generally the code written is for things that happen once the page has loaded (on document ready). To make sure your code doesn't run until the page is loaded, wrap it like this:

(function($) { $(function() {
    // code here
}); })(jQuery);

One tip to watch out for here is that images don't count in determining whether the page is loaded yet. So if you are doing any manipulation on images, make sure they are loaded first by wrapping the code in a window ready function (below). If you don't do this, the code may not apply each time the page is loaded, and you'll notice that checking for the height and width returns 0.

$(window).load(function(){
    // do something with images
});

Similarly, you cannot retrieve the height or width from elements set to display: none, but you can get this information if the element has visibility: hidden.

Use in Plone

jQuery has been included in Plone since Plone 3.1. More recently, the library was split out into a separate package (plone.app.jquery) in order to make updating jQuery itself easier. The jQuery community is very active in updating the code and pushing out new releases. One thing to watch out for while updating is functions being deprecated, which happened with .live() (one of my regulars) in 1.7.

Examples of Use

collective.easyslideshow, our slideshow product, uses a jQuery plugin called jquery.cycle. indianahistory.org displays easyslideshow in the upper right corner of the homepage. Just beneath that is a custom slider that was also created with jQuery.

Resources

1. The jQuery site contains great tutorials and documentation for each function: http://jquery.com
2. jsFiddle provides a handy testing resource for various JavaScript libraries, including jQuery: http://jsfiddle.net
3. For testing the quality of your code, there is JSLint: jslint.com. JSLint is also available as an add-on for various code editors like TextMate and Sublime Text; mine is set to check for errors and warnings each time I save.

23 Jul 2014 12:00pm GMT

This is a bugfix release which fixes several problems reported by users.

Lingua is a Python package that helps you find translateable texts in your code, and generate POT-file for them. Think of it as xgettext on steroids for Python.

Read entire article.

23 Jul 2014 12:00am GMT

Recently we upgraded a Plone 3.3 site to 4.3. Here are some thoughts regarding this.

Recently we moved a site from a long-time customer to Plone 4.3.

First off, the list of step to follow in http://plone.org/documentation/manual/upgrade-guide/version/ were quite exhaustive. There were some things that we did differently:

• We went straight from Plone 3.3 to 4.3.3, skipping the intermediate versions (4.0, 4.1 and 4.2). For our site this worked.
• After that, we ran all upgrade steps at once.
• We had to manually remove some Javascripts (mostly Kupu) from the registry, in order for TinyMCE to work.

The sooner you upgrade to a more recent stable version, the smaller the hassle. Also, there's a performance improvement (see image), and some nice new features. See http://www.sixfeetup.com/blog/five-reasons-why-waiting-isnt-the-best-upgrade-strategy for more reasons to upgrade.

18 Jul 2014 9:10am GMT

TL;DR: rest_toolkit is a new framework to create REST applications, build using Pyramid.

For two recent projects involved implementing a REST service. The experience of doing that led me to create a new simple but extendible toolkit for buidling REST applications. Want to see how simple? This is a full REST application:

@resource('/')
class Greeting(object):
    def __init__(self, request):
        pass

@Greeting.GET()
def show_root(root, request):
    return {'message': 'Hello, world'}

quick_serve()

Read entire article.

18 Jul 2014 12:00am GMT

Over the last couple of years, we have changed our internal development tools. We do our own version of Agile development and have found these tools best meet our needs.

For Sprint planning, we use Trello. Our team members are in many locations, so having a web-based tool to outline our Sprints has been very important. We create cards for User Stories or issues and work from the boards for Sprint planning and working on Sprints.

Our code is stored in Github. We previously ran our own SVN server, but slowly migrated all of our code to Github. It is a great tool. Our workflow for using Github is

• Each component has a separate Repository.
• Each Repository has a Master branch.
• Each Repository has a production branch that contains the code currently in production. This allows us to continue working and merging to Master, but also be able to fix problems in production easily from the production branch,
• Developers branch off Master and code the Trello card they are working on. Once the code is completed and unit tested, the developer creates a Pull Request in Github. The Pull Request is to merge the code into Master.
• A Pull Request triggers a code review by another team member. Code reviews generally result in a review of the code as well as a manual test of the code.
• Pull Requests are also unit tested using automated testing via Travis-CI
• Once a Pull Request is approved, it is merged and the branch is deleted from Github.

Most of our meetings are held on Skype. Skype has the simplicity of making a phone call and is mostly reliable. We also use Google Hangouts when we need to share code or other screen sharing. It works really well, but if not as easy to start up as a Skype call.

Our team relies on IM. We have a Jabber server that some of the team uses and others use Google Talk or Hangouts. IM is our virtual hallway and is a key part of our communication.

14 Jul 2014 8:14pm GMT