fidzu : Plone

This is a bugfix release which fixes several problems reported by users.

Lingua is a Python package that helps you find translateable texts in your code, and generate POT-file for them. Think of it as xgettext on steroids for Python.

Read entire article.

23 Jul 2014 12:00am GMT

Recently we upgraded a Plone 3.3 site to 4.3. Here are some thoughts regarding this.

Recently we moved a site from a long-time customer to Plone 4.3.

First off, the list of step to follow in http://plone.org/documentation/manual/upgrade-guide/version/ were quite exhaustive. There were some things that we did differently:

• We went straight from Plone 3.3 to 4.3.3, skipping the intermediate versions (4.0, 4.1 and 4.2). For our site this worked.
• After that, we ran all upgrade steps at once.
• We had to manually remove some Javascripts (mostly Kupu) from the registry, in order for TinyMCE to work.

The sooner you upgrade to a more recent stable version, the smaller the hassle. Also, there's a performance improvement (see image), and some nice new features. See http://www.sixfeetup.com/blog/five-reasons-why-waiting-isnt-the-best-upgrade-strategy for more reasons to upgrade.

18 Jul 2014 9:10am GMT

TL;DR: rest_toolkit is a new framework to create REST applications, build using Pyramid.

For two recent projects involved implementing a REST service. The experience of doing that led me to create a new simple but extendible toolkit for buidling REST applications. Want to see how simple? This is a full REST application:

@resource('/')
class Greeting(object):
    def __init__(self, request):
        pass

@Greeting.GET()
def show_root(root, request):
    return {'message': 'Hello, world'}

quick_serve()

Read entire article.

18 Jul 2014 12:00am GMT

Over the last couple of years, we have changed our internal development tools. We do our own version of Agile development and have found these tools best meet our needs.

For Sprint planning, we use Trello. Our team members are in many locations, so having a web-based tool to outline our Sprints has been very important. We create cards for User Stories or issues and work from the boards for Sprint planning and working on Sprints.

Our code is stored in Github. We previously ran our own SVN server, but slowly migrated all of our code to Github. It is a great tool. Our workflow for using Github is

• Each component has a separate Repository.
• Each Repository has a Master branch.
• Each Repository has a production branch that contains the code currently in production. This allows us to continue working and merging to Master, but also be able to fix problems in production easily from the production branch,
• Developers branch off Master and code the Trello card they are working on. Once the code is completed and unit tested, the developer creates a Pull Request in Github. The Pull Request is to merge the code into Master.
• A Pull Request triggers a code review by another team member. Code reviews generally result in a review of the code as well as a manual test of the code.
• Pull Requests are also unit tested using automated testing via Travis-CI
• Once a Pull Request is approved, it is merged and the branch is deleted from Github.

Most of our meetings are held on Skype. Skype has the simplicity of making a phone call and is mostly reliable. We also use Google Hangouts when we need to share code or other screen sharing. It works really well, but if not as easy to start up as a Skype call.

Our team relies on IM. We have a Jabber server that some of the team uses and others use Google Talk or Hangouts. IM is our virtual hallway and is a key part of our communication.

14 Jul 2014 8:14pm GMT

This is a recap of Calvin Parker's presentation at the Plone Symposium Midwest 2014

Content is King

Everyone in marketing has become obsessed with content marketing, and the demand for more people in more departments to have a way to create content has driven the creation of more websites.

Fast Forward

The problem with this rapid explosion of content across organizations is many websites have been rapidly created on different platforms with not central strategy.

Plone & Drupal have a 70% Coexistence in Higher Ed

Because of this we have seen that as recent as March of 2014 about 70% of every U.S. university that uses Plone also uses Drupal to some level.

How Do you Control Web Branding, Content & Infrastructure?

Consolidating is an option

Some organizations choose the obvious approach of trying to consolidate all of their websites onto a single platform. This can work and we even have a solution for it called WebUnity, but it can also be expensive and time consuming. You have to:

• Evaluate CMSs and vendors
• Migrate all your content and themes
• Deal with Bit Rot
• Train everyone on the new system

This can also be demotivating and polarizing when people refuse to change.

There is another option: Integration!

• Keep all your content and websites in their existing CMS
• Connect those sites so they can syndicate and track content

UCLA Case Study

Large University with a central IT department, but all of the content management is done independently by the various departments. Integration via a tool like PushHub allows them to have independent teams share content across sites and keep the content up to date as it changes or is retracted.

What is PushHub?

PushHub is a content syndication system built with:

• Pyramid w/ ZODB
• Redis
• Feedparser
• Solr

It uses the Pubsubhubbub standard from Google.

PHP - I can't believe I'm about to do this...

PushHub can easily be called from PHP based CMSs like Drupal and WordPress as well as from Plone. See slides 18-21 above for sample code.

Demo

You can easily create content across several Plone and Drupal websites.

• When creating content, simply publish and share to PushHub
• Then on another site you can search PushHub and select content to show up in a related articles widget or other content areas
• You can control if just the title shows up or an extract
• You can even copy an entire article from one site to the other and keep it in sync and with the canonical source marked correctly for SEO
• When you update the master copy, all other copies get a notification and update instantly

See the slides for examples or the webinar recording in the link below.

Learn More:

• PushHub website: http://pushhub.io
• PushHub overview & webinar recording: http://pushhub.io/overview/
• UCLA Case Study
• Web Syndication Comparison: PushHub vs. RSS
• 5 Great Reasons to Syndicate Content Across Your Websites
• Why Consider Integrating Your Websites?
• Enterprise search solutions

14 Jul 2014 8:10pm GMT

After another successful development sprint, we are pleased to announce the release of ploneintranet.workspace 1.0!

We covered some of the background to this development in our previous blog post, but this sprint was focussed on finishing, tidying, documenting and getting an initial release out!

ploneintranet.workspace

A core building block of the Plone Intranet solution, ploneintranet.workspace aims to provide a flexible team/community workspace solution, allowing teams of users to communicate and collaborate effectively within their own area of an intranet. Plone's extensive permissions are distilled into a set of distinct policies that control who can access a workspace, who can join a workspace, and what users can do once they are part of a workspace.

An Intro to Workspace Policies

Three realms of access are controlled via a single 'policies' tab on the workspace container:

External visibility

Who can see the workspace and its content?

• Secret
  Workspace and content are only visible to members
• Private
  Workspace is visible to non-members
  'Published' Workspace content only visible to members
  'Public' Workspace content visible to all
• Open
  Workspace is visible to non-members
  'Published' Workspace content visible to all

Join policy

Who can join / add users to a workspace?

Participation policy

What can members of the workspace do?

• Consumers
  Members can read all published content

• Producers
  Members can create new content, and submit for review
• Publishers
  Members can create, edit and publish their own content (but not the content of others)
• Moderators
  Members can create, edit and publish their own content and content created by others.

Policy Scenarios

Give it a try!

We would love you to try this package out and give us your feedback. For more information including how to download and install, see the documentation at readthedocs.org.

Further reading:

• Designing an open source social intranet (Cosent Blog)
• Development Sprint update (May 2014)
• ploneintranet.workspace documentation @ RTD

14 Jul 2014 1:43pm GMT

Firmenänderung

11 Jul 2014 3:52pm GMT

11 Jul 2014 3:46pm GMT

Firmenmitteilung

11 Jul 2014 11:46am GMT

A completely redefined user interface, with improvements on usability and accessibility, is just the more visible advantage that the new version of Plone will provide.

10 Jul 2014 3:51pm GMT

This is a recap of my presentation at the Plone Symposium Midwest 2014.

Managing Chaos: Merging 120 Sites into a single Plone Multisite Solution

Who Am I?

10 Jul 2014 3:40pm GMT

This is a loose series of blog posts about parts of Plone that I consider as broken from the prospective of a programmer. The blog entries are based on personal experiences with Plone over the last few months collected in new Plone 4.3 projects and some legacy projects but they also reflect experienced learned from other non-core Plone developers involved in these projects (developers on the customer side).

10 Jul 2014 6:56am GMT

This is a loose series of blog posts about parts of Plone that I consider as broken from the prospective of a programmer. The blog entries are based on personal experiences with Plone over the last few months collected in new Plone 4.3 projects and some legacy projects but they also reflect experienced learned from other non-core Plone developers involved in these projects (developers on the customer side).

10 Jul 2014 6:56am GMT

Summary of Plone News from around the world for the month of June.

09 Jul 2014 2:35pm GMT

This blog posts presents rolling time window counting and rate limiting in Redis. You can apply it to activate login CAPTCHA on your site only when it is needed. For the syntax highlighted Python source code please see the original blog post.

1. About Redis

Redis is a key-value store and persistent cache. Besides normal get/set functionality it offers more complex data structures like lists, hashes and sorted sets. If you are familiar with memcached think Redis as memcached with steroids.

Often Redis is used for rate limiting purposes. Usually the rate limit recipes are count how many times something happens on a certain second or a certain minute. When the clock ticks to the next minute, rate limit counter is reset back to the zero. This might be problematic if you are looking to limit rates where hits per integration time window is very low. If you are looking to limit to the five hits per minute, in one time window you get just one hit and six in another, even though the average over two minutes is 3.5.

This posts presents an Python example how to do a rolling time window based counting, so that rate counting does not reset itself back to the zero in any point, but counts hits over X seconds to the past. This is achieved using Redis sorted sets.

2. rollingwindow.py:

If you know any better way to do this with Redis - please let me know - I am no expert here. This is the first implementation I figured out.

"""

    Redis rolling time window counter and rate limit.

    Use Redis sorted sets to do a rolling time window counters and limiters.

    http://redis.io/commands/zadd

"""

import time


def check(redis, key, window=60, limit=50):
    """ Do a rolling time window counter hit.

    :param redis: Redis client

    :param key: Redis key name we use to keep counter

    :param window: Rolling time window in seconds

    :param limit: Allowed operations per time window

    :return: True is the maximum limit has been reached for the current time window
    """

    # Expire old keys (hits)
    expires = time.time() - window
    redis.zremrangebyscore(key, '-inf', expires)

    # Add a hit on the very moment
    now = time.time()
    redis.zadd(key, now, now)

    # If we currently have more keys than limit,
    # then limit the action
    if redis.zcard(key) > limit:
        return True

    return False


def get(redis, key):
    """ Get the current hits per rolling time window.

    :param redis: Redis client

    :param key: Redis key name we use to keep counter

    :return: int, how many hits we have within the current rolling time window
    """
    return redis.zcard(key)

3. Problematic CAPTCHAs

Everybody of us hates CAPTCHAs. They are two-edged swords. On one hand, you need to keep bots out from your site. On the other, CAPTCHAs are turn off for your site visitors and they drive away potential users.

Even though the most popular CAPTCHA-as-a-service, Google's reCAPTCHA, has made substantial progress to make CAPTCHAs for real visitors and hard for bots, CAPTCHAs still present a usability problem. Also in the case of reCAPTCHA, JavaScript and image assets are loaded from Google front end services and they tend to get blocked in China, disabling your site for Chinese visitors.

4. CAPTCHAs and different login situations

There are three cases where you want the user to complete CAPTCHA for login

• Somebody is bruteforcing a single username (targeted attack): you need to count logins per usename and not let the login proceed if this user is getting too many logins.
• Somebody is going through username/password combinations for a single IP: you count logins per IP.
• Somebody is going through username/password combinations and the attack comes from very large IP pool. Usually these are botnet-driven attacks and the attacker can easily have tens of thousands of IP addresses to burn.

The botnet-driven login attack is tricky to block. There might be only one login attempt from each IP. The only way to effectively stop the attack is to present pre-login CAPTCHA i.e. the user needs to solve the CAPTCHA even before the login can be attempted. However pre-login CAPTCHA is very annoying usability wise - it prevents you to use browser password manager for quick logins and sometimes gives you extra headache of two minutes before you get in to your favorite site.

Even services like CloudFlare do not help you here. Because there is only one request per single IP, they cannot know beforehand if the request is going to be legitimate or not (though they have some global heurestics and IP blacklists for sure). You can flip on the "challenge" on your site, so that every visitors must complete the CAPTCHA before they can access your site and this is usability let down again.

5. Mitigating botnet-driven login attack with on-situation CAPTCHA

You can have the best of the both worlds: no login CAPTCHA and still mitigate botnet-driven login atttacks. This can be done by

• Monitoring your site login rate
• In normal situation do not have pre-login CAPTCHA
• When there is clearly an abnormal login rate, which means there might be an attack going on, enable the pre-login CAPTCHA for certain time

Below is an pseudo-Python example how this can be achieved with using rollingwindow Python module from the above.

6. captchamode.py

from redis_cache import get_redis_connection

import rollingwindow


#: Redis sorted set key counting login attempts
REDIS_LOGIN_ATTEMPTS_COUNTER = "login_attempts"

#: Key telling that CAPTCHA become activated due to
#: high login attempts rate
REDIS_CAPTCHA_ACTIVATED = "captcha_activated"

#: Captcha mode expires in 120 minutes (attack cooldown)
CAPTCHA_TIMEOUT = 120 * 60

#: Are you presented CAPTCHA when logging in first time
#: Disabled in unit tests.
LOGIN_ATTEMPTS_CHALLENGE_THRESHOLD = 500  # per minute


def clear():
    """ Resets the challenge system state, per system or per IP. """
    redis = get_redis_connection("redis")
    redis.delete(REDIS_CAPTCHA_ACTIVATED)
    redis.delete(REDIS_LOGIN_ATTEMPTS_COUNTER)


def get_login_rate():
    """
    :return: System global login rate per minute for metrics
    """
    redis = get_redis_connection("redis")
    return rollingwindow.get(redis, REDIS_LOGIN_ATTEMPTS_COUNTER)


def check_captcha_needed(redis):
    """ Check if we need to enable login CAPTCHA globally.

    Increase login page load/submit counter.

    :return: True if our threshold for login page loads per minute is exceeded
    """

    # Count a hit towards login rate
    threshold_exceeded = rollingwindow.check(redis, REDIS_LOGIN_ATTEMPTS_COUNTER, limit=LOGIN_ATTEMPTS_CHALLENGE_THRESHOLD)

    # Are we in attack mode
    if not redis.get(REDIS_CAPTCHA_ACTIVATED):

        if not threshold_exceeded:
            # No login rate threshold exceeded,
            # and currently CAPTCHA not activated ->
            # allow login without CAPTCHA
            return False

        # Login attempt threshold exceeded,
        # we might be under attack,
        # activate CAPTCHA mode
        redis.setex(REDIS_CAPTCHA_ACTIVATED, "true", CAPTCHA_TIMEOUT)

    return True


def login(request):

    redis = get_redis_connection("redis")

    if check_captcha_needed(request):
        # ... We need to CAPTCHA before this login can proceed ..
    else:
        # ... Allow login to proceed without CAPTCHA ...

Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

09 Jul 2014 11:56am GMT