fidzu : Plone

This is a talk about migrating from Plone 4 to 6 with the newest toolset.

There are several challenges when doing Plone migrations:

• Highly customized source instances: custom workflow, add-ons, not all of them with versions that worked on Plone 6.
• Complex data structures. For example a Folder with a Link as default page, with pointed to some other content which meanwhile had been moved.
• Migrating Classic UI to Volto
• Also, you might be migrating from a completely different CMS to Plone.

How do we do migrations in Plone in general?

• In place migrations. Run migration steps on the source instance itself. Use the standard upgrade steps from Plone. Suitable for smaller sites with not so much complexity. Especially suitable if you do only a small Plone version update.
• Export - import migrations. You extract data from the source, transform it, and load the structure in the new site. You transform the data outside of the source instance. Suitable for all kinds of migrations. Very safe approach: only once you are sure everything is fine, do you switch over to the newly migrated site. Can be more time consuming.

Let's look at export/import, which has three parts:

• Extraction: you had collective.jsonify, transmogrifier, and now collective.exportimport and plone.exportimport.
• Transformation: transmogrifier, collective.exportimport, and new: collective.transmute.
• Load: Transmogrifier, collective.exportimport, plone.exportimport.

Transmogrifier is old, we won't talk about it now. collective.exportimport: written by Philip Bauer mostly. There is an @@export_all view, and then @@import_all to import it.

collective.transmute is a new tool. This is made to transform data from collective.exportimport to the plone.exportimport format. Potentially it can be used for other migrations as well. Highly customizable and extensible. Tested by pytest. It is standalone software with a nice CLI. No dependency on Plone packages.

Another tool: collective.html2blocks. This is a lightweight Python replacement for the JavaScript Blocks conversion tool. This is extensible and tested.

Lastly plone.exportimport. This is a stripped down version of collective.exportimport. This focuses on extract and load. No transforms. So this is best suited for importing to a Plone site with the same version.

collective.transmute is in alpha, probably a 1.0.0 release in the next weeks. Still missing quite some documentation. Test coverage needs some improvements. You can contribute with PRs, issues, docs.

We saw that cookieplone was coming up, and Docker, and as game changer uv making the installation of Python packages much faster.

With cookieplone you get a monorepo, with folders for backend, frontend, and devops. devops contains scripts to setup the server and deploy to it. Our sysadmins already had some other scripts. So we needed to integrate that.

First idea: let's fork it. Create our own copy of cookieplone. I explained this in my World Plone Day talk earlier this year. But cookieplone was changing a lot, so it was hard to keep our copy updated.

Maik Derstappen showed me copier, yet another templating language. Our idea: create a cookieplone project, and then use copier to modify it.

What about the deployment? We are on GitLab. We host our runners. We use the docker-in-docker service. We develop on a branch and create a merge request (pull request in GitHub terms). This activates a piple to check-test-and-build. When it is merged, bump the version, use release-it.

Then we create deploy keys and tokens. We give these access to private GitLab repositories. We need some changes to SSH key management in pipelines, according to our sysadmins.

For deployment on the server: we do not yet have automatic deployments. We did not want to go too fast. We are testing the current pipelines and process, see if they work properly. In the future we can think about automating deployment. We just ssh to the server, and perform some commands there with docker.

Future improvements:

• Start the docker containers and curl/wget the /ok endpoint.
• lock files for the backend, with pip/uv.

[Missed the first part.]

Vision: plone.restapi aims to provide a complete, stable, documented, extensible, language-agnostic API for the Plone CMS.

New services

• @site: global site settings. These are overall, public settings that are needed on all pages and that don't change per context.
• @login: choose between multiple login provider.
• @navroot: contextual data from the navigation root of the current context.
• @inherit: contextual data from any behavior. It looks for the closest parent that has this behavior defined, and gets this data.

Dynamic teaser blocks: you can choose to customize the teaser content. So the teaser links to the item you have selected, but if you want, you can change the title and other fields.

Roadmap:

• Don't break it.
• 10.0 release for Plone 6.2: remove setuptools namespace.
• Continue to support migration path from older versions: use an old plone.restapi version on an old Plone version to export it, and being able to import this to the latest versions.
• Recycle bin (work in progress): a lot of the work from Rohan is in Classic UI, but he is working on the restapi as well.

Wishlist, no one is working on this, but would be good to have:

• @permissions endpoint
• @catalog endpoint
• missing control panel
• folder type constraints
• Any time that you find yourself going to the Classic UI to do something, that is a sign something is missing.
• Some changes to relative paths to fix some use cases
• Machine readable specifications for OpenAPI, MCP
• New forms backend
• Bulk operations
• Streaming API
• External functional test suite, that you could also run against e.g. guillotina or Nick to see if it works there as well.
• Time travel: be able to see the state of the database from some time ago. The ZODB has some options here.

This is a talk I did not want to give.

I am team lead of the Plone Admin team, and work at kitconcept.

The current state: see the keynotes, lots happening on the frontend. Good.

The current state of our IT: very troubling and daunting.

This is not a 'blame game'. But focussing on resources and people this conference should be a first priority. We are a real volunteer organisation, nobody is pushing anybody around. That is a strength, but also a weakness. We also see that in the Admin team.

The Admin team is 4 senior Plonistas as allround admin, 2 release managers, 2 CI/CD experts. 3 former board members, everyone overburdened with work. We had all kinds of plans for this year, but we have mostly been putting out fires.

We are a volunteer organisation, and don't have a big company behind us that can throw money at the problems. Strength and weakness. In all society it is a problem that volunteers are decreasing.

Root causes:

• We failed to scale down in time in our IT landscape and usage.
• We have no clean role descriptions, team descriptions, we can't ask a minimum effort per week or month.
• The trend is more communication channels, platforms to join and promote yourself, apps to use.

Overview of what have have to keep running as admin team:

• Support main development process: github, CI/CD, Jenkins main and runners, dist.plone.org.
• Main communication, documentation: pone.org, docs.plone.org, training.plone.org, conf and country sites, Matomo.
• Community office automation: Google docds, workspacae, Quaive, Signal, Slack
• Broader: Discourse and Discord

The first two are really needed, the second we already have some problems with.

Some services are self hosted, but also a lot of SAAS services/platforms. In all, it is quite a bit.

The Admin team does not officially support all of these, but it does provide fallback support. It is too much for the current team.

There are plans for what we can improve in the short term. Thank you to a lot of people that I have already talked to about this. 3 areas: GitHub setup and config, Google Workspace, user management.

On GitHub we have a sponsored OSS plan. So we have extra features for free, but it not enough by far. User management: hard to get people out. You can't contact your members directly. E-mail has been removed, for privacy. Features get added on GitHub, and no complete changelog.

Challenge on GitHub: we have public repositories, but we also have our deployments in there. Only really secure would be private repositories, otherwise the danger is that credentials or secret could get stolen. Every developer with access becomes an attack vector. Auditing is available for only 6 months. A simple question like: who has been active for the last 2 years? No, can't do.

Some actionable items on GitHub:

• We will separate the contributor agreement check from the organisation membership. We create a hidden team for those who signed, and use that in the check.
• Cleanup users, use Contributors team, Developers
• Active members: check who has contributed the last years.
• There have been security incidents. Someone accidentally removed a few repositories. Someone's account got hacked, luckily discovered within a few hours, and some actions had already been taken.
• More fine grained teams to control repository access.
• Use of GitHub Discussions for some central communication of changes.
• Use project management better.
• The elephant in the room that we have practice on this year, and ongoing: the Collective organisation. This was free for all, very nice, but the development world is not a nice and safe place anymore. So we already needed to lock down some things there.
• Keep deployments and the secrets all out of GitHub, so no secrets can be stolen.

Google Workspace:

• We are dependent on this.
• No user management. Admins have had access because they were on the board, but they kept access after leaving the board. So remove most inactive users.
• Spam and moderation issues
• We could move to Google docs for all kinds of things. Use Google workspace drives for all things. But the Drive UI is a mess, so docs can be in your personal account without you realizing it.

User management:

• We need separate standalone user management, but implementation is not clear.
• We cannot contact our members one on one.

Oh yes, Plone websites:

• upgrade plone.org
• self preservation: I know what needs to be done, and can do it, but have no time, focusing on the previous points instead.

I make bad life choices. Years ago, for my first web programming language I picked PHP. I created my own CMS, of course. I skipped the cryptocurrencies long time ago, so today I am not rich.

I saw a job offer on the wall: "Help wanted". That was for Plone, which I did not know, but if someone wanted to pay me for learning it: cool.

I learned that architecture is important. I wrote a database application, but it become unmaintainable by all the spagghetti code and missing architecture. Same in lots of CMSes. But in Plone I saw architecture. Not everything was good by today's standards, but there was an idea behind it.

JavaScript become popular. Plone had KSS before jQuery even was created. But I was skeptic. That may be surprising when you know my role in Volto today.

Around the Barcelona Plone conference in 2017, Ramon approached me and was totally enthousiasic about AI, or I prefer to call it machine learning. Again, I was skeptic. I read three books about AI, but only to proof him wrong. The Plone community attracts smart and innovative people. I don't need to have FOMO, fear of missing out. We were talking about AI before it got hip. There are plans for an MCP to let an AI agent interact with Plone, for example creating news items.

Main competitors for Plone are Drupal, Typo3, other open source CMSes. Usually the UI is outdated, using old technology. Plone is the only open source CMS with a modern JavaScript stack. You have Wordpress, but that is not truly open source, look at the fight between two big Wordpress companies. Others have gotten modern, but their community is split.

Plone 6.2 will be released end of 2025, beginning 2026. No revolution, small improvements. In Volto: restore unsaved content, improve image upload widget, relative path support, blocks control panel (how many blocks of which type are in my site?) And one last important thing in the Python backend: move to native namespaces. That is a needed improvement because of changes in the Python packaging ecosystem.

Plone 6.3 is for end of 2026. Again small improvements. Improved and renamed Listing/Search block variants, improving link widget, keyword manager, recycle bin.

Plone 6.x: peace of mind. We offer 5 years of security support for all Plone 6.x minor versions. We might extend that, depending on when Plone 7 is released.

Plone 7 will be innovation. Expected in 2027.

• It will be a lot faster. Main problem in performance measurements is the bundle size. How? Static site generators? Wait, why did I not hear about those the past years? Reason: javascript frameworks have gotten so fast that static site generators are basically not needed anymore. We will use those techniques.
• We will use React Router 7.
• Plone will be more flexible, component based, so you can build your own UI.
• We will remove Semantic UI, and switch to react-aria. This is for accessibility, especially in the editor interface.

Can we do this in small steps? Unfortunately not. We modernize the frontend. Volto was created in 2017 ,following best practices then. Now it needs an update. The codename for that is: Seven. We did 5 code sprints this year, working on this, and others. See the pictures with the people who were there. Do you see someone on all five photos? Victor Fernandez d'Alba and Piero Nicolli. They are leading this effort.

Improved editing experience. With Block model 3 we improve the wysiwyg experience: there is almost no change when you view a page and then start editing it. This model also helps a lot when you want to have an own design system.

Plone: between stability and innovation.

Bonnie Tyler Sprint

On 12 August 2026 there is a total solar eclipse that can be seen from Valencia, Spain. So we organise a sprint there.

This conference

We had 291 participants, 234 in person and 57 online. 13 Brazilian states (that is all of them), 14 countries.

24.5 percent women, was 13% in 2013, so that has gone up, but we are not there yet. Thank you to PyLadies and Django Girls for making this happen.

We had more than 80 presenters, about 30 lightning talks, lots of talk in the hall ways.

Thanks also to the team!

Ramiro Luz: Yoga time

Yoga exercise.

Rikupekka: University case student portal

We have a student portal at the university. But mostly:

Welcome to Jyväskylä university in Finald for Plone conference 2025, October 13-19!

Jakob: Beethovensprint

26-30 mei 2025 in Bonn, Duitsland.

Afterwards, on May 30 and June 1 there will be FedCon in Bonn, a SciFi convention.

Piero/Victor: BYOUI

Add-ons first development with @plone/registry. See https://plone-registry.readthedocs.io/

It allows for development that is framework agnostic, so it is not only for Plone. It is around configuration that can be extended and injected, which is tricky in most javascript frameworks.

Imagine it.

Ana Dulce: 3D printing

For a difficult model I had trust the process, it took a week, but it worked.

Renan & Iza: Python Brasil

We organised the Python Brasil conference from 16 to 23 October this year in Rio de Janeiro.

Next year 21-27 October in São Paulo.

Erico: Python Cerrado

31 July to 2 August 2025 is the next Python Cerrado conference.

Link to talk information on Plone conference website.

I work for the Clean Clothes Campaign: https://cleanclothes.org/

After three large disasters in factories in 2012 and 2013 with over 1000 deaths, it took three years to get an agreement with clothes manufacturers to get 30 million dollar compensation. It does not bring lives back, but it helps the survivors.

See Open Supply Hub for open data that we collected, for checking which brands are produced in which factories.

Documenting history matters. Stories must be told.

The global closing industry is worth around 1.8 trillion dollars, in a country that would put them on the 12th place in the world. 75 million workers.

Our strongest weapon: backlinks. We have links from OECD, UN, wikipedia, school curriculum, books. Especially those last two don't change ever, so you should never change urls.

Plone: enable the sitemap, please, why not by default? Create a good robots.txt. I weekly check Google Search console, looking for broken links. Tag early, tag often, great tool, even if you have an AI do it.

Our website: started 1998 written in Notepad, 2004 Dreamweaver, 2006 Bluefish, 2010 Joomla, 2013 Plone 4, 2020 Castle CMS (opinionated distribution of Plone, but does not really exist anymore) 2024 Plone 6 with Volto Light Theme (work in progress). Thank you kitconcept for all the help, especially Jonas.

Migrations are painful. Along the years we used wget to csv to SQL to csv, Python script, "Franken-mogrifier", collective.exportimport.

Lessons learned: stable urls are awesome, migrations are painful. Please don't try to salvage CSS from your old site, just start fresh in your new system. Do not try to migrate composite pages or listings.

What if your website does not provide an export? Use wget, still works and is better than httrack. sed/awk/regex are your friend. archivebox (WARC).

Document your steps for your own sanity.

To manage json, jq or jello can be used. sq is a Swiss knife for json/sql/csv. emuto is a hybrid between jq and GraphQL.

Normalize import/export. We have `plone.exportimport` in core now.

In the future I would like a plone exporter script that accepts a regex and exports only matching pages. Switch backends: ZODB, relstorage, nick, quantum-db. Sitewide search/replace/sed. Sneakernet is useful in difficult countries where you cannot send data over the internet: so export to a usb stick.

A backup is only a backup if it regularly gets restored so you know that it works.

• Keeping content and URL stability is a superpower.
• Assuming that export/import/backup/restore/migration are rare occurrences, is wrong.
• Quick export/import is very useful.

Do small migrations, treat it as maintenance. Don't be too far behind. Large migrations one every five years will be costly. Do a small migration every year. Do your part. Clients should also do their part, by budgeting this yearly. That is how budgeting works. Use every iteration to review custom code.

Make your sites live long and prosper.

Link to talk information on Plone conference website.

Sorry, I ran out of time trying to set up https://coolify.io

So let's talk about another problem. Running applications (stacks) in containers is the future. Well: abstraction and isolation is the future, and containers is the current phase.

I am on the Plone A/I team, with Paul, Kim, Erico. All senior sysadmins, so we kept things running. In 2022 we worked on containerisation. Kubernetes was the kool kid then, but Docker Swarm was easier. Checkout Erico's training with new cookieplone templates.

Doing devops well is hard. You have a high workload, but still need to keep learning new stuff to keep up with what is changing.

I want to plug Coolify, which is a full open source product. "Self-hosting with super powers." The main developer, Andras Bacsal, believes in open source and 'hates' pay by usage cloud providers with a vengeance.

Coolify is still docker swarm. We also want Kubernetes support. But we still need sysadmins. Someone will still need to install coolify, and keep it updated.

I would like to run an online DevOps course somewhere January-March 2025. 4-6 meetings of 2 hours, maybe Friday afternoon. Talk through devops and sysadmin concepts, show docker swarm, try coolify, etc.

Link to talk information on Plone conference website.

I am studying computer science. This summer I have participated in the Google Summer of Code for Plone. Thank you to mentors Paul, Roeland and Mike!

I have built several full-stack web apps with React, but am new to open source.

What is RSS? It is Really Simple Syndication. It is used by websites to publish updated information, such as blog or news. It lets users decide what they read every day, instead of algorithms.

In Plone the RSS feed should use a catalog query to filter contents, for example a feed of science articles or of events.

For an RSS feed content type we could use a collection behavior or a blocks behavior with a listing block. The listing block is the way to go for Volto. So I did that. Plus extra fields like maximum title length and maximum description length.

I created a view component for the content type. Important is the Express middleware with which we got information from the backend.

We decided on `rss` as Node package for the feed. We looked at another newer one called `feed`, but it did not work for us or was missing features.

There were challenges for me as person new to Plone. Do I really understand the code? Often the answer is: no. For example ZCML? i18n? XML files in the profiles? There is a really nested structure. Create a new project, then you have backend / src / rss / src / rss, at least something very deeply nested. It was always confusing to me as I did not really know what should end up in git or in a release. The worst is that I don't know what are the most important files.

And what if my code fails, it was hard to understand errors on the front end. There are just too many things that may be the source of the error.

Another challenge: what if I want to create something new? The training lets you learn by example. But what does each line of the example code actually do? Are there other options?

Missing documentation: how do prevent the user from removing the title or rss block? We found out about initialBlocks per content type. And prevent adding new blocks? There is a feature disableNewBlocks, that could be set on an individual block, but what about doing this for all instances of this content type?

An overview of the code base would be good. How different components are connected, how different files work together. A guide about how to approach the source code, what files I should look at first.

For developers new to Plone: the source code is your friend, the Plone community is helpful, but be specific in the questions you ask, and tell what you have already tried.

Source code:

• https://github.com/collective/volto-rss-provider
• https://github.com/collective/rss-provider
• https://github.com/collective/rss-receiver
• https://github.com/collective/volto-rss-receiver

Link to talk information on Plone conference website.

Plone loves tests. The first was committed in October 2002 by Sidnei da Silva. We have used lots of different testing framework. I like using pytest. The unittest framework was inspired by Java testing and you see it. pytest is simple and pythonic. It does dependency injection, which can look magical, okay. There is a large ecosystem of plugins. It has support for unittest. There is a Brazilian maintainer.

I have created pytest-plone, built upon zope.pytest. It provides fixtures and helpers. it is the default in cookieplone templates. Take a look at plone.distribution and plone.exportimport for examples.

Golden rules

If you need a Plone site in your test, you need to wrap your test in a class. That helps for not tearing down everything after every test. Start with a method that gets portal and probably get_fti, and use @pytest.fixture(autouse=True).

Use @parametrize to run the same test function with multiple inputs and output.

Use fixtures.

Trust and love pytest plugins. There are plugins for running the tests in random order, or in parallel. Code coverage is a plugin. You can start a docker container.

Use helpers in pytest-plone to check that your latest upgrade step matches the metadata profile version. Check that your add-on is installed.

plone-pytest 1.0 is coming. Small problem: the tests are breaking on Python 3.9.

Write more tests with less code. I will sprint on pytest-plone.

Link to talk information on Plone conference website.

Kubernetes (k8s) is an open source system for managing containerised applications It provides mechanisms for deployment and scaling. It is part of the Cloud Native Computing Foundation.

Kubernetes' benefits are scalability, fault tolerant, self-healing, efficient resource utilisation, declarative configuration, rich ecosystem.

Kubernetes basics:

• pods: the smallest deployable unit of computing, consisting of one or more containers. Containers in one pod can talk with each other like they are all on localhost.
• deployments: manage the replication and updates of pods
• RepicaSets: manage a fixed number of identical pods.
• StatefulSets: manage stateful apps, lik databases
• Jobs: one-off tasks
• CronJob: regular tasks
• ConfigMap: store configuration data
• Secrets: store sensitive information
• PersistentVolumeClaims: request storage
• PersistentVolumes: actual storage
• Pod Disruption Budgets (pdb): limit the number of pods that can be unavailable at any given time, for example during upgrades, self-healing during disaster recovery.
• Autoscaler: automatically scale the number of nodes in a cluster based on workload. Resource optimisation.
• Ingress: routing external traffic to services, SSL termination, load balancing, caching, rate limiting.
• Operator: software extensions to Kubernetes that make use of custom resources to manage applications and their components. Think of this as a human operator who would go in and do some stuff.

Plone components as Kubernetes resources:

• Plone backend: StatefulSet
• Plone frontend: Depoloyment
• PostgreSQL: StatefulSet
• Varnish: Deployment
• nginx
• cronjobs

Kubernetes for Devops. It has a steep learning curve. For your laptop you have microk8s and k3d so you have the Kubernetes api, although without the options that you would have with multiple machines. Basics: `apt install docker.io` and `snap install kubectl --classic` and `snap install heml --classic`. And read https://k3d.io/

At the Alpine City Sprint this year we worked on helm charts for Plone: https://github.com/plone/helm-charts/

Tips and tricks:

• You can use ArgoCD to automatically do a deploy when something changes in a git repo. You need to configure this correctly of course.
• Plone can do better on integration with proper observability tools like Grafana and Prometheus. For example, how can such tools get the logs when they are in multiple places.
• Use ZODB with PostgreSQL and relstorage. I think it is the most stable implementation today.
• Varnish Operator from IBM looks promising when you want to run multiple instances of varnish.
• If you run the `test.sh` from Plone help charts this may use a lot of bandwidth downloading docker images. k3d supports import of docker images so they are in a cache.

Link to talk information on Plone conference website.

We needed a video platform for our university. Difficult, but we had a dev team of 8 with some architecture experience, so let's try. We did a successful POC in 2023.

We use: Plone 6, VideoJS player, Wowza streaming engine, Keycloak, WhisperX, Camunda 7, Nomad. Most important: open source and license free.

Security. We have it in a dedicated and separate server environment. All data is stored on own servers of the university. All data and files are encrypted. Plone is very secure. All transactions are logged. There is a process for data protection, and it has been reviewed and documented.

There was no definite deadline, which is both good and bad. We could focus on quality, but also needed to focus on other projects.

Now a nice working demo.

[Sorry, I was distracted by client work.]