fidzu : plone

This is a talk about migrating from Plone 4 to 6 with the newest toolset.

There are several challenges when doing Plone migrations:

• Highly customized source instances: custom workflow, add-ons, not all of them with versions that worked on Plone 6.
• Complex data structures. For example a Folder with a Link as default page, with pointed to some other content which meanwhile had been moved.
• Migrating Classic UI to Volto
• Also, you might be migrating from a completely different CMS to Plone.

How do we do migrations in Plone in general?

• In place migrations. Run migration steps on the source instance itself. Use the standard upgrade steps from Plone. Suitable for smaller sites with not so much complexity. Especially suitable if you do only a small Plone version update.
• Export - import migrations. You extract data from the source, transform it, and load the structure in the new site. You transform the data outside of the source instance. Suitable for all kinds of migrations. Very safe approach: only once you are sure everything is fine, do you switch over to the newly migrated site. Can be more time consuming.

Let's look at export/import, which has three parts:

• Extraction: you had collective.jsonify, transmogrifier, and now collective.exportimport and plone.exportimport.
• Transformation: transmogrifier, collective.exportimport, and new: collective.transmute.
• Load: Transmogrifier, collective.exportimport, plone.exportimport.

Transmogrifier is old, we won't talk about it now. collective.exportimport: written by Philip Bauer mostly. There is an @@export_all view, and then @@import_all to import it.

collective.transmute is a new tool. This is made to transform data from collective.exportimport to the plone.exportimport format. Potentially it can be used for other migrations as well. Highly customizable and extensible. Tested by pytest. It is standalone software with a nice CLI. No dependency on Plone packages.

Another tool: collective.html2blocks. This is a lightweight Python replacement for the JavaScript Blocks conversion tool. This is extensible and tested.

Lastly plone.exportimport. This is a stripped down version of collective.exportimport. This focuses on extract and load. No transforms. So this is best suited for importing to a Plone site with the same version.

collective.transmute is in alpha, probably a 1.0.0 release in the next weeks. Still missing quite some documentation. Test coverage needs some improvements. You can contribute with PRs, issues, docs.

We saw that cookieplone was coming up, and Docker, and as game changer uv making the installation of Python packages much faster.

With cookieplone you get a monorepo, with folders for backend, frontend, and devops. devops contains scripts to setup the server and deploy to it. Our sysadmins already had some other scripts. So we needed to integrate that.

First idea: let's fork it. Create our own copy of cookieplone. I explained this in my World Plone Day talk earlier this year. But cookieplone was changing a lot, so it was hard to keep our copy updated.

Maik Derstappen showed me copier, yet another templating language. Our idea: create a cookieplone project, and then use copier to modify it.

What about the deployment? We are on GitLab. We host our runners. We use the docker-in-docker service. We develop on a branch and create a merge request (pull request in GitHub terms). This activates a piple to check-test-and-build. When it is merged, bump the version, use release-it.

Then we create deploy keys and tokens. We give these access to private GitLab repositories. We need some changes to SSH key management in pipelines, according to our sysadmins.

For deployment on the server: we do not yet have automatic deployments. We did not want to go too fast. We are testing the current pipelines and process, see if they work properly. In the future we can think about automating deployment. We just ssh to the server, and perform some commands there with docker.

Future improvements:

• Start the docker containers and curl/wget the /ok endpoint.
• lock files for the backend, with pip/uv.

[Missed the first part.]

Vision: plone.restapi aims to provide a complete, stable, documented, extensible, language-agnostic API for the Plone CMS.

New services

• @site: global site settings. These are overall, public settings that are needed on all pages and that don't change per context.
• @login: choose between multiple login provider.
• @navroot: contextual data from the navigation root of the current context.
• @inherit: contextual data from any behavior. It looks for the closest parent that has this behavior defined, and gets this data.

Dynamic teaser blocks: you can choose to customize the teaser content. So the teaser links to the item you have selected, but if you want, you can change the title and other fields.

Roadmap:

• Don't break it.
• 10.0 release for Plone 6.2: remove setuptools namespace.
• Continue to support migration path from older versions: use an old plone.restapi version on an old Plone version to export it, and being able to import this to the latest versions.
• Recycle bin (work in progress): a lot of the work from Rohan is in Classic UI, but he is working on the restapi as well.

Wishlist, no one is working on this, but would be good to have:

• @permissions endpoint
• @catalog endpoint
• missing control panel
• folder type constraints
• Any time that you find yourself going to the Classic UI to do something, that is a sign something is missing.
• Some changes to relative paths to fix some use cases
• Machine readable specifications for OpenAPI, MCP
• New forms backend
• Bulk operations
• Streaming API
• External functional test suite, that you could also run against e.g. guillotina or Nick to see if it works there as well.
• Time travel: be able to see the state of the database from some time ago. The ZODB has some options here.