(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

A microcontroller is a combination of CPU, memory, storage and IO on a single cheap chip. He showed: ESP32, STM32, raspberry pi 2350. The lines can be a bit blurred. Raspberry pi variants that are almost a normal computer, for instance.

Computer: multi-user, multi-tasking, always ready to do something. Microcontroller: single task. Can sleep when it doesn't have to do the task. So: low power consumption, perhaps even suitable for battery operation.

Years and years ago, microcontrollers meant resources measured with k. Now it is M or even G. From 4 kB to 4 GB...

For microcontrollers you have MicroPython. Very simple to learn and it feels much the same as regular python. Libraries like "requests" and "numpy" are available.

He demoed how he set up monitoring of his power consumption at home with microcontrollers.

Photo explanation: random picture from Darmstadt (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Full title: you don't think about your streamlit app optimization until you try to deploy it to AWS

Streamlit is a quick way to show your models/data to stakeholders.

She once made a streamlit app to exercise writing Japanese characters. It used some character recognition model. But... the way streamlit works, it normally downloads the model every single time. For some local testing, it is probably OK, but if you put it in production... In production, network traffic might cost money.

Solution? You can cache it with streamlit, but you can also download it when building the docker image and store it inside the image and load it from there.

On to authentication. You can handle everything yourself as data scientist: login widget, auth logic, user privs, etc. You can also use an external provider like Amazon Cognito. Then you only have to hook up cognito in your code, but the OPS engineer has to set up cognito for you.

On to security. For all of them you'll need the OPS engineer, probably.

• Set up https with Route 53 and TLS certificates.
• Configure CloudFront to protect against DDoS attacks and improve performance.
• Use AWS web application firewall to block malicious traffic.

On to credential storage. You can use AWS secret manager instead of putting API_KEY = "1234abcd" right in your code. Using the secret manager is much more secure and that will make your OPS engineer happy.

Photo explanation: random picture from Darmstadt (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Full title: code & community: the synergy of community building and task automation

Cosima organizes PyLadies and R-Ladies events.

Visibility is important. You have to be visible to be noticed. Time is also important: you have to put in work to create good content. But you also want to have time for other things. So she thought about creating an "automated megaphone" to help her and other PyLadies members be more visible.

She created the pyladies bot on bluesky and mastodon. It regularly shares portraits of "amazing women in tech". And it reposts messages when tagged or mentioned. It also monitors blogs and posts about them. See https://github.com/cosimameyer/awesome-pyladies-blogs

The bot runs as a github action "cron job".

She started using google's "gemini" LLM to create short summaries of the blog posts to make it more likely for people to click on it. She picked a cheap, small model as that was good enough. In addition she does an extra automated check on harassment, dangerous content, etc.

Lessons learned:

• You can use powerfull LLMs to enhance your applications.
• Integrating modern LLMs is straightforward and easy.
• No need to go all the way to the cloud, you can just use the models via an API.
• It is cost-effective for smasll projects and/or developers.

Photo explanation: random picture from Darmstadt (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Streamlit is used a lot for dashboards. Fasthtml is a new contender. He demoed both of them at the same time (well done, btw, one of the better comparisons I've seen!), I wrote down some notes:

• Streamlit runs through your python file and puts stuff on your screen (like a "h1") immediately. So you see the page being build up, which tells the user something is happening. Fasthtml only sends over the page once it is fully ready.
• Fasthtml is based on htmx! So you can put placeholders on the screen that are filled in later with htmx. This helps putting something on the screen right away.
• There's a fh_plotly library to create plotly graphs.
• In fasthtml, it is fast to react to clicks and add something to the page. Only the new element is calculated, the rest stays the same. In streamlit, the whole page is calculated again, which is expensive. You can speed it up a bit with caching.
• Adding login functionality is easier in fasthtml, especially as the layout of the page is more explicit. You can have "components" in the layout and you can swap them. So a login form component can be shown if you're not logged in and swapped out for the actual content once you're logged in. In streamlit, such interaction is harder as you have to manage some state variables and re-run the full script.
• A slider below a graph to influence the size of the markers on the graph is easy in fasthtml. But in streamlit, you need advanced tricks as the graph is rendered first, even though it has to be influenced by a slider value, even though the slider isn't defined yet.
• Multiple pages are a pain in streamlit. In fasthtml it is just another url.

Streamlit gets you started fast. Often fewer lines of code. Many third-party integrations. Whole-page refresh only. Confusing control flow. And... if you learn streamlit, you only learn streamlit.

Fasthtml needs some boilerplate. More lines of code. Fewer integrations, but that's because the project is pretty new. If you learn fasthtml, you learn regular web development. He thinks he can build pretty big projects with it.

See also the streamlit vs reflex talk from yesterday.

The code for his presentation: https://github.com/tilman151/streamlit-vs-fasthtml

Photo explanation: random picture from Darmstadt (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Can AI be build in the open? Especially LLMs. Key components are data, compute, pretraining, posttraining, scaling. Can these components be build in the open? As open source?

Many models are closed, like claude, gemini, OpenAI o1. There are models with open model weights (but a black box otherwise): LLaMA, deepseek, Mistralai. And you have fully open models: granite, bloom, olmo, starcoder2.

Why would we want to do it? Doing it in the open?

• Transparency on pretraining. What data was used? How was my data used. How was the data filtered? This addresses biases, attribution and trust.
• Transparency on alignment. Models are aligned for safety/values. Which values are in the model? Are values global? In closed models, there are only a few people that define how the model behaves, but it has lots of influence, potentially world-wide.

Deepseek-R1 is not open source, but at least the weights are open. This helped shift the discussion a lot, as previously all the big ones were closed api-only models. At hugging face they try to make a fully open version, open-R1.

Open is closing the gap. When GPT-4 came out, it took a year for open models to catch up. At the moment it is more like just two months.

• In Europe, there are multiple public data centers with 10.000+ GPUs.
• Collaborative training: BOOM.
• Building the know-how together? For that you need large, high quality datasets for pre/posttraining. Good training recipes. Learning to scale to 1000s of GPUs.
• At hugging face, they try to build open datasets, also multilingual. See a "fineweb" blog post. And also small models. And some tools to build LLMs.

Some trends:

• Scaling will continue. Bigger is simply better. But.... it is exponential: it gets harder and harder. More compute, more power consumption.
• A new frontier: scaling the test time compute. This could really help improve accuracy.
• Reasoning, as done by deepseek, is interesting.
• Challenge: agents. Agency requires multi-step reasoning, which is more harder.
• AI in science.

What can you do?

• Build high-quality datasets.
• Fine-tune task specific models.
• Work on open source tooling.
• Join an open collaboration.

It is still early days for AI, there's a lot of stuff you can do. Don't think that everything is already figured out and build.

Photo explanation: random picture from Darmstadt (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Avada kedavra for life - Joakim

As a researcher, you start writing simple jupyter scripts. Then docker and suddenly you're hip deep in kubernetes and helm charts. It is a one-way trip.

How can you simplify? Think about "idempotent" and "stateless systems". You program your code to do a thing and not do it again. So you can call it multiple times and nothing can go wrong. Much safer. Even if colleagues have to use it later on.

If your code can't survive a rerun, it does not deserve to be in production.

Idempotency is the killing curse for complexity!

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

What is orchestration? Coordinated execution of multiple commputer systems, applications or services. It is more than automation. Some things you can think of:

• Containers/dockers can be managed.
• Coordinating multiple workflows/tasks.
• Syncronizing/managing two or more apps.
• Coordinating microservices, data services, networks, etc.

You can run code on-prem: a physical server in your cellar or a data center. You can also rent servers from a cloud provider. Another level up is serverless: you pay the specific compute resources you have used. AWS lambda is an example of a serverless function, this popularized the serverless paradigm.

Why would you combine them?

• Resilience: no orchestration tool to keep running.
• Cost efficiency: you only pay for what you use.
• Scalability: automatically handled.

Some options: AWS step functions, azure logic apps, azure durable functions, google's gcp workflows. A drawback for all of them is that they take a no-code/low-code approach, allowing you to click/drag/drop your workflows in the browser. It is stored in json, so as a proper software developer you are limited to uploading the json with terraform or so.

There are also open source solutions. Argo workflows, for instance. Drawback of those solutions: you again have to worry about infrastructure. If your company already has something set up, it might be a handy idea.

His conclusions:

• Operations: you can orchestrate workflows with minimal OPS overhead.
• Cost: you can build solutions at low cost.
• Perfect for event-driven workflows.
• The future? Probably not (vendor lock-in, mostly). But it is a great extension of your developer-toolbox.

His slides are available on github

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

He asked everyone who has used streamlit to stand up (75% stood). Then everyone who thought their dashboards were getting much too complex could sit down again. Only a few were left standing.

In 2015 he started out with a streamlit precursor based on bokeh. Around 2020 streamlit came around and it quickly gained a lot of popularity. He still uses streamlit and likes it. It is simple and easy.

But... people ask for more functionality. Or they add multiple pages. Then it slowly starts to break down. It doesn't fit the streamlit paradigm anymore. You can use @st.cache_data to speed it up a bit if you do an expensive calculation. @st.fragment limits the execution scope: changes within a fragment only trigger the fragment to re-run. After a while, the cache_data and fragment decorators are only band-aid on a bigger problem. It breaks down.

He recently discovered https://reflex.dev/ . An open source framework to quickly build and deploy web apps. It is a pure python framework, so you don't have to write typescript. But the big difference with streamlit is that the code is explicitly divided in frontend and backend.

• The UI is a result of a function.
• Changes are handled via events.
• State variables are accessible by the frontend. And they are session-scoped (so every user gets its own state).

You have a State class/object with @rx.event-decorated methods that implement the events.

• Input data is transformed to output data.
• Input data can be changed by a widget.
• Output data can end up in a widget.
• Widget changes can trigger transformations.

You start with python code. A compiler/transpiler turns it into a next.js frontend and a fastapi backend. Events are passed back and forth via an api between the frontend and backend. The backend gives state updates to the frontend via a websocket.

Because it is compiled once before starting the dashboard, it is less dynamic/on-the-fly than streamlit: you have to do some rx.for_each(....) to loop over values.

• If you know django+react or fastapi+react: use that if you know how to do it.
• If you want simple dashboards: streamlit, dash, gradio.
• The newcomers that aim at the middle ground because they are more explicit: reflex, rio, fasthtml. There's a fasthtml-vs-streamlit talk tomorrow!

Some closing comments on why you might not want to use reflex:

• Runtime errors in javascript are hard to debug.
• You need to have a mental model of frontend vs backend.
• The framework is moving fast (which is also good).
• Performance and wide adoption: that's still to be seen, it is still early days.

Something really useful that's not in the documentation: AppHarness, which allows you to test the backend.

He tried to create a wrapper for streamlit in reflex, mostly to make it easier to move your existing dashboards slowly over to reflex. It is called relit, but it doesn't work completely yet (and he hasn't released it publicly yet, just ask him). And it was pretty hard to get to work :-) He thinks it might be used for writing tests for streamlit dashboards.

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Full title: safeguard your precious API endpoints built on fastapi using OAuth 2.0.

She introduced herself by showing an openid oauth2 access token payload :-)

Several big companies wanted a way to have people log in more securely into their services. Originally, you'd use a username/password everywhere. They came up with oauth: a way to securely logging in on a website using an identity from an identity provider ("logging into a different website with your google account").

• Oauth2 is a generic mechanism for authorization.
• OpenID builds upon oauth2 and provides authentication.

Note: oauth 2.1 is under development, they will incorporate pkce. pkce is used by openid, so they'll mandate 2.1 once it is ready. It is handy for authentication from the frontend (on the frontend, you cannot store private secrets, so a priv/pub mechanism isn't usable).

Fastapi has a HTTPBearer scheme, which extracts a "bearer" token from the Bearer header. You can use this for oauth2.

(She showed some example code that I of course couldn't type over :-) Plus a demo.)

Look at RFC 9700 "best current practice for OAuth 2.0 security".

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Barak is a founding engineer at lakeFS.

Local storage is simple and easy. But cloud storage is real handy: scalability, security, etc. But the flexibility is a problem: every cloud storage service introduced its own way of working with it. Slightly different APIs.

fsspec is a python library providing a unified interface for interacting with various storage systems, local and remote. The goal is to make remote systems work as local ones.

Your python code talks to the "fsspec unified interface", which accepts a file system identifier/type (like s3), which activates that filesystem functionality. And then a file path within that type of filesystem.

Why fsspec?

• It simplifies your code.
• Consistency.
• Enhanced capabilities.
• Ecosystem integration.
• Extensible and open source.

fsspec implements the standard python .read(), .write(), .glob() etc. And also .seek(...), which you can use to do range requests, something you'd have to do yourself with s3/boto otherwise.

Pandas can read files from s3 and so, but it needs libraries for it. You can use fsspec and pass the file pointer to pandas.

fsspec has some additional capabilities. For instance caching. By prepending simplecache:: in front of the regular connection string, fsspec will cache the file if you read it multiple times.

What if you want to use a file system that isn't supported (yet)? You can implement your own! Inherit from an AbstractFileSystem and implement the necessary methods. He showed a funny toy example.

He works for lakeFS and showed lakefs-spec, a lakeFS plugin for fsspec: it wasn't build by them, but contributed by the community! Nice.

LakeFS works pretty much like git does: cheap branches, versioning, files pointing at full files, easy diffs, etc. The actual files are stored in a cloud storage, lakeFS manages the pointers to the files.

LakeFS wants to be atomic. But the cloud providers don't provide multi-file guarantees. Files will show up, but there might be a second or so delay. So they solve it by adding multiple files to a temporary branch and to only merge it once all files are really there. You can configure if you want to retain the temporary branch in case of an error (for debugging purposes) or if you want to have it automatically removed. He showed a convincing demo.

• Cloud storage: necessary, but challenging.
• fsspec simplifies it with a unified python interface.
• fsspec is extensible and open source.
• fsspec integrates nicely and enables caching, transactions and more.

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Full title: cache me if you can: boosted application performance with redis and client-side caching.

Redis can be used:

• As an in-memory database.
• As a cache.
• For data streaming.
• As a message broker.
• Even some vector database functionality.

Redis develops client libaries, for instance redis-py for python. Though you probably use redis through some web framework integration or so.

Why cache your data? Well, performance, scalability, speed. Instead of scaling your database, for instance, you can also put some caching in front of it (and scale the cache instead).

Caching patterns built into redis:

• "Look-aside": app reads from cache and if there's a miss, it looks in the actual data source instead.
• "Change data capture": the app reads from the cache and writes to the data source. Upon a change, the data source writes to the cache.

Redis has the regular cache features like expiration (time to live, expiration keys, etc), eviction (explixitly removing known stale items), LRU/LFU (least recently used, least frequently used). Redis behaves like a key/value store.

Why client side caching? Again performance and scalability. Some items/keys in redis are accessed much more often than others: "hot keys". Caching those locally on the client can help improvement.

Redis has a feature called client tracking. The client's cache is connected to the "main" one: the main one can invalidate keys on the client side.

Now to the redis-py client library. Some responsibilities are endpoint discovery, topology responsibility, authentication, etc. And, since recently, handling a local cache, connected to the main cache.

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

a11y = AccessibilitY.

Documention is an important part of every project. But what does it mean to write documentation for everyone?

You can make a "maslov pyramid" for documentation. Accurate content and install instructions are the basic necessity. For many writers, accessibility is somewhere at the top: a nice cherry when you get around to it.

But for some people it is a basic necessity. Accessibility means removing barriers. Making sure people can use what you build.

And: with accessibility you often think about blind people or someone without an arm. But if you solve a problem for the arm-less person, you also solve it for someone who broke their arm. Or someone holding a baby.

Common accessibility problems in docs:

• Low contrast text.
• Poor heading structure.
• Unlabeled buttons/links.
• No visible focus indicators.

Every one of those problems adds some friction for everyone. And... docs are often read when there's pressure to fix something, so any fiction is bad.

Now, how do you determine if your docs are accessible? An audit can help. It can be manual or automated or a mix. There are plenty of free tools: microsoft accessibility insights for web is one of the many tools (and the one she will use). The gold standard of testing, though, is to do real user testing. The best insights come from disabled users.

As a test she looked at the pydata sphinx theme. When they started improving it, they converted the output of microsoft's accessibility test into issues. For such issues, be specific. Include the page, element and what's failing. Reference an accessibility standard if possible. A screenshot or a short screencast.

Common problem: brand colors. Your project or company has a specific color already and if you use that color directly, it often isn't accessible. There just is not enough contrast. The solution can be to take the color you want and to create lighter and darker versions of it.

Hover states problems are also common. Often there's a slight change in color to signify that you've clicked something, but that's often not clear for colorblind people. Use borders, underlines, etc.

They've got a "figma" explanation if you use figma.

She then did a quick investigation of the pycon.de website with microsoft's accessibility tool. Most of the clearest problems were due to color contrast. White text on light gray, white text on light green, white text on yellow, etc. White on gray had an contrast ratio of 1.6:1, where 3:1 is recommended.

What can you do yourself?

• Run a quick audit on your docs.
• Review focus styles and heading structure.
• Share or reuse the pydata design system.
• Encourage accessibility discussions in your project.
• Open an bug in your project for encouraging collecting accessibility issues.

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

When the web started, you had a few pages. After a while you couldn't remember all the URLs anymore, so curated overview pages cropped up. Then even more pages, so the first automated search engines started appearing. And thus "keyword stuffing" as search engines only looked at the manual keywords. So search engines started looking at links between pages and sites. So people started gaming that system, too...

Same with email. With email came spam. And thus automated mail filtering. And spammers adjusting to it. And spam filters adjusting in turn. And on and on. A cat and mouse game. Not everyone in the audience remembered this cat and mouse game with search engines and spam.

If you have a security mechanism, you can expect the mechanism being attacked. A virus scanner can be used to attack the system it protects...

She once saw a quote from Harold Innes, 1952: "it should be clear that improvements in communication tends to divide mankind". For example the invention of the printing press. Soon afterwards you had someone named Luther and a split in the church and some wars, for instance...

If you're evil, which service to you target?

• Popularity with target audience.
• Ease of influence.
• More user features.

What helps you as an evil person?

• Anything that looks natural.
• AI anyone? Text, images. Helps a lot.
• Business metrics.
• Intransparency of the site/platform you're attacking.

LLM training. What are you training on? On crawled web pages? Pages that nowadays are filled with LLM-generated content? Bias upon bias, perhaps? There was a project called "wordfreq" that contained word frequencies. It stopped recently as "post-2021 language isn't usable anymore".

OODA loop: observe, orient, decide, act. A quote by Elen Friedman, 2007: "the ability to look implies the ability to influence".

As said at the beginning: not everyone in the audience remembered this cat and mouse game with search engines and spam. So watch out with LLMs and AI training: there is a cat and mouse game here, too. Watch out what you're doing yourself. Observe what your users are doing. Observe the ways bad actors might influence your training. Orient yourself, be aware of your surroundings.

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

Full title: chasing the dark universe with euclid and python: unveiling the secrets of the cosmos.

Guadalupe works on the ESA Euclid mission. Euclid tries to understand the universe and especially its history: how did it came to be? What is dark matter/energy and how does it work? 95% of the matter-energy content is unknown! 5% is ordinary matter ("baryonic matter"), 25% "dark matter" and 70% "dark energy".

How do we know it? It is all about statistics. Bayes' theorem. Comparing distributions. Comparing data and theory.

We want to study dark matter: we cannot see it. We want to study dark energy: we don't even know what it is. So We must learn how to measure the useeable. Dark magic...

A start is gravitational lensing. Light from far-away objects is distorted by galaxies in between. Or by the mass of dark matter. They're doing statistics with positions, shapes and distances on galaxies. Lots and lots of galaxies. Lots and lots of data.

Data from the Euclid satellite, launched in 2023 with a Falcon 9 rocket. It contains the largest camera ever sent to space. It also has a near-infrared spectrometer and photometer to measure distances to galaxies.

Data. Lots of data. Euclid has collected as much as Hubble in the last 35 years. The first real big data release will be done in 2026. 19 March 2025 they released their first quick data release. Publicly. You can go to https://sky.esa.int/esasky/ to visually look at the images. 24kx24k. But you can also actually download the data at https://datalabs.esa.int . You can even help classifying galaxy shapes. They have info at https://github.com/euclid-consortium, also look at https://github.com/euclidlib/euclidlib . Lots of python.

She herself worked on python software that was used internally in the project. They realised it was a black box. Making it public was a much better idea. https://github.com/cloe-org (some still has to be made public). Since they opened it up, they got help improving the code, improving the software pipelines, improving code structure, etc. That's the way scientific software should be done :-) In the open. Python-powered science at scale.

They'd love help and contributions, especially regarding using AI to classify the information.

Photo explanation: picture from our 2024 vacation around Kassel (DE)

(One of my summaries of the 2025 pycon.de conference in Darmstadt, DE).

The earth is no longer flat, at least in the spherely/geopandas world :-)

In shapely and geopandas, latitude/longitude are normally treated as rectangular x/y coordinates. This is incorrect as the earth is round. So distances you calculate will be wrong.

A solution is to work with projected coordinates. So if you use a local projection (like "Lambert 72" for Belgium), your calculations will be perfectly fine within that region. The projected "flat" coordinates will be good enough.

Why don't we use projected coordinates for everything? Well, because it isn't possible. You get distortion, like the regular "mercator" projection that dramatically enlarges Greenland, for instance. Data that's near the poles also gets funny. Oh, and if you have data on Fiji, which is around 180 degrees longitude, your data might be projected partially on the left and partially on the right side of the map...

Well, what is the shape of the earth? You can treat it as a perfect sphere. That's a useful approximation, which is also used in geopandas. But, yes, it is an approximation: the earth is actually more a spheroid. But a perfect sphere is good enough for most applications.

s2geometry is an open source geometry library (made by google) that's useful. Spherely is a python binding for s2geometry, intended to be familiar to shapely users. It was made with PyBind11. It tries to use a numpy vector interface like shapely 2.0.

It is still in an early stage of development, but most of the basic features are already implemented and work just fine. Some functionality that is missing (April 2025): buffer, distance-within, bounds, project/interpolate. Spatial indexing should be faster. (Note: the geopandas integration is almost ready, but it has not been released yet).

He showed some examples where spherely gave the right answer (and shapely the wrong one).

An advantage of using the spherical method directly is that it is faster than first converting the data to a local coordinate system.

Photo explanation: picture from the recent "on traxs" model railway exhibition in Utrecht (NL)