fidzu : kernel

I have a Samsung Q800A soundbar. It ships with built-in support for the Apple Airplay protocol, but I'm not really an Apple kind of guy so that's never been particularly helpful for me. So, instead, I have a Chromecast Audio plugged into it via TOSLINK, and that works fine - except for the fact that there's no way to explicitly pick TOSLINK as the audio input. Instead, there's a "Digital In" option that refers to both the optical in and eARC input from a TV. Not inherently a problem, given I'm unlikely to be watching TV while listening to music, but my TV is a Samsung Frame, a device that's intended to display art whenever someone is standing nearby, and which infuriatingly generates an eARC signal even if it's only in art mode and isn't going to make any noise. This switches the soundbar away from optical and I get sad. I could plug a video cast device into the HDMI port on the soundbar and cast audio to that, but then the art would get replaced with album covers whenever I'm playing music, and again that's not quite the ambience I want. Still, none of this is that important.

Until I learned that the Australian version of the soundbar does have Google Cast support. The firmware image was downloadable from Samsung support, so I grabbed that, but unfortunately found that it was encrypted. However, if I downloaded the firmware image for the US model, the encrypted blob was identical - there's no difference in the underlying firmware between the two of them. So, how come one has support, and the other doesn't?

I'd need an unencrypted copy of the firmware to have a real chance of figuring this out, and the easiest way to get one was to just run an ebay watch for a broken model of the same series. Once I had one in my hands I used hot air to remove the eMMC storage from the device, cleaned up the chip, and then stuck it in one of these adapters to read it on a local system. Samsung uses a weird in-house filesystem called VDFS4 on some of their devices and this was one of them, so I needed to cobble together a kernel that was compatible with their source dump for this device (not all VDFS drivers seem to be able to read all VDFS filesystems), but eventually I had it all extracted, and then I got distracted by other stuff, so let's fast forward to this week.

Samsung makes use of a modified version of the Android Debug Bridge called sdb, which is included in the Tizen SDK. The firmware showed that this device had sdbd on it, and it would be executed if a network device called "huart0" was plugged in. Samsung does, thankfully, provide the source code for their GPLed components, so I could dig through that to discover that drivers/net/usb/usbnet.c had a bunch of references to huart (which apparently stands for "HyperUART"). If CONFIG_SAMSUNG_USBNET_HYPERUART_IFNAME is set, and if the ethernet device is using either the asix or ax88179_178a drivers, and if "/opt/etc/HyperUART_IP_Addr"¹ isn't just "0", it sets the interface name to "huart0" and in turn triggers sdbd startup. So now I just needed to figure out how to set /opt/etc/HyperUART_IP_Addr to something else.

Some digging revealed a binary called wap-factory that referenced the file, which then led to /usr/share/mobile-mgr/jsons/com.samsung.audio.waapp-verification-test.json which mentioned a "SetUartOnOff" call, which brought me here², which led me into a bit of a rabbit hole about the Samsung WAM protocol. Long story short, you connect to either port 55001 or 56001 and make two requests over the same connection:

1
2
3
4
5
6
7
8
9

GET /connect HTTP/1.1
Host: 192.168.87.126:56001
User-Agent: SamsungMultiroom/1.0
Connection: interactive

GET /UIC?cmd=%3Cname%3ESetUartOnOff%3C/name%3E%3Cp%20type%3D%22str%22%20name%3D%22uartOnOff%22%20val%3D%22on%22/%3E HTTP/1.1
Host: 192.168.87.126:56001
User-Agent: SamsungMultiroom/1.0
Connection: interactive

and then plug in an asix chipset ethernet adapter into the USB port (which on mine is micro-B, so I needed an OTG adapter), and suddenly it's sitting there on 192.168.250.250 and will speak sdb to you. sdb root on switches it into root mode, and sdb shell now gives you a root shell. I was in, and now I could investigate what was happening.

First up, there was a process called "com.samsung.audio.google-cast-control-app" running, which seemed like a good start. Unfortunately it wasn't doing anything. After some tracing I found that it was making a call to Samsung at startup time, and getting back a response telling it that Cast support wasn't available. The request (which was helpfully dumped to stdout when I ran this by hand) included the country code, in my case "US". Repeating the request by hand with curl and changing the code to "AU" told me it was supported, so now I needed to figure out how to get the soundbar to believe it was Australian. Some digging showed it was reading this from a config variable called db/comss/countrycode, and running

1

buxton2ctl set-string system db/comss/countrycode AU

got me past that gate. Now, however, it was complaining that it couldn't provision a key. /usr/apps/com.samsung.audio.google-cast-control-app/shared/res included a bunch of encrypted blobs and an app called teeStoreKey.elf, and tracing com.samsung.audio.google-cast-control-app showed it was trying to call that, but it seemed like it was failing with a permissions issue. Running com.samsung.audio.google-cast-control-ap directly as root triggered key provisioning successfully, and a few seconds later the rest of the Cast API came up and I could talk to port 8443 with standard Cast API calls. Unfortunately it still wasn't working as a Cast device, though - it wasn't advertising itself via mDNS and so nothing could see it. Querying the /setup/eureka_info endpoint gave me a clue - the 'tos_accepted' field was false. Normally this would be set as part of initial device setup, but the Samsung app wasn't doing that because it didn't think the device had Cast support. It turns out that POSTing some json with {"tos_accepted": true} to setup/set_eureka_info made it work, and suddenly there was an additional Cast device available to me.

(Update: I realised I didn't mention anything about why it was doing key provisioning in the first place! The Cast protocol doesn't enforce cryptographic identity as such, but Google's reference client implementation which is what almost everything uses verifies that the device is presenting an identity that chains back to Google. Unsurprisingly they don't want the private key material to be easily extractable and used to produce clone devices, so it's all supposed to be encrypted and only accessible to some sort of trusted execution environment, which is why keys here need to be loaded into Trustzone and never be visible in the clear to the main OS)

Mission accomplished. Unfortunately the device vanished from my Smartthings home aferwards, presumably because there's a mismatch between its reported country code and my account country. To get around that I'd somehow need to get com.samsung.audio.google-cast-control-app to either change the query without changing the global device country, or to ignore the response from Samsung (it does this every 24 hours, ugh), and that's made more annoying by the fact that this thing will only execute signed binaries. Although, weirdly, one of the signed binaries shipped is a full Python interpreter, so if I cared enough I could probably cobble together enough Python to execute arbitrary binaries. I currently don't.

More modern soundbars don't implement the listener on port 55001 or 56001. I don't know how you get sdb working on them.

30 Oct 2026 6:03pm GMT

The sched_ext Microconference was very active this year at Linux Plumbers in Prague. It featured
six talks, preceded by an opening discussion on the future direction of sched_ext.
(Written by Andrea Righi)

Three main topics dominated the discussions

• Priority inversion and how to handle it in sched_ext: lock holder and
  lock waiter preemption, time slice extension, and making proxy execution
  work together with BPF schedulers.
• Paravirtualized scheduling: letting the host and guest schedulers
  cooperate across the VM boundary, with sched_ext as the policy engine on
  the host.
• Device and topology-aware scheduling: using hardware feedback (Intel
  HFI), device/accelerator locality and cache topology to drive task
  placement.

Intro: where sched_ext is going

The opening discussion outlined three directions for sched_ext development:

• Move more into BPF. With BPF arenas, queuing and idle CPU management can
  be implemented entirely in BPF, and several schedulers are already
  converging on this model. Once they all move, a significant amount of
  in-kernel sched_ext code can be removed.
• Composability. Long discussed, now a good time to tackle it: reusable
  BPF/user-space building blocks (load balancer from one scheduler, idle CPU
  selection from another, etc.) that can be combined into new schedulers.
• Generate BPF code from Rust. Most schedulers today are Rust in
  user space plus C for the BPF part. The trend, aligned with the wider
  BPF community, is to generate the BPF code from Rust as well, so a
  single language and shared abstractions can be used to access
  scheduling data structures from both sides. Prototypes exist but this
  is not there yet.

Priority inversion

Making proxy execution compatible with sched_ext
(Andrea Righi and John Stultz)

Proxy execution lets a mutex-blocked task lend its scheduling context to the
lock owner. The work presented makes this compatible with sched_ext, with
support enabled explicitly by each BPF scheduler. Dispatching a blocked task
triggers the core proxy machinery, keeping the BPF scheduler in control.

The compatibility changes and follow-up improvements are queued for Linux v7.4.
Distributions will be able to enable both `CONFIG_SCHED_PROXY_EXEC` and
`CONFIG_SCHED_CLASS_EXT` in the same kernel, removing their previous mutual
exclusion. The follow-up changes will also make proxy migrations transparent to
BPF, without exposing intermediate bookkeeping through extra enqueue callbacks.

Discussion points:

• User-space locks: futexes are not covered since `FUTEX_WAIT` does not
  identify the owner. A new futex op that carries ownership is being explored;
  passing user-space owner hints (e.g., via BPF maps or futex flags) is
  inherently racy. A "directed yield" (timed proxy to a specific task) was
  mentioned as future work.
• rwsems / semaphores: writer boosting is supported via a patch; boosting
  multiple readers is harder and has overhead. Plain semaphores have no owner
  and cannot be proxied.
• Observability: there are no upstream tracepoints for proxy execution
  yet, which makes traces confusing (the owner runs under the donor's
  context). Something similar to the existing PI tracepoints was requested.

Taming latency spikes caused by lock holder and lock waiter preemption
(Changwoo Min)

On the Steam Frame VR headset, camera processing must finish within 33ms but
needs only about 7-8ms of CPU time. Under heavy gaming load, preempting a lock
holder or delaying a woken lock waiter can stretch the pipeline enough to miss
that deadline.

LAVD addresses both cases with time slice extension and boosting of lock
waiters. The mitigations reduced scheduling delays above 2ms across four games,
with no missed camera frames observed during an hour of gameplay in the tested
titles.

A common interface for lock ownership and waiting information, covering kernel
and user-space locks, would help replace scheduler-specific heuristics.

Discussion points:

• Abuse of the extension is bounded by fairness: a task that constantly
  extends its slice is deprioritized.
• Why not just rely on proxy execution? Proxy is reactive (owner scheduled
  out, donor scheduled in, owner scheduled in again), with extra overhead.
  The consensus was that the two approaches are complementary: slice
  extension is like a spinlock (cheap, good for short critical sections),
  proxy execution is like a mutex (heavier, but handles long ones).

Paravirtualized scheduling

Bridging the VM Boundary: Scheduling Passthrough via pvsched and sched_ext
(Josh Don and Vineeth Remanan Pillai)

A host scheduler normally sees opaque vCPU threads, without knowing whether
they are running urgent guest work or background jobs. pvsched provides a
bidirectional channel: guests communicate priorities and preemption hints,
while hosts report which vCPUs they can run concurrently.

The proposed v3 design moves negotiation into the VMM through a virtual PCI
device, keeping policy outside KVM. On the host, sched_ext can implement bounded
vCPU boosting and gang scheduling. Guests can concentrate work on the vCPUs the
host can actually run, reducing the cost of double scheduling.

The next revision is still to be posted upstream.

Discussion points:

• BPF arenas as an alternative to a dedicated device: a host scheduler
  could allocate an arena, hand out per-guest chunks and get zero-copy
  communication with no kernel changes, needing only a way to pin arena
  memory. Someone from the audience pointed out that allocating memory on
  the host and mapping it into the guest on the fly is slow (memslot
  creation, guest notification), which is why guest-initiated allocation was
  chosen. However, this cost can be avoided: the host BPF scheduler can
  create the arena once when it is started and hand out per-guest chunks
  from the preallocated arena, mapping each chunk when the guest is set up
  rather than on the fly, which keeps the fast path cheap. It was also
  noted that the device model keeps the handshake arch-agnostic and
  evolvable without kernel changes.
• There is related work from others in this area; the agreement was that
  the various efforts should converge on a single kernel module, possibly
  alongside a BPF-arena-based path.
• Standardization: post the kernel module first, then the QEMU device, then
  other VMMs.
• The preferred-CPU framework (steal governor) is related, but reactive,
  while pvsched allows an explicit policy.

Device and topology-aware scheduling

Automatic placement of accelerator workloads
(Balbir Singh and Lee Trager)

CPU and memory placement can have a large effect on accelerator performance,
but manual pinning requires applications or administrators to know the topology
and device usage in advance. The proposed alternative combines hints about
which device a task is using, sched_ext task placement, and memory-access
telemetry to guide page placement. PMU counters were also suggested as a source
of remote-access and bandwidth telemetry to inform those decisions.

Two interfaces are needed: a generic process-to-device association and BPF
control over NUMA placement preferences. A series enabling NUMA balancing under
sched_ext had recently been posted, together with a kfunc for requesting a
preferred memory node.

Discussion points:

• Exposing PMU telemetry to non-root applications raises side-channel
  concerns; IPC is fairly safe, cache/TLB misses are a gray area. An
  allowlist of audited events was suggested.
• Is a BPF map the right interface, or is a common user-space library needed
  so all frameworks can provide hints?
• Migrating network-heavy tasks across nodes may interact badly with per-CPU
  queues and TCP flows.
• NUMA balancing needs improvements, including asynchronous migration and
  charging migration work to the right task. Enabling it under sched_ext
  reuses the existing mechanism, so that work can proceed in parallel with
  those fixes. Telemetry-driven `move_pages()` can serve as an interim solution.

HFI plug-in for sched_ext
(Srinivas Pandruvada)

Intel's Hardware Feedback Interface supplies per-CPU performance and efficiency
hints that reflect the platform's changing power budget. Integrating these
hints into LAVD improved Shadow of the Tomb Raider performance by about three
frames per second on a hybrid Intel handheld at a 10W power limit.

The prototype took less than a month. A Rust user-space thread receives HFI
updates over netlink and passes them to BPF through a map; LAVD then adjusts CPU
selection and effective capacity.

Discussion points:

• Why is HFI not used by the in-kernel scheduler? Older generations were too
  noisy, and the firmware was tuned mostly on Windows workloads; the last two
  generations treat Linux on equal footing, so it is worth revisiting.
• Qualcomm's WALT has a similar CPU isolation feature (thermal and
  low-utilization cases), implemented downstream via Android vendor hooks.
• The preferred-CPU framework (steal governor) could be a natural in-kernel
  hook for HFI exclusion. The thermal pressure signal filter currently only allows
  slowing down the reaction time, and patches to make it configurable in both
  directions may be revisited given how fast firmware can now report changes.
• The HFI code is structured as a library and was proposed for inclusion in
  the scx library so other schedulers can reuse it.

Stickiness: keeping tasks cache-warm in the work-conserving LAVD scheduler
(Gavin Guo)

Keeping a task on a cache-warm CPU can improve performance, but waiting for that
CPU competes with the benefit of running immediately elsewhere. The session
compared cache-warmth modelling, CPU reservation, and look-ahead dispatch from a
shared queue.

Results depended on the workload and approach. Warmth-based placement improved
P99 latency for short waits, while look-ahead dispatch showed no noticeable
memcached latency improvement under light or moderate load and added overhead
under heavier load.

Discussion points:

• Staying on a busy CPU is a short-term gain, while an idle CPU may offer
  long-term uninterrupted execution, so the right answer is very workload-
  and topology-dependent (shared vs non-shared L2/L3, snoop efficiency).
• Suggestions included using PMU counters (as scx_cosmos already does) or Intel
  RDT/CMT for cache occupancy (limited by the number of monitoring IDs),
  tracking only latency-critical tasks, and broader workloads such as
  stress-ng and video encoders.

28 Oct 2026 2:57pm GMT

The Kernel Testing & Dependability Micro-Conference was back again at Plumbers this year, in Prague and with a wide variety of topics. In addition to the well-established frameworks such as KUnit and kselftest, some new frontiers are being explored with the spotlight on LLM-assisted tools. Developers are also getting special treats with new kci-dev features and expert insights into local build testing at scale.

Primary author: Guillaume Tucker

kci-dev

What Changed, What Works, and What Kernel Developers Still Need - by Arisu Tachibana (virtual)

The kci-dev command line tool enables developers to interact with KernelCI. Here's a summary of the latest changes:

• Core implementation moved to the kcidev Python module as an API library
• Standalone patch file testing enabled on top of a Git checkout
• Submission of results to KCIDB enabled with artifacts uploaded to storage
• Experimental MCP added on top of the API

The tool also benefits from these new plugins:

• kci-patchwork to apply patch series from Patchwork and submit test job
• kci_release_review to compare revisions and create reports

Plugins can help extend functionality so they should be a good way of driving future kci-dev development. The MCP may also enable new use cases. Please join the KernelCI mailing list and Discord channel as per the contact page if you wish to take part and discuss things further.

Build Identity

Kernel Builds Deserve an Identity: Portable Kernel Artifacts and Traceable Test Results - by Cong Wang

The raw metadata provided by uname, Git, IDs from various CI systems etc. does not provide any standard or consistent way to identify a build. As such, bug reports can't directly refer to a single reproducible binary.

Cong Wang's proposal is the Kernel Bundle Image (KBI) project. It draws a parallel with how OCI solved this for container images. The idea is essentially to create OCI images for kernel builds. However, it still has a low level of adoption and isn't used by current automated testing systems.

How may KBI gain more traction?

Where should the line be drawn about what to include in the kernel build ID? There hasn't been a clear consensus on this yet, whether it should include device trees, firmware blobs and other artifacts.

Yocto / OE also has a way to reproduce entire OS builds, including the kernel. This may be used as a guideline or potentially combined with the KBI approach.

It's also worth noting that TuxMake is able to reproduce binary-identical builds for a given set of inputs (container image and build parameters), which does not constitute a build ID per se but could still be potentially very helpful for this purpose.

A related initiative is KernelSbom since a bill of materials could define the input to a KBI image: source with metadata, main build artifacts, additional contents (firmware, BPF).

KUnit Configuration

Rethinking KUnit Configuration - by David Gow

The session started with a poll: roughly half of the room use KUnit regularly and 3 people answered yes to the last question about running KUnit with multiple configurations. This tends to show that the tool is well established for standard usage but few developers use it in an expert way.

The main issue covered by this session is about limitations with the .kunitconfig file and qemu_config Python module, especially the lack of output format options and the fact that each config covers only a single KUnit run.

Proposal: Determine a single standard configuration mechanism to go beyond limitations while being easy to share between developers. Passing metadata through to the results would also be especially useful for CI systems which can deal with more details and trace the results back to their origin.

Should this be solved with a new config system or by extending an existing one?

• Option 1: extend .kunitconfig (currently a plain Kconfig fragment)
• Option 2: native Python module since kunit.py is already in Python
• Any other options?

David's preferred option is to extend .kunitconfig.

For multiple runs bundled together, it's important to be able to parse the results as if they were individual ones e.g. objects embedded within a bigger blob. One potential solution is to produce results in separate files (KTAP).

Multiple runs are useful for CI integration with a single entry point. It also removes the overhead of setting up many short-lived individual jobs. This doesn't however appear to be too useful for manual runs. It can help a bit with git bisect run although multiple KUnit commands may be bundled within a single script.

It's also worth noting that having isolated kernel runs would prevent a single crash from breaking the whole series.

The overall consensus is that multiple runs in a single command or config aren't probably that useful except in particular cases e.g. CI hooks.

Build Testing

Optimizing local kernel build testing - by Arnd Bergmann

As a sequel from Arnd's talk at Kernel Recipes, this session is to discuss using a manual, local setup to achieve wide build testing coverage. The focus is on linux-next, with -Werror enabled to try and catch all new build issues.

The best approach when something breaks is to send a fix if possible as it is a lot more effective than just reporting bugs. By doing this for many years, Arnd has fixed thousands of bugs upstream (3000+) and touched every subsystem in the kernel.

The most common problems are Kconfig dependency bugs since its logic is complicated and deceptive, especially with tristate options. The new kconfirm tool aims at solving this issue via static analysis.

Arnd keeps a small config fragment to maximise the effectiveness of build testing:

# maximize search space, disable options not worth testing
CONFIG_COMPILE_TEST=y
# avoid long build times
# PROFILE_ALL_BRANCHES is not set
# reduce compile-time dependencies
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y

Should orchestration of parallel builds be integrated upstream?

This sounds useful although the discussion didn't expand on how to approach this topic in practice. The general makefile and arm64-specific testrandconfig64 files attached have been kindly provided by Arnd as a reference based on his local setup. Please feel free to reach out to him via email should you wish to discuss this further.

Should local build regression reports be sent to KCIDB? With a proposed fix in a patch?

Several people would welcome this and a follow-up discussion is intended to take place with the KernelCI team.

kcov and kselftest

Kselftest coverage with kcov - by Shuah Khan

Thousands of tests are available in-tree via kselftest and KUnit, but what is their coverage? Shuah has made an experimental tool to run kselftest with kcov and start looking into the issue. Coverage data is not ready to be published yet so the discussion was more about the potential use cases. You can already expect to see a more in-depth follow-up analysis on this topic next year and other general kselftest sessions at every future LPC.

How is code coverage useful?

When working with unknown customer workloads, comparing their coverage with well-known tests gives an idea of how much the tests are covering these workloads.

One issue with enabling kcov at build time is that it's not suitable for production kernels. However, ftrace is usually enabled so it could be used in combination with kcov with different granularity (function-level only coverage with ftrace).

Another dimension to the problem is about evaluating other tools e.g. LLVM/Clang coverage. See also the LLM fuzzing session on a related topic.

KFuzzTest

KFuzzTest: One year on - by Ethan Graham

The aim of KFuzzTest is to combine the benefits of fuzzing and unit testing. A new, experimental API is now available in the latest version of the patch series to make it easier to write tests and facilitate adoption.

The next core feature which is currently under development is stateful fuzzing. Initial trials found a known bug which had already been fixed but this showed the tool's potential.

Can fuzzing target specific code?

This is not trivial. In some cases, hardware mocks may be used to enable coverage on specific drivers. Generally speaking, KFuzzTest's aim is to narrow down fuzzing to isolated functions like a unit test would typically do. The effectiveness of this method is still being debated, perhaps the sweet spot lies in between this and syzkaller's broader approach.

How about stateful fuzzing?

Stateful fuzzing in userspace exists and enables setting extra boundaries. Kernel fuzzing is typically run from outside the kernel, kind of guessing how to invoke the syscalls. Having the kernel asking a userspace fuzzer has led to successful experiments in the past. So overall, stateful fuzzing is feasible.

How about KUnit overlap?

David Gow is in favour of adding functionality that lets userspace pass data into KUnit tests. This could also facilitate making reproducers. However, the stateful approach is not a good fit for KUnit with long strings of operations as unit tests are meant to be run in isolation.

LLM Fuzzing

LLM-Assisted Fuzzing with Syzkaller - by Chuyang Wang & Aleksandr Nogikh

The issue when fuzzing with high-level descriptions is that they don't go deep enough to reach good code coverage. Hand-written seed programs partially solve this but the approach doesn't scale.

The proposed solution here is to rely on LLM-generated ones instead. While attempting to reach areas never covered by syzbot, an overall 15% increased coverage was reached over a few months using this approach. This uncovered a new issue with the LLM trying to evade syzkaller's sandboxing environment. The next step is going to integrate with syzbot.

Is AI-driven cheaper?

Probably but not measured in terms of actual costs. One seed program could then expand coverage.

How about fuzzing drivers?

This is tricky because testing drivers often requires access to real hardware. USB is an exception as physical devices are not strictly necessary. This has enabled the team to significantly increase fuzzing coverage and report many bugs over a short period of time.

Still useful to upstream hand-written seed programs?

Yes, these are complementary approaches.

KernelCI in the Cloud

Testing the kernel where the fleet runs: KernelCI in the cloud - by Norbert Manthey & Stanislav Uschakow

Going beyond the historical use cases of KernelCI which originated on embedded platforms, this session is about testing kernels deployed in the Cloud. The system covers x86_64 and Graviton (arm64) VMs in an AWS lab. Additionally, nested KVM guests on particular VM types are also supported. It provides the typical scalability and variety of platforms one would typically expect to find in this environment. Developers may also run the tests themselves using their own AWS account.

Current upstream kernel coverage includes mainline, linux-next and LTS. Runtime coverage is limited to baseline boot testing and basic checks. Performance testing is next on the roadmap.

How is A/B testing done in the AWS Cloud?

The approach is to run both A and B kernels with the same VM definition to reduce dependencies on the environment (hardware of physical host, virtualization stack, firmware, …). Keeping the kernel as the single moving part enforces it to be the only root cause for any difference found in test results. The code for this A/B testing is made available as a GitHub repository under the KernelCI project umbrella.

How does A/B testing on AWS work with live migrations?

Tests are not relying on a file system for now to keep resource dependencies low. Hypervisor updates can cause one VM to stop but tests are run multiple times and in a distributed way so the impact is mitigated.

Are you running KUnit?

KernelCI is running KUnit in other environments, not in AWS. It's unclear whether doing this would increase existing coverage.

Which kernel trees have the priority?

Testing linux-next and mainline kernels should be the priority. Covering LTS is also useful to verify that fixes get backported successfully as they would typically be applied to the AWS forks too although those are outside the scope of KernelCI.

Are any more details of the test runs available?

In order to give maintainers access to more details about test runs, the AWS lab setup needs to expand its integration with KernelCI in general. Additional features have been requested during LPC such as being able to automatically bisect issues and sending regressions as direct replies to patches via emails threads. Those reports should also be sent to the kernel regressions list.

Testing BoF

Note - The speaker has apologised for not being able to host the last session of the schedule due to personal reasons. It was replaced with an impromptu Testing BoF instead.

One particular point was discussed:

How to address kselftest failing to build and run?

• Tests may fail to build due to missing dependencies. The kselftest_deps.sh tool can be used to discover those. Integrating it into the kselftest Makefile is being considered in order to report them directly to the user.
• Test build failures don't cause the overall kselftest build to fail. This is good, although reporting this in a better way might be helpful.
• Build and run regressions need better reporting, they could be made more visible using a tag in email subjects.

Wrapping Up

One lesson learnt is that AI-assisted development tends to be more about data analysis while the initial scope of the Testing MC is rather to focus on low-level tools. These may be seen as optimised, primitive functions producing raw data that LLMs can then leverage and potentially use to orchestrate test runs. Having a session about LLM-assisted fuzzing was a good fit though as it showcased how AI can make use of existing testing components in novel ways.

Last but not least, many thanks to all the speakers for reviewing this blog post, the conference committee and all the attendees for making Plumbers such a great experience. It was a pleasure to be a co-organiser of the Testing MC again this year.

See you at LPC 2027

20 Oct 2026 7:06pm GMT