fidzu : k8s

This article is a mirror of the original announcement

Today, SIG etcd is releasing etcd v3.7.0, the latest minor release of the popular distributed key-value store and core Kubernetes component. v3.7 ships the long-requested RangeStream feature, delivers several other performance improvements, removes the last remnants of the legacy v2store, and completes a major protobuf overhaul.

You can download etcd v3.7.0 here:

• Source code
• Binaries
• Official container images

This release also includes new versions of the two core etcd dependencies, bbolt v1.5.0 and raft v3.7.0.

For instructions on installing etcd, see the install documentation. For the full list of changes, see the etcd v3.7 changelog.

A heartfelt thank you to all the contributors who made this release possible!

Major features

The most significant changes in v3.7.0 include:

• RangeStream - stream large result sets in chunks instead of buffering the whole response.
• Keys-only range requests, faster and more reliable leases, and several other performance improvements.
• etcd now boots entirely from v3store, eliminating a long-standing dependency on the legacy v2 store
• A completed protobuf overhaul, replacing outdated protobuf libraries with fully supported ones.
• etcd v3.7 ships with bbolt v1.5.1 and raft v3.7.0.

Features

RangeStream

In etcd v3.6 and earlier, it is challenging to work with requests that return large result sets. The database would buffer the full result set before sending, leading to unpredictable latency and memory usage, both on the server and the client. The RangeStream RPC lets calling applications accept result sets in chunks, reducing latency and making buffering memory usage more predictable.

Instructions on how to use RangeStream in gRPC calls and in etcdctl can be found in the etcd documentation. Users should try it out for their own applications.

In coordinated releases, the RangeStream feature will become available to users running the upcoming v1.37 of Kubernetes by enabling the EtcdRangeStream feature gate. This early and planned adoption is possible thanks to the merger of etcd and Kubernetes development in 2023.

Performance improvements

v3.7 delivers multiple specific performance improvements, both for the Kubernetes control plane and for other use cases. Kubernetes users should see a significant decrease in overall CPU usage by the etcd members, compared with v3.6.

Keys-only range optimization

etcd v3.7.0 includes a keys-only Range optimization (#21791: keys-only Range optimization). When processing a keys_only Range request or etcdctl get --keys-only, etcd reads solely from its in-memory index. It returns the matched keys without loading all serialized values from bbolt as it did previously. The only exception where loading from bbolt is still required is when keys_only Range requests must be sorted by value (i.e., when SortTarget is set to VALUE).

This reduces unnecessary backend reads and memory use for workloads that only need key names, making large keys-only range requests more efficient.

Faster, more reliable etcd leases

v3.7 improves lease expiration and renewal:

• LeaseRevoke requests are now prioritized to ensure timely lease expiration during overload (#20492: stability enhancement during overload conditions).
• The new FastLeaseKeepAlive feature enables faster lease renewal by skipping the wait for the applied index (#20589: etcdserver: improve linearizable renew lease).

Faster find() operations

etcd 3.7 improves the performance of concurrent watches on keys by making find() operations faster (#19768: adt: split interval tree by right endpoint on matched left endpoints).

Other features

Protobuf overhaul

v3.7 migrates and replaces multiple outdated protobuf libraries with fully supported dependencies. This includes replacing github.com/golang/protobuf and github.com/gogo/protobuf with the fully-supported google.golang.org/protobuf (#14533: Protobuf: cleanup both golang/protobuf and gogo/protobuf), and migrating grpc-logging to grpc-middleware v2 (#20420: Migrate grpc-logging to grpc-middleware v2).

As well as improving security and maintainability, this refactor has been shown to reduce CPU usage by etcd components.

While these changes are not expected to directly affect users running etcd via official binaries or container images, they may affect users who depend on etcd Go modules, such as the client SDK or packages under api/ or pkg/. These consumers may need to update their code or dependencies due to protobuf and related API changes introduced in this release. More detailed information is available from the API change tracking issue.

Unix socket support

etcd now supports Unix socket endpoints (#19760: Add Support for Unix Socket endpoints), enabling local communication without a TCP port. Since this is restricted to single-member clusters, it is mainly aimed at development, testing, and edge device use-cases.

Bootstrap from v3store

One of the major changes in etcd v3.7 is that the server now bootstraps entirely from the v3 store (#20187 Bootstrap etcdserver from v3store), eliminating its dependency on the legacy v2 store during startup.

This milestone is the result of a long-term effort spanning multiple releases, from v3.4 through v3.7. It resolves a long-standing technical debt, significantly simplifies the bootstrap workflow, and lays the foundation for future improvements to etcd.

To maintain backward compatibility, etcd v3.7 continues to generate v2 snapshots. As a result, the --snapshot-count flag is also retained in v3.7. This is the last remaining dependency on the legacy v2 store, and both the v2 snapshot generation and the --snapshot-count flag will be removed in v3.8.

etcdutl timeouts

All etcdutl commands now have a timeout command line argument (#20708: etcdutl: enable timeout functionality for all commands), so offline utility commands no longer block indefinitely when holding a lock.

Setting the authentication token directly

Client v3 now allows users to set the JWT directly, offering more flexibility in authentication options (#16803: clientv3: allow setting JWT directly, #20747: clientv3: disable auth retry when token is set),

Retrieve AuthStatus without authenticating

Clients can check their AuthStatus without attempting to authenticate first, eliminating some application overhead (#20802: etcdserver: remove permission check on AuthStatus api).

New watch metrics

v3.7 adds optional watch send-loop metrics (#21030: Instrument watchstream send loop) for better observability of the watch path:

• etcd_debugging_server_watch_send_loop_watch_stream_duration_seconds
• etcd_debugging_server_watch_send_loop_watch_stream_duration_per_event_seconds
• etcd_debugging_server_watch_send_loop_control_stream_duration_seconds
• etcd_debugging_server_watch_send_loop_progress_duration_seconds

There is also a new etcd_server_request_duration_seconds metric (#21038: Add metric etcd_server_request_duration_seconds).

etcdctl command cleanup

etcdctl commands were reorganized for clarity (#20162: etcdctl: organize etcdctl subcommand) and global command line arguments are now hidden to streamline help output (#20493: etcdctl: hide global flags).

Upgrading

This release contains breaking changes, particularly around the removal of legacy v2 components. Users should review the upgrade guide before upgrading their nodes. As with all minor releases, perform a rolling upgrade one member at a time and confirm cluster health between steps.

Experimental flags removed

All deprecated experimental flags have been removed (#19959: Cleanup the deprecated experimental flags). Features in etcd now follow the Kubernetes-style feature-gate lifecycle (Alpha → Beta → GA) introduced in v3.6, rather than the old --experimental prefix. If your configuration still relies on --experimental-* command line arguments, migrate to using the corresponding feature gates or stable command line arguments before you upgrade to etcd 3.7.

Legacy V2 API packages and code cleanup

To remove the dependencies on v2store, the following components have been removed:

• v2 discovery (#20109: Remove v2discovery) packages removed,
• v2 request support (#21263: Remove v2 Request and apply_v2.go)
• v2 client support (#20117: Remove client/internal/v2).

These changes may create some breakage for users, particularly those who have not already updated to v3.6.11 or later. Users should report any blockers encountered, or cases that need better upgrade documentation.

Non-blocking client creation

etcd no longer honors the deprecated grpc.WithBlock dial option ( #21942: Make the etcd client creation non-blocking). To preserve the previous blocking behavior when needed, follow the guidance in grpc-go's anti-patterns documentation.

Multiarch container images only

For users relying on the official etcd container images, v3.7 will be distributed only as multiarch containers. Architecture-tagged images will not be available, so adjust deployments accordingly.

API changes

As with every etcd release, there are a number of API changes. These are designed to be backwards-compatible to the extent possible, but may require adjustment by some users. See our API documentation page for full information.

bbolt v1.5.1

etcd v3.7 depends on, and includes, v1.5.1 of the bbolt storage engine. v1.5 includes several improvements to functionality and performance, including:

• Database file size limits: users may set, and bbolt will enforce, file size limits. When a bolt database exceeds these limits it will refuse to accept writes until the database is compacted or the limit is changed.
• Disable statistics for performance: users may set NoStatistics to limit overhead from locks taken by the database statistics viewer.
• More efficient hashmap processing: merge spans faster and with less overhead.

raft v3.7.0

etcd 3.7 depends on, and includes, v3.7.0 of the raft consensus engine. v3.7 includes several improvements, including:

• Update the bootstrap process: v3.7 now allows booting from partly initialized snapshots, supporting etcd's initializing directly from v3store.
• Improve the ReadIndex flow to prevent stale reads by injecting a unique identifier into the heartbeat context for read-only operations.

raft v3.7.0 also includes the same protobuf library updates and refactoring as etcd does.

Dependency updates

Other dependency updates include a bump to golang.org/x/crypto v0.52.0 for CVE resolution (#21903: [release-3.7] Bump golang.org/x/crypto to v0.52.0), an OpenTelemetry contrib update to v0.61.0 (#20017: Update otelgrpc to v0.61.0), and compilation with Go 1.26.4 (#21891: [release-3.7] Update Go to 1.26.4).

Contributors

etcd v3.7.0 is the product of more than a hundred contributors across the community. Thank you to everyone who wrote code, reviewed PRs, filed and triaged issues, and helped test the alpha, beta, and release candidates.

Leads

The SIG etcd leads for the v3.7 release are ivanvc, serathius, ahrtr, fuweid, siyuanfoundation, and jberkus. Ivan leads our release team.

Other contributors

ah8ad3, ajaysundark, aladesawe, amosehiguese, ArkaSaha30, ashikjm, AwesomePatrol, dims, Elbehery, gangli113, henrybear327, Jille, jmhbnz, joshuazh-x, kishen-v, lavishpal, liggitt, marcelfranca, miancheng7, mmorel-35, MrDXY, mrueg, purpleidea, qsyqian, redwrasse, ronaldngounou, skitt, spzala, tcchawla, tjungblu, vivekpatani, wenjiaswe

New contributors

A special welcome to the contributors who made their first etcd contribution in this cycle - including Jeffrey Ying, whose work drove the RangeStream feature. New contributors can have a substantial impact on etcd; if you'd like to get involved, see the contributor guide.

1911860538, 4rivappa, aaronjzhang, abdurrehman107, ABin-Huang, adeptvin1, aditya7880900936, AHBICJ, akstron, alliasgher, aman4433, aojea, apullo777, AR21SM, arturmelanchyk, AshrafAhmed9, asttool, asutorufa, BBQing, beforetech, boqishan, caltechustc, carsontham, christophsj, chuanye-gao, cnuss, cuiweixie, dmvolod, Dogacel, dongjiang1989, EduardoVega, evertrain, eyupcanakman, gaganhr94, goingforstudying-ctrl, greenblade29, Himanshu-370, HossamSaberX, huajianxiaowanzi, hwdef, ishan-gupta2005, ishan16696, ivangsm, JasonLove-Coding, Jefftree, jihogh, jonathan-albrecht-ibm, joshjms, kairosci, kei01234kei, kjgorman, kovan, kstrifonoff, Kunalbehbud, letreturn, lorenz, m4l1c1ou5, madhav-murali, madvimer, majiayu000, marcus-hodgson-antithesis, mattsains, mcrute, mingl1, MohanadKh03, mstrYoda, NAM-MAN, neeraj542, nicknikolakakis, nihalmaddala, niuyueyang1996, notandruu, ntdkhiem, nwnt, olamilekan000, pigeio, pjsharath28, progmem, Qian-Cheng-nju, quocvibui, ravisastryk, robin-vidal, robinkb, rockswe, roman-khimov, rsafonseca, sahilpatel09, SalehBorhani, SebTardif, seshachalam-yv, shashwat010, shivamgcodes, shuan1026, silentred, sneaky-potato, socketpair, srri, subrajeet-maharana, sxllwx, tchap, tsujiri, tzfun, upamanyus, uzairhameed, varunu28, vihasmakwana, wendy-ha18, xiaoxiangirl, xigang, xUser5000, yagikota, yajianggroup, yedou37, Zanda256, zechariahkasina, zhijun42, zhoujiaweii

Feedback can be shared through:

• GitHub issues
• #sig-etcd slack channel in Kubernetes Slack
• etcd-dev mailing list

08 Jul 2026 12:00pm GMT

AI has really changed the game around software development. More people are leveraging AI than ever to contribute patches to projects they use. To me, this is a good thing as more folks will contribute patches rather than fork or not fix them. The main problem is that AI has made generating code fast but there has been very little improvement in maintaining code bases. In this post, we will highlight the ways the Kubernetes community is adapting to the world of AI assisted coding.

The first step of this journey was to develop an AI policy. This seems mundane and bureaucratic but there were many PRs that derailed into discussions around AI usage. The AI policy helps steer the conversation around the project's stance on AI and provides a clear signal to contributors on how to use these tools responsibly.

Kubernetes AI policy

The Kubernetes project has established clear guidelines for AI-assisted contributions that balance innovation with accountability. These policies are designed to maintain code quality and ensure human oversight while acknowledging that AI tools can be valuable aids in the development process.

Transparency first

Contributors must disclose when AI tools have been used to assist with a pull request. A simple statement in the PR description such as "This PR was written in part with the assistance of generative AI" is sufficient. This transparency helps reviewers understand the context and apply appropriate scrutiny.

Human accountability

While AI tools can assist, the human contributor remains fully responsible for every change. The policy explicitly prohibits:

• Listing AI as a co-author on commits
• Using AI co-signing on commits
• Adding trailers like "assisted-by" or "co-developed" that attribute work to AI

This isn't about diminishing AI's role as a tool-it's about maintaining clear accountability. If something breaks, there needs to be a human who understands why and can fix it.

CLA enforcement for co-authors

The CNCF provides a tool for verifying the contributor license agreements on each pull request. AI agents are not able to solve these contributor license agreements so one enforcement the project made is to enable the CLA check for co-authors. This provides a flag to reviewers that the PR is not ready to merge.

Human engagement required

Perhaps the most critical aspect of the policy: reviewers expect to engage with humans, not with AI. Contributors cannot rely on AI to respond to review comments. If you cannot personally explain changes that AI helped generate, your PR will be closed. This requirement ensures that knowledge transfer happens and that contributors genuinely understand the code they're submitting.

Verification obligations

Contributors must verify AI-generated changes through code review, testing, and personal understanding. It's not enough for the code to work-you need to know why it works and be able to maintain it.

These policies reflect a mature approach to AI: embrace it as a tool, but never let it replace human judgment, understanding, or responsibility.

Automated AI reviews

There exist many tools to aid in reviewing code. AI pull request tools introduce governance challenges so one of the first tasks the community took on was to document the process for what is needed to bring in new AI tools. One of the major evaluation criteria for these tools is to find maintainers willing to test drive them in kubernetes-sigs repositories. Kueue, JobSet and Agent-Sandbox have been experimenting with these tools to provide more support for maintainers.

Copilot

One tool that many maintainers started using was GitHub Copilot. The CNCF provides access for maintainers so this ended up being the first tool many started using. It provides some good experience on tuning reviews but there were some growing pains with this tool. The biggest blocker for community adoption is relying on contributors to have a copilot license. Only maintainers were able to request copilot reviews and automated reviews of pull requests was out of reach for the community. One of the goals of AI review tools is to provide an automated review tool that maintainers don't need to request. This demonstrated the need for organization control rather than relying on contributors having access.

CodeRabbit

In mid 2026, the Kubernetes community has rolled out CodeRabbit to a few projects. As with copilot, some tuning has been required to provide better reviews but the overall feedback has been positive. There is a lot of configuration available for this tool and one of the most interesting uses of this tool comes from agent-sandbox.

AI pull request tools can be a quality gate. Contributors can at least get a quick spot check review without waiting for a maintainer. Agent-sandbox has added a label on PRs to reflect that there is still a need to resolve some of the comments from AI tools.

Next steps

The reality is that leveraging AI in open source projects is an area of active exploration. The community could use your help in tuning reviews tools, evaluating tools or evaluating emerging technologies in the AI space.

Some areas we are exploring more:

• The use of AI skills to reduce maintainer burnout.
• AI assisted triage of failing tests.
• Skills to aid the operational aspects of Kubernetes.

26 Jun 2026 6:00pm GMT

Headlamp is an open-source, extensible Kubernetes SIG UI project designed to let you explore, manage, and debug cluster resources directly from a browser.

Cluster API (CAPI) is a Kubernetes sub-project that brings declarative, Kubernetes-style APIs to cluster lifecycle management. It lets platform teams provision, upgrade, and manage the lifecycle of Kubernetes clusters using standard Kubernetes objects stored and reconciled in a management cluster.

Managing Cluster API resources has historically required raw kubectl commands and deep familiarity with ownership hierarchies. The Headlamp Cluster API plugin brings visual clarity, faster debugging, and simplified operations for platform teams, directly inside Headlamp.

What this plugin provides

The Cluster API plugin adds a dedicated Cluster API section to Headlamp and brings full visibility into core CAPI resources through consistent list and detail views.

Feature	Description
Cluster overview	View clusters with live control plane and worker replica status.
Machine visibility	Inspect MachineDeployments, MachineSets, Machines, and MachinePools with status and conditions.
Cluster API dashboard	Get a centralized view of Cluster API resource health, active condition issues, provider information, and remediation guidance.
Control plane monitoring	Track KubeadmControlPlane replicas, versions, and associated Machines.
Scale from the UI	Scale MachineDeployments and MachineSets directly from Headlamp.
Owned resource hierarchy	Trace relationships between clusters, deployments, sets, and machines.
KubeadmConfig inspection	View bootstrap configs, files, kubelet args, and join/init settings.
Topology awareness	Automatically detect and label ClusterClass-managed resources.
Map view	Visualize Cluster, Control Plane, and Worker relationships.
Dynamic API versioning	Supports both v1beta1 and v1beta2 Cluster API versions.
Prometheus metrics	View live metrics from the Headlamp Prometheus plugin inline on Cluster API resource detail pages.

A tour of the plugin

The Headlamp Cluster API plugin brings core Cluster API resources into a consistent, visual interface inside Headlamp. Here are some of the key views included in the first release.

Cluster API dashboard

The dashboard provides a centralized view of Cluster API resources and their health across a management cluster.

The overview summarizes the status of clusters, Machines, MachineDeployments, MachinePools, MachineSets, and control planes. It also highlights active condition issues, provider information, and configuration template counts to help operators quickly identify degraded or unhealthy resources.

Selecting a cluster opens a detailed health view showing control plane and worker status, machine information, infrastructure details, and resource conditions. When issues are detected, the dashboard provides remediation guidance and diagnostic commands to assist with troubleshooting.

Bring full Cluster API visibility into Headlamp

The cluster list view shows all Cluster resources in the management cluster, including control plane and worker replica status. This gives you an at-a-glance understanding of overall cluster health.

The cluster detail view provides resource status, conditions, infrastructure references, control plane references, and related Machines on a single page.

Explore Cluster API resources in a visual interface

Dedicated views are available for MachineDeployments, MachineSets, Machines, and MachinePools. These pages surface replica counts, ownership relationships, provider IDs, versions, and conditions to support day-to-day operations and debugging.

Scale workloads directly from Headlamp

MachineDeployments and MachineSets include a built-in Scale action, allowing you to adjust replica counts directly from Headlamp without using terminal commands.

For topology-managed clusters, the plugin also indicates when scaling should be performed at the Cluster level.

Inspect bootstrap configuration without raw YAML

Bootstrap configurations can be viewed in a structured format, including inline files, kubelet arguments, extra volumes, and join or init settings. This removes the need to inspect raw YAML or secrets manually.

Visualize cluster relationships with map view

A visual map view displays the relationships between Cluster, control plane, and worker resources. It offers a faster way to understand ownership hierarchies and overall cluster structure.

Prometheus metrics integration

The Cluster API plugin integrates with the Headlamp Prometheus plugin to surface metrics directly inside Cluster API resource detail pages.

When the Prometheus plugin is installed and configured, metrics are embedded inline on the detail pages for Clusters, MachineDeployments, MachineSets, and Machines. You can view resource health and performance data alongside status conditions and ownership relationships, without switching to a separate dashboard.

This makes it easier to correlate infrastructure state with live metrics during debugging or day-to-day cluster operations, all from within Headlamp.

How to use

See the plugins/cluster-api/README.md for installation and usage instructions.

Developed during LFX Mentorship

This plugin was developed as part of the CNCF LFX Mentorship program under the Headlamp project. The mentorship provided an opportunity to work closely with the Headlamp community while building features to improve the Cluster API management experience.

The focus was not only on implementing features but also on understanding real-world usability challenges around Cluster API operations. Discussions with mentors and community members helped shape the plugin's direction, improve the user experience, and prioritize features most useful to platform teams.

The mentorship also provided valuable experience contributing to large open-source projects: collaborating with maintainers, participating in design discussions, handling release feedback, and iterating on features based on community input.

Work on the plugin is ongoing, with additional improvements and features planned beyond the initial Alpha release.

Feedback and questions

This is an Alpha release, and community feedback directly shapes what comes next.

• Bug reports: Open an issue
• Feature requests: Start a discussion
• Contributing: PRs are welcome
• Kubernetes Slack: Join the #headlamp channel for questions and discussion

25 Jun 2026 10:00pm GMT