26 Jul 2017

feedPlanet Ubuntu

Valorie Zimmerman: Artful Alpha 2 release candidate is ready for testing!


Test if you can, on real hardware if possible, in virtual machines if not.

Kubuntu Desktop amd64 testcases in Artful Daily:
http://iso.qa.ubuntu.com/qatracker/milestones/379/builds/153034/testcases

The label "Daily" will change once the Alpha 2 has been milestoned. Remember to report bugs and link them on the qa site. The easy way to file bugs is in the commandline: ubuntu-bug packagename. Ask us in IRC (#kubuntu-devel) if you need a package name.

Kubuntu Desktop i386 testcases in Artful Daily:
http://iso.qa.ubuntu.com/qatracker/milestones/379/builds/153035/testcases

It is so important to get this preliminary testing done if we want a 32-bit Alpha 2 ISO! We could not do an alpha 1 of Artful for this reason.


Thank you folks. That alpha 2 ISO will be spun on the 27th -- IF we have all tests done. If not.......

Valorie

PS: sorry for the late notice; I'm currently in Spain at KDE Akademy. You can subscribe to get the testing notices directly: http://iso.qa.ubuntu.com/qatracker/subscription

26 Jul 2017 3:56pm GMT

Adam Stokes: conjure-up dev summary highlights: screen ordering and improved deployment resilience

conjure-up dev summary highlights: screen ordering and improved deployment resilience

conjure-up dev summary highlights: screen ordering and improved deployment resilience conjure-up dev summary highlights: screen ordering and improved deployment resilience conjure-up dev summary highlights: screen ordering and improved deployment resilience

Screen Ordering

Previously, you would go through the install journey by picking your spell, selecting a cloud, modifying charm options, and then waiting for bootstrap/applications to be completely deployed prior to performing the post processing steps. In this latest update we've moved all steps to be completed prior to any of those longer blocking tasks.

This allows you to see the complete picture and giving you the ability to go back and make updates to your configuration items prior to running the longer tasks of the deployment.

Deployment resilience

We use a mechanism for determining when a deployment has "finished", meaning, all hooks have been fired and each application is in a ready state to handle the next set of instructions from the installer. In some cases an application may get into a state of error but quickly resolve itself.

This was causing us some issues as we were taking a very strict approach that if there is an error in the application then the deployment should fail. Even though this is the proper thing to do we've made a decision that if the application is able to fix itself then we shouldn't pass on that failed experience to the user since technically the deployment is able to finish and function as intended.

To combat this we've made our deployment checker a little more lenient and performing retries (up to 5 times) to validate if an application does fix itself. However, since an error was seen in the application it should not go unnoticed and needs to be fixed in the charm itself. In our integration tests we have an environment variable CONJURE_UP_MODE that can be set to test and gives us the ability to fail on charm failures and get those bugs reported upstream and resolved.

Spell authors can make use of this feature in their own spells by updating the 00_deploy-done script with the following bits:

#!/bin/bash
set -eux

. "$CONJURE_UP_SPELLSDIR/sdk/common.sh"

retry_arg="-r5"  
if [[ "${CONJURE_UP_MODE-}" == "test" ]]; then  
    retry_arg=""
fi

if ! juju wait $retry_arg -vwm "$JUJU_CONTROLLER:$JUJU_MODEL"; then  
    setResult "Applications did not start successfully"
    exit 1
fi

setResult "Applications Ready"  
exit 0  

Getting these changes

Installing the snap from the edge channel will give you the latest work outlined in these developer summaries:

sudo snap install conjure-up --classic --edge  

26 Jul 2017 1:47am GMT

25 Jul 2017

feedPlanet Ubuntu

Ubuntu Insights: ss: another way to get socket statistics

In an earlier blog post I mentioned ss, another tool that comes with the iproute2 package and allows you to query statistics about sockets. The same thing that can be done with netstat, with the added benefit that it is typically a little bit faster, and shorter to type.

Just ss by default will display much the same thing as netstat, and can be similarly passed options to limit the output to just what you want. For instance:

$ ss -t
State       Recv-Q Send-Q       Local Address:Port                        Peer Address:Port
ESTAB       0      0                127.0.0.1:postgresql                     127.0.0.1:48154
ESTAB       0      0            192.168.0.136:35296                      192.168.0.120:8009
ESTAB       0      0            192.168.0.136:47574                     173.194.74.189:https

[…]

ss -t shows just TCP connections. ss -u can be used to show UDP connections, -l will show only listening ports, and things can be further filtered to just the information you want.

I have not tested all the possible options, but you can even forcibly close sockets with -K.

One place where ss really shines though is in its filtering capabilities. Let's list all connections with a source port of 22 (ssh):

$ ss state all sport = :ssh
Netid State      Recv-Q Send-Q     Local Address:Port                      Peer Address:Port
tcp   LISTEN     0      128                    *:ssh                                  *:*
tcp   ESTAB      0      0          192.168.0.136:ssh                      192.168.0.102:46540
tcp   LISTEN     0      128                   :::ssh                                 :::*

And if I want to show only connected sockets (everything but listening or closed):

$ ss state connected sport = :ssh
Netid State      Recv-Q Send-Q     Local Address:Port                      Peer Address:Port
tcp   ESTAB      0      0          192.168.0.136:ssh                      192.168.0.102:46540

Similarly, you can have it list all connections to a specific host or range; in this case, using the 74.125.0.0/16 subnet, which apparently belongs to Google:

$ ss state all dst 74.125.0.0/16
Netid State      Recv-Q Send-Q     Local Address:Port                      Peer Address:Port
tcp   ESTAB      0      0          192.168.0.136:33616                   74.125.142.189:https
tcp   ESTAB      0      0          192.168.0.136:42034                    74.125.70.189:https
tcp   ESTAB      0      0          192.168.0.136:57408                   74.125.202.189:https

This is very much the same syntax as for iptables, so if you're familiar with that already, it will be quite easy to pick up. You can also install the iproute2-doc package, and look in /usr/share/doc/iproute2-doc/ss.html for the full documentation.

Try it for yourself! You'll see how well it works. If anything, I'm glad for the fewer characters this makes me type.

25 Jul 2017 6:44pm GMT

Sebastian Kgler: Plasma’s Vision

Plasma -- Durable, Usable, Elegant.Plasma - Durable, Usable, Elegant.Over the past weeks, we (KDE's Plasma team) have distilled the reasons why we do what we do, and what we want to achieve into a vision statement. In this article, I'd like to present Plasma's vision and explain a bit what is behind it. Let's start with the statement itself, though:

Plasma is a cross-device work environment by the KDE Community where trust is put on the user's capacity to best define her own workflow and preferences.

Plasma is simple by default, a clean work area for real-world usage which intends to stay out of your way.
Plasma is powerful when needed, enabling the user to create the workflow that makes her more effective to complete her tasks.

Plasma never dictates the user's needs, it only strives to solve them. Plasma never defines what the user is allowed to do, it only ensures that she can.

Our motivation is to enable actual work to happen, across devices, across different platforms, using any application needed.

We build to be durable, we create to be usable, we design to be elegant.

I've marked a few bits which are especially important in a bold font, let's get into a bit more detail:

Cross-device - Plasma is a work environment for different classes of devices, it adapts to the form-factor and offers a user interface which is suitable for the device's characteristics (input methods such as touchscreen, mouse, keyboard) and constraints (screen size, memory and CPU capabilties, etc.).

Define the workflow - Plasma is a flexible tool that can be set up as the user wishes and needs to make her more effective, to get the job done. Plasma is not a purpose in itself, it rather enables and gets out of the way. This isn't to say that we'll introduce every single option one can think of, but we strive to serve many users' use cases.

Simple by default means that Plasma is self-explanatory to new users, and that it offers a clean and sober interface in its default state. We don't want to overwhelm the user, but present a serene and friendly environment.

Powerful when needed on the other hand means that under the hood, Plasma offers tremendous power that allow to get almost any job done efficiently and without flailing.

We build to be durable, we create to be usable, we design to be elegant. - The end result is a stable workspace that the user can trust, that is friendly and easy to use, and that is beautiful and elegant in how it works.

25 Jul 2017 10:52am GMT

Didier Roche: Ubuntu Make as a classic snap: other distro versions and continuous delivery

This is a suite of blog posts explaining how we snapped Ubuntu Make which is a complex software study case with deep interactions with the system. For more background on this, please refer to our first blog post. For how we got here, have a look at the last blog post. Testing on 14.04 LTS Ok, we now have a working classic snap on 16.04 LTS. However, we saw that a word of caution needs to be taken care of due to the way classic snaps enable us to access the system.

25 Jul 2017 7:46am GMT

24 Jul 2017

feedPlanet Ubuntu

Ubuntu Insights: Ubuntu Desktop Weekly Update: July 24, 2017

GNOME

We're working on adding captive portal detection to Artful. We think it would be good if there were an option in the privacy settings to enable or disable this. We've done some initial work as preparation - some patches have been submitted to Network Manager's upstream to enable an option to be created. They are currently awaiting review.

ISO cleanups

Work continues to move deprecated components out of the desktop ISO and to universe. This week's list includes indicator-application and indicator-messages.

Snaps

Updated versions of gedit, gnome-sudoku, quadrapassel, gnome-dictionary, gnome-calculator, and gnome-clocks are in the edge channel of the store now built using the gnome-3-24 platform snap and content interface.

Video & Audio

A workaround has been uploaded for GDM which was blocking the A2DP high quality bluetooth profile from being activated in the user session.

Updates

Unity 7

16.04 continues to be supported, and we've been working on some further improvements for Unity that are targeted there. In particular there are improvements currently being developed to the low graphics mode that will benefit users of low powered systems and VMs.

QA

GNOME Software has a documented test plan now.

24 Jul 2017 7:09pm GMT

Robie Basak: Developing Ubuntu using git

git ubuntu illustration

Back in 2014, I published some information and tooling on using git for Ubuntu development, even though most Ubuntu development wasn't done with git at the time.

Three years on, this work has expanded significantly. Most of the server team is using git for daily work when touching Ubuntu packaging. We have expanded our tooling. With the significant interest we've received, we're now interested in developing this work to make git become the way of working with Ubuntu's source code. Our plan is to do this with no disruption to existing developer workflows.

This post is part of a blog series. Here's an index of all our planned posts. We'll amend this index and update links as we go.

Why is this so hard?

Most Free Software development projects already use git. So why has Ubuntu taken so long?

Unlike most software projects, Ubuntu (like other distributions) derives its sources from upstreams, rather than being the originator of the software. This means that we do not "control" the git repositories. Repository elements such as branch names, tag names, branching policies and so forth are not up to us; and nor is the choice to use git in the first place.

For git use in Ubuntu development to be effective, we need these repository elements to follow the same schemes across all our packages. But upstream projects use different schemes for their branches, tags and merge workflows. So our task isn't as trivial as just adopting upstreams' git trees.

Existing packaging repositories

While Ubuntu makes key changes to packages as needed, the long tail of packages that we ship are derived from Debian with no changes. Debian package maintainers use the VCS system of their choice, which may be nothing, git, or something else like Mercurial or Bazaar. These repositories are arranged in the manner of the package maintainers' choices. They may be derived from their upstreams' respective repositories, or they may instead be based on wholesale upstream code "import" commits done when the packaging is updated to a latest upstream release.

Right now, 68% of source packages in Ubuntu are listed as having their packaging maintained in git (whether in Debian or Ubuntu):

Note: the data I have used cannot tell us the difference between a package not being maintained in a VCS and the package maintainer not having declared in metadata that a particular VCS is in use.

Choices for Ubuntu and git

We're not in a position to mandate that everyone uses git. Even if we did do that for Ubuntu, we cannot expect mandate it in Debian and certainly not in every upstream project in our repositories.

One of the problems we want to solve is to be able to answer the question "where do I git clone from to get the Ubuntu source for package X?". We don't want to be forced to say "ah, but for package X, it's the same as Debian, and they're using Mercurial in this case, so you can't git clone you have to use hg clone from this other place", and then have the answer be different for package Y and different again for package Z. We know this will happen for 3 out of 10 packages. We want to eliminate all the edge cases so that, for git clone against any Ubuntu package, the repository structures are all consistent and all subsequent developer expectations always work.

To achieve this consistency, we need to find a way to use git for all Ubuntu packages: regardless of what VCS Debian or upstreams use for each package and project; and regardless of their different branching, tagging and merging models.

We think we've achieved this with our design; more on this in a future post.

Ubuntu, Bazaar, and UDD

Some readers may be familiar with a previous effort in Ubuntu, UDD, which was largely a similar effort but with Bazaar. Nine years later, git has largely won the "VCS wars", and appears to be preferred by the majority of developers. Our current effort could be seen as "UDD, but with git", if you wish.

Project goals

We'd like to avoid flag days and forced workflow changes. Ubuntu git integration will develop over time, but we don't expect Ubuntu developers to be forced to switch to it. We'd prefer for developers to choose to use our integration on its own merits, switching over if and when they feel it appropriate. If, after further consultation with users and Launchpad developers, we did switch to git as the primary source of truth from Launchpad, we expect to be able to wrap for backwards compatibility with dput.

Being central to all code, "moving to git" can be somewhat all-encompassing in terms of desirable use cases. Our original goal was very specific: to make what we call "Ubuntu package merges" easier. I achieved this back in 2014, and the server team has since made big improvements to this particular use case. Now we want to use git for much more, so this necessarily encompasses a wide range of use cases. We have accepted the following use cases as falling within the scope of our project:

For drive-by contributors and new developers

For routine Ubuntu development

For experienced Ubuntu developers

Current status

We have an importer running that automatically imports new source package publications into git, so the entire publication history of that package becomes available to git users. Until we're ready to scale this up, we're importing a subset of packages from a whitelist, with other packages imported on request for interested developers. You can also run the importer yourself locally on any package.

Tooling is available as an extension to git providing a set of subcommands (git ubuntu ...). The CLI is still experimental and subject to change, and we have a set of high-level subcommands planned that we have yet to write.

Experimental status

If you're interested, please do take a look! We'd appreciate feedback. However, note that we aren't "production ready" yet:

What would make a 1.0

  1. Launchpad shared object support.
  2. Hash stability declared.
  3. Developer UX issues fixed.
  4. Anything else? Please let us know what you think should be essential.

The wrapper

On our way, we hit a bunch of edge cases which may confuse developers. Some examples:

These edge cases can be worked around, often automatically, but this won't happen when a new developers use git clone directly.

To avoid having to introduce too much at once, we have written a wrapper that handles these edge cases automatically, or at least warns you about them on the occasions that they are relevant. There are also some common repetitive actions that are specific to our workflows; the wrapper also composes these for convenience to save developer time.

We don't want to mandate use of our wrapper. To better suit advanced developers, we've designed everything to be directly accessible without the wrapper, and we consider this method of access to be a first class citizen in our work. We'll talk more about the wrapper and its capabilities in a future post.

Next

In the next post, we'll cover details of where the imported repositories are and what they look like.

24 Jul 2017 4:29pm GMT

Jono Bacon: Video: Measuring Community Health

One of the most challenging components of building a community is how to (a) determine what to measure, (b) measure it effectively, and (c) interpret those measurements in a way that drives improvements.

Of course, what complicates this is that communities are a mixture of tangible metrics (things we can measure with a computer), and intangible (things such as "enablement", "happiness", "satisfaction" etc).

Here is a presentation I delivered recently that provides an overview of the topic and plenty of pragmatic guidance for how you put this into action:

If you can't see the video, click here.

The post Video: Measuring Community Health appeared first on Jono Bacon.

24 Jul 2017 3:46am GMT

22 Jul 2017

feedPlanet Ubuntu

Sebastian Kgler: Software from the Source

In this article, I am outlining an idea for an improved process of deploying software to Linux systems. It combined advantages of traditional, package mangement based systems with containerized software through systems such as Flatpak, Snap, or AppImage. An improved process allows us to make software deployment more efficient across the whole Free software community, have better supported software on users systems and allow for better quality at the same time.

Where we are goingWhere we are goingIn today's Linux and Free software ecosystems, users usually receive all their software from one source. It usually means that software is well integrated with the system, can be tested in combination with each other and support comes from a single vendor. Compared to systems, in which single software packages are downloaded from their individual vendors and then installed manually, this has huge advantages, as it makes it easy to get updates for everything installed on your system with a single command. The base system and the software comes from the same hands and can be tested as a whole. This ease of upgrading is almost mind-boggling to people who are used to a Windows world, where you'd download 20 .exe installer files post-OS-install and have to update them individually, a hugely time-consuming process and at time outright dangerous as software easily gets out of date.Traditional model of software deploymentTraditional model of software deploymentThere are also downsides to how we handle software deployment and installation currently, most of them revolve around update cycles. There is always a middle man who decides when and what to upgrade. This results in applications getting out of date, which is bad in reality and leads to a number of problems, as security and bug fixes are not making it to users in a timely fashion,

The value of downstreams

One argument that has been made is that downstreams do important work, too. An example for that legal or licensing problems are often found during reviews at SUSE, one of KDE's downstream partners. These are often fed back to KDE's developers where the problems can be fixed and be made part of upstream. This doesn't have to change at all, in fact, with a quicker deployment process, we're actually able to ship these fixes quicker to users. Likewise, QA that currently happens downstream should actually shift more to upstream so fixes get integrated and deployed quicker.

One big problem that we are currently facing is the variety of software stacks our downstreams use. An example that often bites us is that Linux distributions are combining applications with different versions of Qt. This is not only problematic on desktop form-factors, but has been a significant problem on mobile as well. Running an application against the same version of Qt that developers developed or tested it against means fewer bugs due to a smaller matrix of software stacks, resulting in less user-visible bugs.

In short: We'd be better off if work happening downstream happens more upstream, anyway.

Upstream as software distributor

Software directly from its sourceSoftware directly from its sourceSo, what's the idea? Let me explain what I have in mind. This is a bit of a radical idea, but given my above train of thoughts, it may well solve a whole range of problems that I've explained.

Linux distributors supply a base system, but most of the UI layers, so the user-visible parts come from downstream KDE (or other vendors, but let's assume KDE for now). The user gets to run a stable base that boots a system that supports all his hardware and gets updated according to the user's flavor, but the apps and relevant libraries come from upstream KDE, are maintained, tested and deployed from there. For many of the applications, the middle-man is cut out.

This leads to

Granted, for a large part of the user's system that stays relatively static, the current way of using packaged software works just fine. What I'm talking about are the bits and pieces that the users relies on for her productivity, the apps that are fast moving, where fixes are more critical to the user's productivity, or simply where the user wants to stay more up to date.

Containerization allows systemic improvements

In practice, this can be done by making containerized applications more easily available to the users. Discover, Plasma's software center, can allow the user to install software directly supplied by KDE and allow to keep it up to date. Users can pick where to get software from, but distros can make smart choices for users as well. Leaner distros could even entirely rely on KDE (or other upstreams) shipping applications and fully concentrate on the base system and advancing that part.

Luckily, containerization technologies now allow us to rethink how we supply users with our software and provide opportunities to let native apps on Linux systems catch up with much shorter deployment cycles and less variety in the stack, resulting in higher quality software on our users' systems.

22 Jul 2017 4:04pm GMT

21 Jul 2017

feedPlanet Ubuntu

Dustin Kirkland: Ubuntu 18.04 LTS Desktop Default Application Survey

Back in March, we asked the HackerNews community, "What do you want to see in Ubuntu 17.10?": https://ubu.one/AskHN

A passionate discussion ensued, the results of which are distilled into this post: http://ubu.one/thankHN

In fact, you can check that link, http://ubu.one/thankHN and see our progress so far this cycle. We already have a beta code in 17.10 available for your testing for several of those:

And several others have excellent work in progress, and will be complete by 17.10:

In summary -- your feedback matters! There are hundreds of engineers and designers working for *you* to continue making Ubuntu amazing!

Along with the switch from Unity to GNOME, we're also reviewing some of the desktop applications we package and ship in Ubuntu. We're looking to crowdsource input on your favorite Linux applications across a broad set of classic desktop functionality.

We invite you to contribute by listing the applications you find most useful in Linux in order of preference. To help us parse your input, please copy and paste the following bullets with your preferred apps in Linux desktop environments. You're welcome to suggest multiple apps, please just order them prioritized (e.g. Web Browser: Firefox, Chrome, Chromium). If some of your functionality has moved entirely to the web, please note that too (e.g. Email Client: Gmail web, Office Suite: Office360 web). If the software isn't free/open source, please note that (e.g. Music Player: Spotify client non-free). If I've missed a category, please add it in the same format. If your favorites aren't packaged for Ubuntu yet, please let us know, as we're creating hundreds of new snap packages for Ubuntu desktop applications, and we're keen to learn what key snaps we're missing.


In the interest of opening this survey as widely as possible, we've cross-posted this thread to HackerNews, Reddit, and Slashdot. We very much look forward to another friendly, energetic, collaborative discussion.

Or, you can fill out the survey here: https://ubu.one/apps1804

Thank you!
twitter.com/@DustinKirkland
On behalf of @Canonical and @Ubuntu

21 Jul 2017 10:05pm GMT

Jorge Castro: TLDRing your way to a Kubernetes Bare Metal cluster

Alex Ellis has an excellent tutorial on how to install Kubernetes in 10 minutes. It is a summarized version of what you can find in the official documentation. Read those first, this is a an even shorter version with my choices mixed in.

We'll install 16.04 on some machines, I'm using three. I just chose to use Weave instead of sending you to a choose your-own-network page as you have other stuff to learn before you dive into an opinion on a networking overlay. We're also in a lab environment so we assume some things like your machines are on the same network.

Prep the Operating System

First let's take care of the OS. I set up automatic updates, ensure the latest kernel is installed, and then ensure we're all up to date, whatever works for you:

sudo -s
dpkg-reconfigure unattended-upgrades
apt install linux-generic-hwe-16.04
apt update
apt dist-upgrade
reboot

Prep each node for Kubernetes:

This is just installing docker and adding the kubernetes repo, we'll be root for these steps:

sudo -s
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main  
EOF

apt update
apt install -qy docker.io kubelet kubeadm kubernetes-cni

On the master:

Pick a machine to be a master, then on that one:

kubeadm init

And then follow the directions to copy your config file to your user account, we only have a few commands left needed with sudo so you can safely exit out and continue with your user account:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Let's install the network, and then allow workloads to be scheduled on the master (for a lab we want to use all our hardware for workloads!):

kubectl apply -f https://git.io/weave-kube-1.6
kubectl taint nodes --all node-role.kubernetes.io/master-

On each worker node:

On each machine you want to be a worker (yours will be different, the output of kubeadm init will tell you what to do:

sudo kubeadm join --token 030b75.21ca2b9818ca75ef 192.168.1.202:6443 

You might need to tack on a --skip-preflight-checks, see #347, sorry for the inconvenience.

Ensuring your cluster works

It shouldn't take long for the nodes to come online, just check em out:

$ kubectl get nodes
NAME       STATUS     AGE       VERSION
dahl       Ready      45m       v1.7.1
hyperion   NotReady   16s       v1.7.1
tediore    Ready      32m       v1.7.1

$ kubectl cluster-info
Kubernetes master is running at https://192.168.1.202:6443
KubeDNS is running at https://192.168.1.202:6443/api/v1/namespaces/kube-system/services/kube-dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'
$

Ok you're cluster is rocking, now

Set up your laptop

I don't like to be ssh'ed into my cluster unless I'm doing maintenance, so now that we know stuff is working let's copy the kubernetes config from the master node to our local workstation. You should know how to copy files around systems already, but here's mine for reference:

 sudo scp /etc/kubernetes/admin.conf jorge@ivory.local:/home/jorge/.kube/config

I don't need the entire Kubernetes repo on my laptop, so we'll just install the snap for kubectl and check that I can access the server:

  sudo snap install kubectl --classic
  kubectl get nodes
  kubectl cluster-info

Don't forget to turn on autocompletion!

Deploy your first application

Let's deploy the Kubernetes dashboard:

   kubectl create -f https://git.io/kube-dashboard
   kubectl proxy

Then hit up http://localhost:8001/ui.

That's it, enjoy your new cluster!

Joining the Community

kubeadm is brought to you by SIG Cluster Lifecycle, they have regular meetings that anyone can attend, and you can give feedback on the mailing list. I'll see you there!

21 Jul 2017 11:00am GMT

Dustin Kirkland: ThankHN: A Thank-You Note to the HackerNews Community, from Ubuntu


A huge THANK YOU to the entire HackerNews community, from the Ubuntu community! Holy smokes...wow...you are an amazing bunch! Your feedback in the thread, "Ask HN: What do you want to see in Ubuntu 17.10?" is almost unbelievable!

We're truly humbled by your response.

I penned this thread, somewhat on a whim, from the Terminal 2 lounge at London Heathrow last Friday morning before flying home to Austin, Texas. I clicked "submit", closed my laptop, and boarded an 11-hour flight, wondering if I'd be apologizing to my boss and colleagues later in the day, for such a cowboy approach to Product Management...

When finally I signed onto the in-flight WiFi some 2 hours later, I saw this post at the coveted top position of HackerNews page 1, with a whopping 181 comments (1.5 comments per minute) in the first two hours. Impressively, it was only 6am on the US west coast by that point, so SFO/PDX/SEA weren't even awake yet. I was blown away!

This thread is now among the most discussed thread ever in the history of HackerNews, with some 1115 comments and counting at the time of this blog post.

 2530 comments   3125 points     2016-06-24      UK votes to leave EU    dmmalam
2215 comments 1817 points 2016-11-09 Donald Trump is the president-elect of the U.S. introvertmac
1448 comments 1330 points 2016-05-31 Moving Forward on Basic Income dwaxe
1322 comments 1280 points 2016-10-18 Shame on Y Combinator MattBearman
1215 comments 1905 points 2015-06-26 Same-Sex Marriage Is a Right, Supreme Court Rules imd23
1214 comments 1630 points 2016-12-05 Tell HN: Political Detox Week - No politics on HN for one week dang
1121 comments 1876 points 2016-01-27 Request For Research: Basic Income mattkrisiloff
*1115 comments 1333 points 2017-03-31 Ask HN: What do you want to see in Ubuntu 17.10? dustinkirkland
1090 comments 1493 points 2016-10-20 All Tesla Cars Being Produced Now Have Full Self-Driving Hardware impish19
1088 comments 2699 points 2017-03-07 CIA malware and hacking tools randomname2
1058 comments 1188 points 2014-03-16 Julie Ann Horvath Describes Sexism and Intimidation Behind Her GitHub Exit dkasper
1055 comments 2589 points 2017-02-28 Ask HN: Is S3 down? iamdeedubs
1046 comments 2123 points 2016-09-27 Making Humans a Multiplanetary Species [video] tilt
1030 comments 1558 points 2017-01-31 Welcome, ACLU katm
1013 comments 4107 points 2017-02-19 Reflecting on one very, very strange year at Uber grey-area
1008 comments 1990 points 2014-04-10 Drop Dropbox PhilipA


Rest assured that I have read every single one, and many of my colleagues has followed closely along as well.

In fact, to read and process this thread, I first attempted to print it out -- but cancelled the job before it fully buffered, when I realized that it's 105 pages long! Here's the PDF (1.6MB), if you're curious, or want to page through it on your e-reader.

So instead, I wrote the following Python script, using the HackerNews REST API, to download the thread from Google Firebase into a JSON document, and import into MongoDB, for item-by-item processing. Actually, this script will work against any HackerNews thread, and it recursively grabs nested comments. Next time you're asked to write a recursive function on a white board for a Google interview, hopefully you've remember this code! :-)

$ cat ~/bin/hackernews.py
#!/usr/bin/python3

import json
import requests
import sys

#https://hacker-news.firebaseio.com/v0/item/14002821.json?print=pretty

def get_json_from_url(item):
url = "https://hacker-news.firebaseio.com/v0/item/%s.json" % item
data = json.loads(requests.get(url=url).text)
#print(json.dumps(data, indent=4, sort_keys=True))
if "kids" in data and len(data["kids"]) > 0:
for k in data["kids"]:
data[k] = json.loads(get_json_from_url(k))
return json.dumps(data)


data = json.loads(get_json_from_url(sys.argv[1]))
print(json.dumps(data, indent=4, sort_keys=False))


It takes 5+ minutes to run, so you can just download a snapshot of the JSON blob from here (768KB), or if you prefer to run it yourself...

$ hackernews.py 14002821 | tee 14002821.json


First some raw statistics...

I'll try to summarize a few of my key interpretations of the trends, having now processed the entire discussion. Sincere apologies in advance if I've (a) misinterpreted a theme, (b) skipped your favorite them, or (c) conflated concepts. If any of these are the case, well, please post your feedback in the HackerNews thread associated with this post :-)

First, grouped below are some of the Desktop themes, with some fuzzy, approximate "weighting" by the number of pertinent discussions/mentions/vehemence.
  • Drop MIR/Unity for Wayland/Gnome (351 weight) [Beta available, 17.10]
    • Release/GA Unity 8 (15 weight)
    • Easily, the most heavily requested, major change in this thread was for Ubuntu to drop MIR/Unity in favor of Wayland/Gnome. And that's exactly what Mark Shuttleworth announced in an Ubuntu Insights post here today. There were a healthy handful of Unity 8 fans, calling for its GA, and more than a few HackerNews comments lamenting the end of Unity in this thread.
  • Improve HiDPI, 4K, display scaling, multi-monitor (217 weight) [Beta available, 17.10]
    • For the first time in a long time, I feel like a laggard in the technology space! I own a dozen or so digital displays but not a single 4K or HiDPI monitor. So while I can't yet directly relate, the HackerNews community is keen to see better support for multiple, high resolution monitors and world class display scaling. And I suspect you're just a short year or so ahead of much of the rest of the world.
  • Make track pad, touch gestures great (129 weight) [Beta available, 17.10]
    • There's certainly an opportunity to improve the track pad and touch gestures in the Ubuntu Desktop "more Apple-like".
  • Improve Bluetooth, WiFi, Wireless, Network Manager (97 weight) [Beta available, 17.10]
    • This item captures some broad, general requests to make Bluetooth and Wireless more reliable in Ubuntu. It's a little tough to capture an exact work item, but the relevant teams at Canonical have received the feedback.
  • Better mouse settings, more options, scroll acceleration (89 weight) [Beta available, 17.10]
    • Similar to the touch/track pad request, there was a collection of similar feedback suggesting better mouse settings out-of-the-box, and more fine grained options.
  • Better NVIDIA, GPU support (87 weight) [In-progress, 17.10]
    • NVIDIA GPUs are extensively used in both Ubuntu Desktops and Servers, and the feedback here was largely around better driver availability, more reliable upgrades, CUDA package access. For my part, I'm personally engaged with the high end GPU team at NVIDIA and we're actively working on a couple of initiatives to improve GPU support in Ubuntu (both Desktop and Server).
  • Clean up Network Manager, easier VPN (71 weight) [Beta available, 17.10]
    • There were several requests around both Network Manager, and a couple of excellent suggestions with respect to easier VPN configuration and connection. Given the recent legislation in the USA, I for one am fully supportive of helping Ubuntu users do more than ever before to protect their security and privacy, and that may entail better VPN support.
  • Easily customize, relocate the Unity launcher (53 weight) [Deprecated, 17.10]
    • This thread made it abundantly clear that it's important to people to be able to move, hide, resize, and customize their launcher (Unity or Gnome). I can certainly relate, as I personally prefer my launcher at the bottom of the screen.
  • Add night mode, redshift, f.lux (42 weight) [Beta available, 17.10]
    • This request is one of the real gems of this whole exercise! This seems like a nice, little, bite-sized feature, that we may be able include with minimal additional effort. Great find.
  • Make WINE and Windows apps work better (10 weight)
    • If Microsoft can make Ubuntu on Windows work so well, why can't Canonical make Windows on Ubuntu work? :-) If it were only so easy... For starters, the Windows Subsystem for Linux "simply" needs to implement a bunch of Linux syscalls, whose source is entirely available. So there's that :-) Anyway, this one is really going too be a tough one for us to move the needle on...
  • Better accessibility for disabled users, children (9 weight)
    • As a parent, and as a friend of many Ubuntu users with special needs, this is definitely a worthy cause. We'll continue to try and push the envelop on accessibility in the Linux desktop.
  • LDAP/ActiveDirectory integration out of the box (7 weight)
    • This is actually a regular request of Canonical's corporate Ubuntu Desktop customers. We're generally able to meet the needs of our enterprise customers around LDAP and ActiveDirectory authentication. We'll look at what else we can do natively in the distro to improve this.
  • Add support for voice commands (5 weight)
    • Excellent suggestion. We've grown so accustomed to "Okay Google...", "Alexa...", "Siri..." How long until we can, "Hey you, Ubuntu..." :-)
Grouped below are some themes, requests, and suggestions that generally apply to Ubuntu as an OS, or specifically as a cloud or server OS.
  • Better, easier, safer, faster, rolling upgrades (153 weight)
    • The ability to upgrade from one release of Ubuntu to the next has long been one of our most important features. A variety of requests have identified a few ways that we should endeavor to improve: snapshots and rollbacks, A/B image based updates, delta diffs, easier with fewer questions, super safe rolling updates to new releases. Several readers suggested killing off the non-LTS releases of Ubuntu and only releasing once a year, or every 2 years (which is the LTS status quo). We're working on a number of these, with much of that effort focused on Ubuntu Core. You'll see some major advances around this by Ubuntu 18.04 LTS.
  • Official hardware that just-works, Nexus-of-Ubuntu (130 weight)
    • This is perhaps my personal favorite suggestion of this entire thread -- for us to declare a "Nexus-of-each-Ubuntu-release", much like Google does for each major Android release. Hypothetically, this would be an easily accessible, available, affordable hardware platform, perhaps designed in conjunction with an OEM, to work perfectly with Ubuntu out of the box. That's a new concept. We do have the Ubuntu Hardware Certification Programme, where we clearly list all hardware that's tested and known to work well with Ubuntu. And we do work with major manufacturers on some fantastic desktops and laptops -- the Dell XPS and System76 both immediately come to mind. But this suggestion is a step beyond that. I'm set to speak to a few trusted partners about this idea in the coming weeks.
  • Lighter, smaller, more minimal (113 weight) [Beta Available, 17.10]
    • Add x-y-z-favorite-package to default install (105 weight)
    • For every Ubuntu user that wants to remove stuff from Ubuntu, to make it smaller/faster/lighter/secure, I'll show you another user who wants to add something else to the default install :-) This is a tricky one, and one that I'm always keen to keep an eye on. We try very had to strike a delicate balance between minimal-but-usable. When we have to err, we tend (usually, but not always) on the side of usability. That's just the Ubuntu way. That said, we're always evaluating our Ubuntu Server, Cloud, Container, and Docker images to insure that we minimize (or at least justify) any bloat. We'll certainly take another hard look at the default package sets at both 17.10 and 18.04. Thanks for bringing this up and we'll absolutely keep it in mind!
  • More QA, testing, stability, general polish (99 weight) [In-progress, 17.10]
    • The word "polish" is used a total of 24 times, with readers generally asking for more QA, more testing, more stability, and more "polish" to the Ubuntu experience. This is a tough one to quantify. That said, we have a strong commitment to quality, and CI/CD (continuous integration, continuous development) testing at Canonical. As your Product Manager, I'll do my part to ensure that we invest more resources into Ubuntu quality.
  • Fix /boot space, clean up old kernels (92 weight) [In-progress, 17.10]
    • Ouch. This is such an ugly, nasty problem. It personally pissed me off so much, in 2010, that I created a script, "purge-old-kernels". And it personally pissed me off again so much in 2014, that I jammed it into the Byobu package (which I also authored and maintain), for the sole reason to get it into Ubuntu. That being said, that's the wrong approach. I've spoken with Leann Ogasawara, the amazing manager and team lead for the Ubuntu kernel team, and she's committed to getting this problem solved once and for all in Ubuntu 17.10 -- and ideally getting those fixes backported to older releases of Ubuntu.
  • ZFS supported as a root filesystem (84 weight)
    • This was one of the more surprising requests I found here, and another real gem. I know that we have quite a few ZFS fans in the Ubuntu community (of which, I'm certainly one) -- but I had no idea so many people want to see ZFS as a root filesystem option. It makes sense to me -- integrity checking, compression, copy-on-write snapshots, clones. In fact, we have some skunkworks engineering investigating the possibility. Stay tuned...
  • Improve power management, battery usage (73 weight)
    • Longer batteries for laptops, lower energy bills for servers -- an important request. We'll need to work closer with our hardware OEM/ODM partners to ensure that we're leveraging their latest and greatest energy conservation features, and work with upstream to ensure those changes are integrated into the Linux kernel and Gnome.
  • Security hardening, grsecurity (72 weight)
    • More security! There were several requests for "extra security hardening" as an option, and the grsecurity kernel patch set. So the grsecurity Linux kernel is a heavily modified, patched Linux kernel that adds a ton of additional security checks and features at the lowest level of the OS. But the patch set is huge -- and it's not upstream in the Linux kernel. It also only applies against the last LTS release of Ubuntu. It would be difficult, though not necessarily impossible, to offer the grsecurity supported in the Ubuntu archive. As for "extra security hardening", Canonical is working with IBM on a number of security certification initiatives, around FIPS, CIS Benchmarks, and DISA STIG documentation. You'll see these becoming available throughout 2017.
  • Dump Systemd (69 weight)
    • Fun. All the people fighting for Wayland/Gnome, and here's a vocal minority pitching a variety of other init systems besides Systemd :-) So frankly, there's not much we can do about this one at this point. We created, and developed, and maintained Upstart over the course of a decade -- but for various reasons, Red Hat, SUSE, Debian, and most of the rest of the Linux community chose Systemd. We fought the good fight, but ultimately, we lost graciously, and migrated Ubuntu to Systemd.
  • Root disk encryption, ext4 encryption, more crypto (47 weight) [In-progress, 17.10]
    • The very first feature of Ubuntu, that I created when I started working for Canonical in 2008, was the Home Directory Encryption feature introduced in late 2008, so yes -- this feature has been near and dear to my heart! But as one of the co-authors and co-maintainers of eCryptfs, we're putting our support behind EXT4 Encryption for the future of per-file encryption in Ubuntu. Our good friends at Google (hi Mike, Ted, and co!) have created something super modern, efficient, and secure with EXT4 Encryption, and we hope to get there in Ubuntu over the next two releases. Root disk encryption is still important, even more now than ever before, and I do hope we can do a bit better to make root disk encryption easier to enable in the Desktop installer.
  • Fix suspend/resume (24 weight)
    • These were a somewhat general set of bugs or issues around suspend/resume not working as well as it should. If these are a closely grouped set of corner cases (e.g. multiple displays, particular hardware), then we should be able to shake these out with better QA, bug triage, and upstream fixes. That said, I remember when suspend/resume never worked at all in Linux, so pardon me while I'm a little nostalgic about how far we've come :-) Okay...now, yes, you're right. We should do better.
  • New server installer (19 weight) [Beta available, 17.10]
    • Well aren't you in for a surprise :-) There's a new server installer coming soon! Stay tuned.
  • Improve swap space management (12 weight)
    • Another pet peeve of mine -- I feel you! So I filed this blueprint in 2009, and I'm delighted to say that as of this month (8 years later), Ubuntu 17.04 (Zesty Zapus) will use swap files, rather than swap partitions, by default. Now, there's a bit more to do -- we should make these a bit more dynamic, tune the swappiness sysctl, etc. But this is a huge step in the right direction!
  • Reproducible builds (7 weight)
    • Ensuring that builds are reproducible is essential for the security and the integrity of our distribution. We've been working with Debian upstream on this over the last few years, and will continue to do so.
Ladies and gentlemen, again, a most sincere "thank you", from the Ubuntu community to the HackerNews community. We value openness -- open source code, open design, open feedback -- and this last week has been a real celebration of that openness for us. We appreciate the work and effort you put into your comments, and we hope to continue our dialog throughout our future together, and most importantly, that Ubuntu continues to serve your needs and occasionally even exceed your expectations ;-)

Cheers,
:-Dustin

21 Jul 2017 10:56am GMT

Jono Bacon: Clarification: Snappy and Flatpak

Recently, I posted a piece about distributions consolidating around a consistent app store. In it I mentioned Flatpak as a potential component and some people wondered why I didn't recommend Snappy, particularly due to my Canonical heritage.

To be clear (and to clear up my in-articulation): I am a fan of both Snappy and Flatpak: they are both important technologies solving important problems and they are both driven by great teams. To be frank, my main interest and focus in my post was the notion of a consolidated app store platform as opposed to what the specific individual components would be (other people can make a better judgement call on that). Thus, please don't read my single-line mention of Flatpak as any criticism of Snappy. I realize that this may have been misconstrued as me suggesting that Snappy is somehow not up to the job, which was absolutely not my intent.

Part of the reason I mentioned Flatpak is that I feel there is a natural center of gravity forming around the GNOME Shell and platform, which many distros are shipping. Within the context of that platform I have seen Flatpak commonly mentioned as a component, hence why I mentioned it. Of course, there is no reason why Snappy couldn't be that component too, and the Snappy team have been doing great work. I was also under the impression (entirely incorrectly) that Snappy is focusing more on the cloud/server market. It has become clear that the desktop is very much within the focus and domain of Snappy, and I apologize for the confusion.

So, to clear up any potential confusion (I can be an inarticulate clod at times), I am a big fan of Snappy, big fan of Flatpak, and an even bigger fan of a consolidated app store that multiple distros use.? My view is simple: competition is healthy, and we have two great projects and teams vying to make app installation and management on Linux easier. Viva la desktop!

The post Clarification: Snappy and Flatpak appeared first on Jono Bacon.

21 Jul 2017 12:26am GMT

20 Jul 2017

feedPlanet Ubuntu

The Fridge: Ubuntu 16.10 (Yakkety Yak) End of Life reached on July 20 2017

This is a follow-up to the End of Life warning sent earlier this month to confirm that as of today (July 20, 2017), Ubuntu 16.10 is no longer supported. No more package updates will be accepted to 16.10, and it will be archived to old-releases.ubuntu.com in the coming weeks.

The original End of Life warning follows, with upgrade instructions:

Ubuntu announced its 16.10 (Yakkety Yak) release almost 9 months ago, on October 13, 2016. As a non-LTS release, 16.10 has a 9-month support cycle and, as such, the support period is now nearing its end and Ubuntu 16.10 will reach end of life on Thursday, July 20th.

At that time, Ubuntu Security Notices will no longer include information or updated packages for Ubuntu 16.10.

The supported upgrade path from Ubuntu 16.10 is via Ubuntu 17.04. Instructions and caveats for the upgrade may be found at:

https://help.ubuntu.com/community/ZestyUpgrades

Ubuntu 17.04 continues to be actively supported with security updates and select high-impact bug fixes. Announcements of security updates for Ubuntu releases are sent to the ubuntu-security-announce mailing list, information about which may be found at:

https://lists.ubuntu.com/mailman/listinfo/ubuntu-security-announce

Since its launch in October 2004 Ubuntu has become one of the most highly regarded Linux distributions with millions of users in homes, schools, businesses and governments around the world. Ubuntu is Open Source software, costs nothing to download, and users are free to customise or alter their software in order to meet their needs.

Originally posted to the ubuntu-announce mailing list on Thu Jul 20 23:23:31 UTC 2017 by Adam Conrad, on behalf of the Ubuntu Release Team

20 Jul 2017 11:35pm GMT

Ubuntu Podcast from the UK LoCo: S10E20 – Wry Mindless Ice - Ubuntu Podcast

We discuss tormenting Mycroft, review the Dell Precision 5520, give you some USB resetting command line lurve and go over your feedback.

It's Season Ten Episode Twenty of the Ubuntu Podcast! Alan Pope, Mark Johnson and Martin Wimpress are connected and speaking to your brain.

In this week's show:

That's all for this week! If there's a topic you'd like us to discuss, or you have any feedback on previous shows, please send your comments and suggestions to show@ubuntupodcast.org or Tweet us or Comment on our Facebook page or comment on our Google+ page or comment on our sub-Reddit.

20 Jul 2017 2:00pm GMT

The Fridge: Ubuntu Weekly Newsletter Issue 513

Welcome to the Ubuntu Weekly Newsletter. This is issue #513 for the weeks of July 3 - 17, 2017, and the full version is available here.

In this issue we cover:

The issue of The Ubuntu Weekly Newsletter is brought to you by:

If you have a story idea for the Weekly Newsletter, join the Ubuntu News Team mailing list and submit it. Ideas can also be added to the wiki!

Except where otherwise noted, content in this issue is licensed under a Creative Commons Attribution 3.0 License BY SA Creative Commons License

20 Jul 2017 1:38pm GMT