Overview
The afternoon today has been absolutely filled with excellent talks. As with this morning, I’ve captured most of them for your entertainment and edification below. These blog entries are getting very long; perhaps I need an index.
Sessions
Fedora Workstation: Goals, Philosophy and Future
“The desktop is not dying!” cries Christian Schaller as he dives into his talk on the Fedora Workstation. In a throwback to the old saw about the Year of Linux on the Desktop, he points out that 35% of laptop sales are Chromebooks and that there are now over six hundred games available on Steam for Linux.
So what is the Fedora Workstation trying to achieve? What are its driving principles? First of all, Fedora Workstation is the only desktop out there that isn’t trying to sell you (in other words, capture information about you to sell to someone else). Fedora Workstation will consider end-user privacy one of its highest goal. By default, Workstation will not share your data.
The recent switch to focusing on delivering specific Fedora Products gave the Workstation an opportunity to start defining standards and maintaining them within a particular Product. Traditionally, Fedora was effectively a collection of exactly what upstreams provided. With the Workstation, we’ll be able to curate this code and integrate it into a cohesive whole (Author’s note: bingo!). One of the key points here is that by providing a known base, we give developers a better idea of what they can rely upon when writing their applications and services.
Christian continues on to discuss more about the future plans of the Workstation. The first of these was a discussion about how containers will fit into the Workstation world. Significant research is going into figuring out how to use container images to isolate and secure applications from one another, which will offer both flexibility and security in the long run. With some of the features offered by Wayland, this will eventually provide an excellent (and safe!) experience for users. Christian also notes that their goal is to make this all seamless from the users’ perspectives. If they notice it’s been done (without being told), they failed.
The Workstation Working group has been doing extensive research on Docker as a container implementation but has yet to make a decision on whether to standardize on Docker or else use a customized namespacing solution.
Christian then talks a bit about how Workstation is going to try to grow the platform. He acknowledges that there has been a slow decline of usage in Fedora and wants to put effort into drawing people back in. Some of the plans for this involve increased publicity through blogging and the Fedora Marketing groups. It will also involve finding ways to reduce the split between applications that run on Fedora and Red Hat Enterprise Linux. In part, that involves making sure that newer APIs are available in RHEL and older interfaces remain available in Fedora.
Another key piece of Fedora Workstation’s long-term strategy involves the creation and maintenance of a true ABI at all levels of the stack. This will help reduce the “moving target” aspect of Fedora and provide more guarantees about long-term sustainability of an application written for Fedora.
Christian takes a moment to talk about the target users. A lot of the conversation so far has been about making Fedora better for developers, but from his perspective it is also necessary to build a platform that’s useful to creators of all sorts. “Creators” may include video editors, 3D-modeling, music mixing and all sorts of other creation tasks.
After this, the conversation moved on to discuss a little bit of the challenges they face. Marketing and PR is going to be a significant challenge, particularly correcting some existing negative press that we’ve gathered over the last few years. We’ve also got to convince ourselves. There have been a lot of big changes in Fedora lately (the Three Product Plan being the most visible) and it’s clear that there’s going to be a period of adjustment as we course-correct and really figure out how we’re going to move forward.
Christian then talks about the advent of web-based applications and how that relates to the Workstation. He notes that there are users (even at Red Hat) that never open any local application other than their web browser. From this perspective, it’s very clear that we need Fedora Workstation to be a powerful mechanism for running web applications. So one of the things being worked on is ways that web applications can be tied into the desktop environment in a more integrated way.
Comments during and after the talk focused a great deal on the publicity and marketing aspects of things. It was noted that the more targeted user groups lends itself better to more controlled messaging. This will bring in more users in those particular groups who will in turn pass their good experience on by word-of-mouth.
Several questions were also asked about specific feature enhancements to things like Gnome Online Accounts (particularly around the earlier discussion of web applications). Christian indicated that the next immediate efforts on that front will likely be focused around a better calendaring experience.
Evolving the Fedora Updates Process
I next attended a talk about the Fedora update process given by Luke Macken, author and maintainer of the Bodhi update system.
Once upon a time in Fedora Core 1 through Fedora Core 3, updates were handled via a manual process involving emails to release engineering. Starting with Fedora Core 4, a private internal updating system that was available only to Red Hat employees.
The modern world of Bodhi began in Fedora 7 at the same time that Fedora Core and Fedora extras were merged. It introduced the concept of Karma and it was written in TurboGears 1.x and it is still in production today, seven years and many revisions later.
Bodhi does a lot of things behind the scenes, being both extremely intricate and very inefficient. Luke described a number of issues that have cropped up over the years, including inflexible SQL routines and the karma process outliving its usefulness.
Luke next took a little side-trip to tell us about some of the more entertaining glitches that have cropped up over the years, including the infamous Fedora 9 GPG re-keying and numerous crashes during update pushes.
After that, he moved on to discussing the plans for the Bodhi2 project. The plan is to have it land sometime after the Fedora 21 release. We don’t want to rely on it for zero-day updates, but we’ll phase it in soon after and it should hopefully be a graceful transition.
Some of the major changes in Bodhi2 will be a comprehensive REST API, a new and improved command-line tool and major changes to the UI that should provide a better experience for the users.
Another great feature of Bodhi2 is that it will integrate with fedmsg and the new Fedora Message Service to reduce the amount of “spam” email that Bodhi sends out. Luke dives in a bit to talk about the datagrepper and datanommer mining tools that power the notification service and the set of filters that you can opt into.
Luke showed off how Bodhi2 will be tightly integrated with Taskotron to perform automated testing on updates, as well as the integration with the Fedora Badges (there are lots of them available for Bodhi!) and then on to the feedback system. He called out both the fedora-easy-karma command-line tool and the fedora-gooey-karma GUI tool for managing karma updates on Bodhi1 (and noted that they will be working together to support Bodhi2 as well).
Then he went and left me slack-jawed with the new submitter process, automating almost everything and making it almost unbelievably simple. Adding to that, the new karma system allows the submitter to select fine-grained karma controls, so they can request that specific tests have to pass karma before accepting it into the stable repository.
The talk finished up with some prognosticating about the future, particularly talking about being able to run AMI and other cloud image updates through as well.
State of the Fedora Kernel
The next stop on my whirlwind tour of the wide world of Fedora was Josh Boyer’s annual discussion on Fedora’s treatment of the kernel. First up on the agenda was an overview of the release process. Fedora focuses on having a common base kernel across all stable releases (which means that bugs and features are shared). Fedora rebases to the latest upstream kernel on a regular basis, staggering the updates back to the two stable releases.
Josh described how the old process for kernel updates was to be more conservative on updates on older Fedora releases. However, a few years ago the process changed and updates are now faster and keeps Fedora much closer to the kernel upstream.
The talk then moved on to discussing the state of open bugs in the kernel. During this talk in 2013, there were 854 open bugs against the Fedora Kernel. After last year, the kernel maintainers sat down and spent a lot of time to knock down the bugs. Today it’s down to 533, but this is still not good enough. There will be a talk on Saturday about some ways to address this.
Josh pointed out several consistent problem areas: WiFI support, suspend/resume, video playback and brightness settings, platform-specific drivers (like Fn keys) and bluetooth. “All the stuff that involves ACPI is usually broken on some platform, somewhere”.
He then moved on to talking about how they handle bugs. He pointed out that if someone files a bug, they’re contacted (the bug is set NEEDINFO) every time the kernel is rebased. If after two weeks the reporter doesn’t confirm a fix (or a continuing bug), the bug is closed INSUFFICIENT_INFO.
What does all this mean? In short, it means that the Fedora kernel maintainers are permanently saturated, the work is never-ending and they would really appreciate if people would take vacations at different times than the maintainers so they don’t always return to 200+ extra bugs. Additionally, they really need help with triage, but it’s difficult to find anyone to do so, mainly because bug triage is admittedly boring. Some steps have been made in the last couple years that really helps, particularly the ABRT bug reports and the retrace server that helps narrow down which bugs are having the widest impact. The retrace server in particular keeps statistics on the number of reports, so that helps in prioritizing the fixing efforts.
After the bug discussion, Josh moved on to talking about the situation with Fedora 21. The plan is to release either kernel 3.16 or 3.17 at release, depending on schedule slips. During the Fedora 21 process, the kernel maintenance has actually been fairly calm, despite a set of new packaging changes.
During the Fedora.next process, the Fedora Cloud Working Group made requests of the kernel team to shrink down its size. There are a lot of optional components built into the kernel and many of these weren’t actually needed in a cloud environment. So the kernel team went and broke out the available modules into a core set and a common set above that. This made the minimal installation set much smaller and reduced the space on the cloud images substantially. In addition to this, they found ways to compress the kernel modules so the storage on disk shrank quite a bit as well.
Another useful feature that was added to packaging in Fedora 21 is support for automatically-generated RPM “Provides” listing the set of modules that are present in each of the packages. This will make it easier for packagers to specify dependencies on the appropriate package (and will continue working if modules move around).
The last major change in Fedora 21 is support for 64-bit ARM hardware (aarch64), which as was noted by an audience member is now available for general purchase. It works fairly well (thanks in large part to a herculean effort by Red Hat ARM engineers) and may be promoted to a primary architecture in Fedora 22 or 23. As a side-effect of this work, it’s going to be possible to replace the slow armv7hl builders in Koji with the new aarch64 builders that will be vastly more performant.
Josh then moved on to discuss the new kernel Playground, which is a new unsupported COPR containing some new experimental features. It tracks Fedora Rawhide and 21 and provides today the Overlayfs v23 (a union filesystem) and kdbus (the high-performance kernel D-BUS implementation). These are fairly stable patches to the kernel that are still out of the main tree and therefore not really suitable for Fedora proper (yet).
In the future, it may include other features such as kpatch and kgraft (the in-kernel infrastructure for supporting live-patching the kernel).
Advocating Fedora.next
After taking a short break to catch my breath and give my fingers a rest (these blog entries are a lot of work!), I went along to Christoph Wickert’s session Advocating Fedora.next. This session was largely (but not exclusively) directed at Fedora Ambassadors, informing them how best to talk about the Fedora.next initiative to the public.
He began his talk by addressing the question of “Why?”. Why did we need to change things so substantially? After providing the delightfully glib answer “Why not?”, Cristoph described a bit about Fedora’s history. He pointed out quite eloquently how Fedora has always been about change. Fedora has never been afraid to try something new to improve.
He then tackled some of the non-reasons behind Fedora.next, specifically the rumors of our demise post-GNOME 3 and similar. The truth is that we have a strong brand with a large contributor base that is extremely closely linked to upstream development (more so than many, if not all, of the other distributions). We’ve had a decade of successes and Fedora 20 has been very positively reviewed overall.
Another such rumor was that the new products are a benefit only to Red Hat. The obvious rebuttal here is that separating products makes good sense, focusing on specific user sets. Also, Red Hat has no particular interest in a consumer workstation product.
A final common criticism is that the new working groups are a power-grab by Red Hat. The older governance has not changed (and remains community-elected). Furthermore, all of the working groups were self-nominated and contains numerous non-Red Hat community members, including Christoph himself.
Christoph then ruminates, “If we didn’t do it for those reasons, why did we do it?”. The first answer is that distributions have become boring. All major distributions have declining market share. The general impression is that the distro is a commodity and that the users don’t care which one they’re running. The user really only cares that their applications run. Things like cloud environments and containers blur this line as well.
Continuing on, Christoph calls out to the Fedora Mission statement: “The Fedora Project’s mission is to lead the advancement of free and open source software and content as a collaborative community” and asks whether we feel like we’ve been living it. With that in mind, he defines Fedora.next for us: an umbrella term for the changes in the way that we create and release Fedora.
Fedora.next was born of two proposals that were first publicised at last year’s Flock conference: “The Architecture for a More Agile Fedora” by Matthew Miller and “The Fedora Crystal Ball: Where are we going for the next five years?” by yours truly, Stephen Gallagher. As the last year has passed, the Fedora.next that we now know has become a merge of these two proposals.
Matthew’s proposal involved having different policies depending on how low in the stack that a package lived, with core functionality having stricter guidelines than packages up in the application layer. My proposal was around having three different development streams (Server, Workstation and Cloud) and possibly different release cycles. The modern vision will be a combination of the two, with three products.
Christoph also warned the Ambassadors to be aware that the Fedora installation DVD will be retired in Fedora 21, but notes that it was never truly well-maintained and that its replacement netinstalls, spins and live media should be at least sufficient.
“What is a product?”, Christoph asks, then answers. A Product is more than simply a Fedora Spin. Each product has a defined target audience, a mission statement, a product requirements document (PRD) and a technical specification. The mission statement, PRD and technical statement were all defined and discussed publicly by the Product Working Groups and ratified by the Board and FESCo. Each product contains features not present in older Fedoras and has its own working group with their own governance models.
Christoph stresses that this is not a power-grab but instead the opposite: it’s an opportunity to give more power to the specific people who are building the Products. As a member of the Workstation Working Group, he calls out to its Mission Statement and then discusses a few of the Workstation-specific cool features. He notes that what Fedora Workstation will look like in Fedora 21 will not be a huge difference from the classic Fedora Live image, but this will change over time as the Workstation comes into its own life.
He then continues on to discuss Fedora Server a bit, calling out the exciting new Cockpit system management console.
Moving on to the Fedora Cloud, Christoph asks for Matthew Miller in the audience to comment further on it. Matthew describes the pets vs. cattle metaphor and explains that Fedora Cloud is really meant to fill the “cattle” side of that metaphor. Matthew notes the work towards the Fedora Base Image and the Fedora Atomic effort.
Christoph notes that this is an excellent example of why Spins are not enough. For example, when distributing cloud images, it doesn’t really meet the definition of a Spin because it doesn’t install via anaconda.
The talk then moves on to discussing the remaining two working groups: “Base Design” and the “Environments and Stacks” groups. Talking about the “Base Design”, he stresses that the idea is for the base to be as small as possible and provide a common framework for the Products to build on. The “Environments and Stacks” working groups are focused on making developers’ lives easier, providing the ability to install known software (development) stacks, possibly different versions side-by-side.
Christoph summarizes that there has been a great deal of misinformation put out there and he calls out to the Ambassadors and everyone else to explain what’s really going on, how it works and why it’s a positive change in Fedora. The message must be positive, because the change is exciting and there’s much more to come. He cautions “it’s not just ‘the next fedora’, it’s ‘Fedora.next’.”
Daily Summary
It’s really hard to pick out one specific thing to say about Flock’s first day. Every one of the speakers I listened to today were excited, engaging and clearly love Fedora, warts and all. I think I’ll leave it there for today.
Stephen Gallagher's Open-Source Blog 