airlied's Fedora aggregator

Dave Airlie: ripping the X server a new driver API

Mon, 14 May 2012 17:05:58 +0000

So I've been slowly writing the hotplug support v3 in between all the real jobs I have to do.

[side note: When I started out on hotplug. one of my goals was to avoid changing the server API/ABI too much so I could continue side by side testing.]

how did I get to v3?
v0.1: was called dynerama it failed miserably and proved that using Xinerama as the plugging layer was a bad plan.
v1: was the first time I decided to use an impedance layer between some server objects and driver objects.
v2: was the a major rebase of v1.

v2 was trucking along nicely and I managed to get the design to the stage where PRIME offloading intel/nouveau worked, USB device hotplug with udl worked, and GPU switch worked between two drivers. However v2 duplicated a lot of code and invented a whole new set of API objects called DrvXRec, so DrvScreenRec, DrvPixmapRec, DrvGCRec etc, this lead me to looking at the pain of merging this into the drivers and the server, and my goals of avoiding changing the API/ABI was getting in my way.

So before starting v3 I decided to rework some of the server "APIs".

The X server has two main bodies of code, one called DIX, and one called DDX. The DIX (device independent X) code and the DDX (Device dependent X code). In the X.org tree the dix lives up in the top level dirs, and for X.org server the DDX lives in hw/xfree86. The main object with info about protocol screens and GPUs is called ScreenRec in the DDX and ScrnInfoRec in the DIX. These are stored in two arrays, screenInfo.screens in the DIX and xf86Screens in the DDX, when code wants to convert between these it can do one of a few things.

a) lookup by index, both structs have an index value, so to go from ScrnInfo to Screen you look at screenInfo.screens[scrninfo->scrnIndex] and other way is xf86Screens[screen->myNum]. This is like the I didn't try and make an API, I just exposed everything.

b) ScrnInfo has a ScreenPtr in it, so some code can do ScrnInfo->pScreen to get the pointer to the dix struct. But this pointer is initialised after a bunch of code is called, so you really can't guarantee this pointer is going to be useful for you.

c) XF86SCRNINFO uses the DIX private subsystem to lookup the Scrn in the Screen's privates. This is the least used and probably slowest method.

So also screenInfo.screens contains the protocol screens we exposed to the clients, so this array cannot really change or move around. So I'd like to add screeninfo.gpuscreens and xf86GPUScreens and not have drivers know which set of info they are working on, however (a) totally screws this idea, since the indices are always looked up directly in the global arrays.

Now lots of the Screen/ScrnInfo APIs exposed to the drivers pass an int index as the first parameter, the function in the driver then goes and looks up the global arrays.

So my first API changes introduce some standard conversion functions xf86ScreenToScrn and xf86ScrnToScreen, and converts a lot of the server to use those. Yay an API. The second set of changes then changes all of the index passing APIs to pass ScrnInfoPtr or ScreenPtr, so the drivers don't go poking into global arrays. Now this is a major API change, it will involve slightly messy code in drivers that want to work with both servers, but I can't see a nicer way to do it. I've done a compat header file that will hopefully allows to cover a lot of this stuff where we don't see it.

I've ono other API introduction on the list, Glyph Pictures are another global array indexed by screen index, I've gone and added an accessor function so that drivers don't use the index anymore to get at the array contents directly.

Once this stuff lands in the server, a team of people will go forward and port the drivers to the new APIs (who am I kidding).

Dave Jones: Weekly Fedora kernel bug statistics – May 11th 2012

Fri, 11 May 2012 16:42:50 +0000

	15	16	17	rawhide
Open:	311	583	76	141	(1111)
Opened since 2012-05-04	3	37	14	0	(54)
Closed since 2012-05-04	3	43	12	1	(59)
Changed since 2012-05-04	10	61	15	4	(90)

F16 bug count is out of control at this point. We’re still managing to close them faster than they’re coming in, but the sheer number of bugs still open is overwhelming. There still exist a lot of older bugs that may have been fixed but the user hasn’t updated the bug. As we’re coming across these bugs that have been in needinfo state for a while we’re closing them out if we have a reasonable expectation that the bug is fixed.

There’s also still a huge number of duplicate bugs, where it’s not immediately obvious that they are duplicates. The end frame in the call chain may be the same, but how we got there differs for example. Or we see multiple instances of a particular trace, from various hardware. (A good example of this case being the numerous irqpoll bugs open for which we still have no real explanation. Likewise softlockup reports).

A curious observation: We’ve had a slight uptick lately in the number of bugs that seem to be caused by bad memory or other hardware problems. Sometimes these stick out like a sore thumb (a single bit flips changing an address from ffffffff81c8bca0 to ffefffff81c8bca0 for example). Other times, they aren’t as obvious.
I have been wondering though if we see more of these in Fedora just due to the extra debug options we leave enabled even in the production builds (like linked list debug, which is pretty much free).

Something that has been interesting though when we’ve found a user with a bad ram report, is searching for other bugs that user filed. Suddenly “weird” oopses make a lot more sense.

Weekly Fedora kernel bug statistics – May 11th 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – April 20 2012 15 16 17 rawhide Open: 308 544 64...

Related posts brought to you by Yet Another Related Posts Plugin.

Dave Jones: Weekly Fedora kernel bug statistics – May 4th 2012

Fri, 04 May 2012 16:16:48 +0000

	15	16	17	rawhide
Open:	311	580	78	142	(1111)
Opened since 2012-04-27	2	36	13	6	(57)
Closed since 2012-04-27	2	20	7	11	(40)
Changed since 2012-04-27	9	61	13	11	(94)

Weekly Fedora kernel bug statistics – May 4th 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – April 20 2012 15 16 17 rawhide Open: 308 544 64...

Related posts brought to you by Yet Another Related Posts Plugin.

Lennart Poettering: Boot & Base OS Miniconf at Linux Plumbers Conference 2012, San Diego

Thu, 03 May 2012 18:42:00 +0000

We are working on putting together a miniconf on the topic of Boot & Base OS for the Linux Plumbers Conference 2012 in San Diego (Aug 29-31). And we need your submission!

Are you working on some exciting project related to Boot and Base OS and would like to present your work? Then please submit something following these guidelines, but please CC Kay Sievers and Lennart Poettering.

I hope that at this point the Linux Plumbers Conference needs little introduction, so I will spare any further prose on how great and useful and the best conference ever it is for everybody who works on the plumbing layer of Linux. However, there's one conference that will be co-located with LPC that is still little known, because it happens for the first time: The C Conference, organized by Brandon Philips and friends. It covers all things C, and they are still looking for more topics, in a reverse CFP. Please consider submitting a proposal and registering to the conference!

Lennart Poettering: The Most Awesome, Least-Advertised Fedora 17 Feature

Tue, 01 May 2012 21:07:00 +0000

There's one feature In the upcoming Fedora 17 release that is immensly useful but very little known, since its feature page 'ckremoval' does not explicitly refer to it in its name: true automatic multi-seat support for Linux.

A multi-seat computer is a system that offers not only one local seat for a user, but multiple, at the same time. A seat refers to a combination of a screen, a set of input devices (such as mice and keyboards), and maybe an audio card or webcam, as individual local workplace for a user. A multi-seat computer can drive an entire class room of seats with only a fraction of the cost in hardware, energy, administration and space: you only have one PC, which usually has way enough CPU power to drive 10 or more workplaces. (In fact, even a Netbook has fast enough to drive a couple of seats!) Automatic multi-seat refers to an entirely automatically managed seat setup: whenever a new seat is plugged in a new login screen immediately appears -- without any manual configuration --, and when the seat is unplugged all user sessions on it are removed without delay.

In Fedora 17 we added this functionality to the low-level user and device tracking of systemd, replacing the previous ConsoleKit logic that lacked support for automatic multi-seat. With all the ground work done in systemd, udev and the other components of our plumbing layer the last remaining bits were surprisingly easy to add.

Currently, the automatic multi-seat logic works best with the USB multi-seat hardware from Plugable you can buy cheaply on Amazon (US). These devices require exactly zero configuration with the new scheme implemented in Fedora 17: just plug them in at any time, login screens pop up on them, and you have your additional seats. Alternatively you can also assemble your seat manually with a few easy loginctl attach commands, from any kind of hardware you might have lying around. To get a full seat you need multiple graphics cards, keyboards and mice: one set for each seat. (Later on we'll probably have a graphical setup utility for additional seats, but that's not a pressing issue we believe, as the plug-n-play multi-seat support with the Plugable devices is so awesomely nice.)

Plugable provided us for free with hardware for testing multi-seat. They are also involved with the upstream development of the USB DisplayLink driver for Linux. Due to their positive involvement with Linux we can only recommend to buy their hardware. They are good guys, and support Free Software the way all hardware vendors should! (And besides that, their hardware is also nicely put together. For example, in contrast to most similar vendors they actually assign proper vendor/product IDs to their USB hardware so that we can easily recognize their hardware when plugged in to set up automatic seats.)

Currently, all this magic is only implemented in the GNOME stack with the biggest component getting updated being the GNOME Display Manager. On the Plugable USB hardware you get a full GNOME Shell session with all the usual graphical gimmicks, the same way as on any other hardware. (Yes, GNOME 3 works perfectly fine on simpler graphics cards such as these USB devices!) If you are hacking on a different desktop environment, or on a different display manager, please have a look at the multi-seat documentation we put together, and particularly at our short piece about writing display managers which are multi-seat capable.

If you work on a major desktop environment or display manager and would like to implement multi-seat support for it, but lack the aforementioned Plugable hardware, we might be able to provide you with the hardware for free. Please contact us directly, and we might be able to send you a device. Note that we don't have unlimited devices available, hence we'll probably not be able to pass hardware to everybody who asks, and we will pass the hardware preferably to people who work on well-known software or otherwise have contributed good code to the community already. Anyway, if in doubt, ping us, and explain to us why you should get the hardware, and we'll consider you! (Oh, and this not only applies to display managers, if you hack on some other software where multi-seat awareness would be truly useful, then don't hesitate and ping us!)

Phoronix has this story about this new multi-seat support which is quite interesting and full of pictures. Please have a look.

Plugable started a Pledge drive to lower the price of the Plugable USB multi-seat terminals further. It's full of pictures (and a video showing all this in action!), and uses the code we now make available in Fedora 17 as base. Please consider pledging a few bucks.

Recently David Zeuthen added multi-seat support to udisks as well. With this in place, a user logged in on a specific seat can only see the USB storage plugged into his individual seat, but does not see any USB storage plugged into any other local seat. With this in place we closed the last missing bit of multi-seat support in our desktop stack.

With this code in Fedora 17 we cover the big use cases of multi-seat already: internet cafes, class rooms and similar installations can provide PC workplaces cheaply and easily without any manual configuration. Later on we want to build on this and make this useful for different uses too: for example, the ability to get a login screen as easily as plugging in a USB connector makes this not useful only for saving money in setups for many people, but also in embedded environments (consider monitoring/debugging screens made available via this hotplug logic) or servers (get trivially quick local access to your otherwise head-less server). To be truly useful in these areas we need one more thing though: the ability to run a simply getty (i.e. text login) on the seat, without necessarily involving a graphical UI.

The well-known X successor Wayland already comes out of the box with multi-seat support based on this logic.

Oh, and BTW, as Ubuntu appears to be "focussing" on "clarity" in the "cloud" now ;-), and chose Upstart instead of systemd, this feature won't be available in Ubuntu any time soon. That's (one detail of) the price Ubuntu has to pay for choosing to maintain it's own (largely legacy, such as ConsoleKit) plumbing stack.

Multi-seat has a long history on Unix. Since the earliest days Unix systems could be accessed by multiple local terminals at the same time. Since then local terminal support (and hence multi-seat) gradually moved out of view in computing. The fewest machines these days have more than one seat, the concept of terminals survived almost exclusively in the context of PTYs (i.e. fully virtualized API objects, disconnected from any real hardware seat) and VCs (i.e. a single virtualized local seat), but almost not in any other way (well, server setups still use serial terminals for emergency remote access, but they almost never have more than one serial terminal). All what we do in systemd is based on the ideas originally brought forward in Unix; with systemd we now try to bring back a number of the good ideas of Unix that since the old times were lost on the roadside. For example, in true Unix style we already started to expose the concept of a service in the file system (in /sys/fs/cgroup/systemd/system/), something where on Linux the (often misunderstood) "everything is a file" mantra previously fell short. With automatic multi-seat support we bring back support for terminals, but updated with all the features of today's desktops: plug and play, zero configuration, full graphics, and not limited to input devices and screens, but extending to all kinds of devices, such as audio, webcams or USB memory sticks.

Anyway, this is all for now; I'd like to thank everybody who was involved with making multi-seat work so nicely and natively on the Linux platform. You know who you are! Thanks a ton!

Dave Jones: Weekly Fedora kernel bug statistics – April 27 2012

Fri, 27 Apr 2012 18:17:54 +0000

	15	16	17	rawhide
Open:	308	562	71	147	(1088)
Opened since 2012-04-20	0	44	17	3	(64)
Closed since 2012-04-20	3	55	11	1	(70)
Changed since 2012-04-20	10	69	18	9	(106)

Weekly Fedora kernel bug statistics – April 27 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – April 13 2012 15 16 17 rawhide Open: 313 549 63...
Weekly Fedora kernel bug statistics – April 20 2012 15 16 17 rawhide Open: 308 544 64...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...

Related posts brought to you by Yet Another Related Posts Plugin.

Lennart Poettering: systemd Status Update

Fri, 20 Apr 2012 22:17:00 +0000

It has been way too long since my last status update on systemd. Here's another short, incomprehensive status update on what we worked on for systemd since then.

We have been working hard to turn systemd into the most viable set of components to build operating systems, appliances and devices from, and make it the best choice for servers, for desktops and for embedded environments alike. I think we have a really convincing set of features now, but we are actively working on making it even better.

Here's a list of some more and some less interesting features, in no particular order:

We added an automatic pager to systemctl (and related tools), similar to how git has it.
systemctl learnt a new switch --failed, to show only failed services.
You may now start services immediately, overrding all dependency logic by passing --ignore-dependencies to systemctl. This is mostly a debugging tool and nothing people should use in real life.
Sending SIGKILL as final part of the implicit shutdown logic of services is now optional and may be configured with the SendSIGKILL= option individually for each service.
We split off the Vala/Gtk tools into its own project systemd-ui.
systemd-tmpfiles learnt file globbing and creating FIFO special files as well as character and block device nodes, and symlinks. It also is capable of relabelling certain directories at boot now (in the SELinux sense).
Immediately before shuttding dow we will now invoke all binaries found in /lib/systemd/system-shutdown/, which is useful for debugging late shutdown.
You may now globally control where STDOUT/STDERR of services goes (unless individual service configuration overrides it).
There's a new ConditionVirtualization= option, that makes systemd skip a specific service if a certain virtualization technology is found or not found. Similar, we now have a new option to detect whether a certain security technology (such as SELinux) is available, called ConditionSecurity=. There's also ConditionCapability= to check whether a certain process capability is in the capability bounding set of the system. There's also a new ConditionFileIsExecutable=, ConditionPathIsMountPoint=, ConditionPathIsReadWrite=, ConditionPathIsSymbolicLink=.
The file system condition directives now support globbing.
Service conditions may now be "triggering" and "mandatory", meaning that they can be a necessary requirement to hold for a service to start, or simply one trigger among many.
At boot time we now print warnings if: /usr is on a split-off partition but not already mounted by an initrd; if /etc/mtab is not a symlink to /proc/mounts; CONFIG_CGROUPS is not enabled in the kernel. We'll also expose this as tainted flag on the bus.
You may now boot the same OS image on a bare metal machine and in Linux namespace containers and will get a clean boot in both cases. This is more complicated than it sounds since device management with udev or write access to /sys, /proc/sys or things like /dev/kmsg is not available in a container. This makes systemd a first-class choice for managing thin container setups. This is all tested with systemd's own systemd-nspawn tool but should work fine in LXC setups, too. Basically this means that you do not have to adjust your OS manually to make it work in a container environment, but will just work out of the box. It also makes it easier to convert real systems into containers.
We now automatically spawn gettys on HVC ttys when booting in VMs.
We introduced /etc/machine-id as a generalization of D-Bus machine ID logic. See this blog story for more information. On stateless/read-only systems the machine ID is initialized randomly at boot. In virtualized environments it may be passed in from the machine manager (with qemu's -uuid switch, or via the container interface).
All of the systemd-specific /etc/fstab mount options are now in the x-systemd-xyz format.
To make it easy to find non-converted services we will now implicitly prefix all LSB and SysV init script descriptions with the strings "LSB:" resp. "SYSV:".
We introduced /run and made it a hard dependency of systemd. This directory is now widely accepted and implemented on all relevant Linux distributions.
systemctl can now execute all its operations remotely too (-H switch).
We now ship systemd-nspawn, a really powerful tool that can be used to start containers for debugging, building and testing, much like chroot(1). It is useful to just get a shell inside a build tree, but is good enough to boot up a full system in it, too.
If we query the user for a hard disk password at boot he may hit TAB to hide the asterisks we normally show for each key that is entered, for extra paranoia.
We don't enable udev-settle.service anymore, which is only required for certain legacy software that still hasn't been updated to follow devices coming and going cleanly.
We now include a tool that can plot boot speed graphs, similar to bootchartd, called systemd-analyze.
At boot, we now initialize the kernel's binfmt_misc logic with the data from /etc/binfmt.d.
systemctl now recognizes if it is run in a chroot() environment and will work accordingly (i.e. apply changes to the tree it is run in, instead of talking to the actual PID 1 for this). It also has a new --root= switch to work on an OS tree from outside of it.
There's a new unit dependency type OnFailureIsolate= that allows entering a different target whenever a certain unit fails. For example, this is interesting to enter emergency mode if file system checks of crucial file systems failed.
Socket units may now listen on Netlink sockets, special files from /proc and POSIX message queues, too.
There's a new IgnoreOnIsolate= flag which may be used to ensure certain units are left untouched by isolation requests. There's a new IgnoreOnSnapshot= flag which may be used to exclude certain units from snapshot units when they are created.
There's now small mechanism services for changing the local hostname and other host meta data, changing the system locale and console settings and the system clock.
We now limit the capability bounding set for a number of our internal services by default.
Plymouth may now be disabled globally with plymouth.enable=0 on the kernel command line.
We now disallocate VTs when a getty finished running (and optionally other tools run on VTs). This adds extra security since it clears up the scrollback buffer so that subsequent users cannot get access to a user's session output.
In socket units there are now options to control the IP_TRANSPARENT, SO_BROADCAST, SO_PASSCRED, SO_PASSSEC socket options.
The receive and send buffers of socket units may now be set larger than the default system settings if needed by using SO_{RCV,SND}BUFFORCE.
We now set the hardware timezone as one of the first things in PID 1, in order to avoid time jumps during normal userspace operation, and to guarantee sensible times on all generated logs. We also no longer save the system clock to the RTC on shutdown, assuming that this is done by the clock control tool when the user modifies the time, or automatically by the kernel if NTP is enabled.
The SELinux directory got moved from /selinux to /sys/fs/selinux.
We added a small service systemd-logind that keeps tracks of logged in users and their sessions. It creates control groups for them, implements the XDG_RUNTIME_DIR specification for them, maintains seats and device node ACLs and implements shutdown/idle inhibiting for clients. It auto-spawns gettys on all local VTs when the user switches to them (instead of starting six of them unconditionally), thus reducing the resource foot print by default. It has a D-Bus interface as well as a simple synchronous library interface. This mechanism obsoletes ConsoleKit which is now deprecated and should no longer be used.
There's now full, automatic multi-seat support, and this is enabled in GNOME 3.4. Just by pluging in new seat hardware you get a new login screen on your seat's screen.
There is now an option ControlGroupModify= to allow services to change the properties of their control groups dynamically, and one to make control groups persistent in the tree (ControlGroupPersistent=) so that they can be created and maintained by external tools.
We now jump back into the initrd in shutdown, so that it can detach the root file system and the storage devices backing it. This allows (for the first time!) to reliably undo complex storage setups on shutdown and leave them in a clean state.
systemctl now supports presets, a way for distributions and administrators to define their own policies on whether services should be enabled or disabled by default on package installation.
systemctl now has high-level verbs for masking/unmasking units. There's also a new command (systemctl list-unit-files) for determining the list of all installed unit file files and whether they are enabled or not.
We now apply sysctl variables to each new network device, as it appears. This makes /etc/sysctl.d compatible with hot-plug network devices.
There's limited profiling for SELinux start-up perfomance built into PID 1.
There's a new switch PrivateNetwork= to turn of any network access for a specific service.
Service units may now include configuration for control group parameters. A few (such as MemoryLimit=) are exposed with high-level options, and all others are available via the generic ControlGroupAttribute= setting.
There's now the option to mount certain cgroup controllers jointly at boot. We do this now for cpu and cpuacct by default.
We added the journal and turned it on by default.
All service output is now written to the Journal by default, regardless whether it is sent via syslog or simply written to stdout/stderr. Both message streams end up in the same location and are interleaved the way they should. All log messages even from the kernel and from early boot end up in the journal. Now, no service output gets unnoticed and is saved and indexed at the same location.
systemctl status will now show the last 10 log lines for each service, directly from the journal.
We now show the progress of fsck at boot on the console, again. We also show the much loved colorful [ OK ] status messages at boot again, as known from most SysV implementations.
We merged udev into systemd.
We implemented and documented interfaces to container managers and initrds for passing execution data to systemd. We also implemented and documented an interface for storage daemons that are required to back the root file system.
There are two new options in service files to propagate reload requests between several units.
systemd-cgls won't show kernel threads by default anymore, or show empty control groups.
We added a new tool systemd-cgtop that shows resource usage of whole services in a top(1) like fasion.
systemd may now supervise services in watchdog style. If enabled for a service the daemon daemon has to ping PID 1 in regular intervals or is otherwise considered failed (which might then result in restarting it, or even rebooting the machine, as configured). Also, PID 1 is capable of pinging a hardware watchdog. Putting this together, the hardware watchdogs PID 1 and PID 1 then watchdogs specific services. This is highly useful for high-availability servers as well as embedded machines. Since watchdog hardware is noawadays built into all modern chipsets (including desktop chipsets), this should hopefully help to make this a more widely used functionality.
We added support for a new kernel command line option systemd.setenv= to set an environment variable system-wide.
By default services which are started by systemd will have SIGPIPE set to ignored. The Unix SIGPIPE logic is used to reliably implement shell pipelines and when left enabled in services is usually just a source of bugs and problems.
You may now configure the rate limiting that is applied to restarts of specific services. Previously the rate limiting parameters were hard-coded (similar to SysV).
There's now support for loading the IMA integrity policy into the kernel early in PID 1, similar to how we already did it with the SELinux policy.
There's now an official API to schedule and query scheduled shutdowns.
We changed the license from GPL2+ to LGPL2.1+.
We made systemd-detect-virt an official tool in the tool set. Since we already had code to detect certain VM and container environments we now added an official tool for administrators to make use of in shell scripts and suchlike.
We documented numerous interfaces systemd introduced.

Much of the stuff above is already available in Fedora 15 and 16, or will be made available in the upcoming Fedora 17.

And that's it for now. There's a lot of other stuff in the git commits, but most of it is smaller and I will it thus spare you.

I'd like to thank everybody who contributed to systemd over the past years.

Thanks for your interest!

Dave Jones: Weekly Fedora kernel bug statistics – April 20 2012

Fri, 20 Apr 2012 16:41:17 +0000

	15	16	17	rawhide
Open:	308	544	64	146	(1062)
Opened since 2012-04-13	1	48	12	3	(64)
Closed since 2012-04-13	7	58	11	0	(76)
Changed since 2012-04-13	13	66	15	8	(102)

Some focus this week on closing out a bunch of old bugs, especially those suspected to have been caused by the now fixed i915 memory corruption bug.

Weekly Fedora kernel bug statistics – April 20 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – April 13 2012 15 16 17 rawhide Open: 313 549 63...
Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...

Related posts brought to you by Yet Another Related Posts Plugin.

Jeremy Katz: A Puppet User Trying Chef

Thu, 19 Apr 2012 15:50:18 +0000

I have a decent amount of experience at this point with puppet both from experience using it to manage the infrastructure running Fedora as well as setting it up at a pretty large scale at HubSpot. But in a new gig, I decided it was worth rounding myself out a bit and giving chef a try. Not out of any deep seated dislike of puppet but there are a few pieces that I’ve continued to run up against which are a little grating and so I figured it was worth broadening my horizons. The nice thing is that both are fairly successful open source communities and realistically, as long as you’re using a system, you probably can’t go that wrong or switch in the future.

Side-note: I’ve also been playing with Michael Dehaan’s new project, ansible which is also interesting. But I don’t think it’s mature enough to use for a production environment yet and I also was mostly interested in it as a better remote execution layer as opposed to another full fledged config management tool. But yeah. It’s there. It’s interesting. I’ll probably write more about it later.

With a little bit of chef time under my belt, I have to say that I’m not struck by drastic differences. The terminologies are different, the DSL used on the config side is a bit different but they act pretty similarly and you can get either of them to do what you want. That said, there are a few things (good and bad) that I’ve noticed about chef and figured I’d share for others who are looking at deciding for themselves. Note that a few of the things in the dislikes section may well just be me missing something and being a n00b… suggestions welcome!

Things I’ve Liked

Hosted Chef is a very very nice option to have. Props to the Opscode team for building an infrastructure to run the server side for youand especially for making the barrier to entry nearly zero by letting you manage up to five hosts for free. Given some of my headaches around running a puppetmaster previously, I’m glad not to be having to pull together everything to run a chef server
Knife is actually pretty cool. I was skeptical before using it but it does a pretty nice job of encapsulating a lot of common tasks for you
Knife gets really cool with the addition of the ec2 plugin. Launch servers, register them with hosted chef and have them ready to go. I’ve built all of the surrounding bits and as the environment I’m dealing with grows, I think I’ll grow out of being able to use knife ec2 effectively, but it’s great for an easy starting point
Chef solo seems to work okay and have a few niceties over a master-less puppet setup but I didn’t spend much time with masterless puppet, so it’s probably just that I didn’t find the related nice pieces

Things I’ve Disliked / Been Annoyed By

The package support in the Fedora/CentOS/RHEL universe is pretty poor. I realize that all the cool kids use Ubuntu these days but tons of server infrastructures are not. Todd does a great job with the puppet (+ ecosystem) packages for Fedora and EPEL. Would love to see someone do similar for all of the Chef stuff
A lot of the cookbooks that are out there and published are Ubuntu specific. Even the ones which strive to work across distros often end up coercing the Fedora universe to look more like Debian. Which isn’t necessarily a path I want to go down

Probably just a side effect of this but a lot of cookbooks using things which aren’t the standard init system (eg, depending on runit)

knife-ec2 makes you think you can get away with using it but I keep tripping across things it doesn’t support and making me consider abandoning it
Trying out cookbooks from others drives me crazy. I’m pretty sure I’m missing the good workflow here but polluting my checkout by adding vendor branches and auto-committing things. There’s gotta be something I’m missing here

So am I now a rabid chef fan? Nope. But it’s a nice system with some definite advantages for certain use cases. I suspect I’ll find more of them as I use it more.

Originally published at Jeremy's Thoughts. Please leave any comments there.

Christopher Blizzard: I miss you guys

Sun, 15 Apr 2012 05:14:26 +0000

Christopher Blizzard: Keys

Fri, 13 Apr 2012 21:34:03 +0000

Pete Zaitcev: Hard numbers from China

Fri, 13 Apr 2012 16:40:01 +0000

It looks like almost every cloud provider hide their numbers, which makes guidance and education unnecesserily difficult. To be fair, I think I saw AWS posting a few items for S3, but forgot to save it. So, I'm going to preserve what one Chinese gentleman posted to OpenStack list, in the context of Swift performance issues:

Our practice of  Sina Web Service Team https://launchpad.net/~sws:

total accounts:          121,961;
total containers:        160,703;
total objects:        14,291,519;
total storage usage:   1.3T

account replication time:      10 hours;
container replication time:    10 hours;
object replication time:       48 hours;
account audit time:             2 hours;
container audit time:           9 hours;
container update time:         19 hours;

Unfortunately he omitted the requests per second and gigabytes per second that the cluster is sustaining from the users, but it's very interesting anyway.

QUICKIE: Apparently I meant the official post about Amazon S3:

total objects:   905,000,000,000
total bytes:     ?
requests/s:           650,000

Dave Jones: Weekly Fedora kernel bug statistics – April 13 2012

Fri, 13 Apr 2012 16:22:56 +0000

	15	16	17	rawhide
Open:	313	549	63	141	(1066)
Opened since 2012-04-06	1	33	11	2
Closed since 2012-04-06	33	43	7	1
Changed since 2012-04-06	48	67	19	5

Slowly putting a dent in the f15/f16 backlog, now that we have 3.3.1 updates available for both with the i915 memory corruption fix. As can be seen from the ‘closed’ links above, lots of the “weird shit happened” bugs are now closed out. Still a huge number of outstanding bugs though.

Note for Fedora 15 users: Kernel updates are languishing so long in updates-testing that they keep getting obsoleted by new builds before the old one recieves enough Karma.

Weekly Fedora kernel bug statistics – April 13 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...

Related posts brought to you by Yet Another Related Posts Plugin.

Christopher Blizzard: state of the union

Wed, 11 Apr 2012 06:14:47 +0000

Lennart Poettering: Control Groups vs. Control Groups

Tue, 10 Apr 2012 17:09:00 +0000

TL;DR: systemd does not require the performance-sensitive bits of Linux control groups enabled in the kernel. However, it does require some non-performance-sensitive bits of the control group logic.

In some areas of the community there's still some confusion about Linux control groups and their performance impact, and what precisely it is that systemd requires of them. In the hope to clear this up a bit, I'd like to point out a few things:

Control Groups are two things: (A) a way to hierarchally group and label processes, and (B) a way to then apply resource limits to these groups. systemd only requires the former (A), and not the latter (B). That means you can compile your kernel without any control group resource controllers (B) and systemd will work perfectly on it. However, if you in addition disable the grouping feature entirely (A) then systemd will loudly complain at boot and proceed only reluctantly with a big warning and in a limited functionality mode.

At compile time, the grouping/labelling feature in the kernel is enabled by CONFIG_CGROUPS=y, the individual controllers by CONFIG_CGROUP_FREEZER=y, CONFIG_CGROUP_DEVICE=y, CONFIG_CGROUP_CPUACCT=y, CONFIG_CGROUP_MEM_RES_CTLR=y, CONFIG_CGROUP_MEM_RES_CTLR_SWAP=y, CONFIG_CGROUP_MEM_RES_CTLR_KMEM=y, CONFIG_CGROUP_PERF=y, CONFIG_CGROUP_SCHED=y, CONFIG_BLK_CGROUP=y, CONFIG_NET_CLS_CGROUP=y, CONFIG_NETPRIO_CGROUP=y. And since (as mentioned) we only need the former (A), not the latter (B) you may disable all of the latter options while enabling CONFIG_CGROUPS=y, if you want to run systemd on your system.

What about the performance impact of these options? Well, every bit of code comes at some price, so none of these options come entirely for free. However, the grouping feature (A) alters the general logic very little, it just sticks hierarchial labels on processes, and its impact is minimal since that is usually not in any hot path of the OS. This is different for the various controllers (B) which have a much bigger impact since they influence the resource management of the OS and are full of hot paths. This means that the kernel feature that systemd mandatorily requires (A) has a minimal effect on system performance, but the actually performance-sensitive features of control groups (B) are entirely optional.

On boot, systemd will mount all controller hierarchies it finds enabled in the kernel to individual directories below /sys/fs/cgroup/. This is the official place where kernel controllers are mounted to these days. The /sys/fs/cgroup/ mount point in the kernel was created precisely for this purpose. Since the control group controllers are a shared facility that might be used by a number of different subsystems a few projects have agreed on a set of rules in order to avoid that the various bits of code step on each other's toes when using these directories.

systemd will also maintain its own, private, controller-less, named control group hierarchy which is mounted to /sys/fs/cgroup/systemd/. This hierarchy is private property of systemd, and other software should not try to interfere with it. This hierarchy is how systemd makes use of the naming and grouping feature of control groups (A) without actually requiring any kernel controller enabled for that.

Now, you might notice that by default systemd does create per-service cgroups in the "cpu" controller if it finds it enabled in the kernel. This is entirely optional, however. We chose to make use of it by default to even out CPU usage between system services. Example: On a traditional web server machine Apache might end up having 100 CGI worker processes around, while MySQL only has 5 processes running. Without the use of the "cpu" controller this means that Apache all together ends up having 20x more CPU available than MySQL since the kernel tries to provide every process with the same amount of CPU time. On the other hand, if we add these two services to the "cpu" controller in individual groups by default, Apache and MySQL get the same amount of CPU, which we think is a good default.

Note that if the CPU controller is not enabled in the kernel systemd will not attempt to make use of the "cpu" hierarchy as described above. Also, even if it is enabled in the kernel it is trivial to tell systemd not to make use of it: Simply edit /etc/systemd/system.conf and set DefaultControllers= to the empty string.

Let's discuss a few frequently heard complaints regarding systemd's use of control groups:

systemd mounts all controllers to /sys/fs/cgroup/ even though my software requires it at /dev/cgroup/ (or some other place)! The standardization of /sys/fs/cgroup/ as mount point of the hierarchies is a relatively recent change in the kernel. Some software has not been updated yet for it. If you cannot change the software in question you are welcome to unmount the hierarchies from /sys/fs/cgroup/ and mount them wherever you need them instead. However, make sure to leave /sys/fs/cgroup/systemd/ untouched.
systemd makes use of the "cpu" hierarchy, but it should leave its dirty fingers from it! As mentioned above, just set the DefaultControllers= option of systemd to the empty string.
I need my two controllers "foo" and "bar" mounted into one hierarchy, but systemd mounts them in two! Use the JoinControllers= setting in /etc/systemd/system.conf to mount several controllers into a single hierarchy.
Control groups are evil and they make everything slower! Well, please read the text above and understand the difference between "control-groups-as-in-naming-and-grouping" (A) and "cgroups-as-in-controllers" (B). Then, please turn off all controllers in you kernel build (B) but leave CONFIG_CGROUPS=y (A) enabled.
I have heard some kernel developers really hate control groups and think systemd is evil because it requires them! Well, there are a couple of things behind the dislike of control groups by some folks. Primarily, this is probably caused because the hackers in question do not distuingish the naming-and-grouping bits of the control group logic (A) and the controllers that are based on it (B). Mainly, their beef is with the latter (which systemd does not require, which is the key point I am trying to make in the text above), but there are other issues as well: for example, the code of the grouping logic is not the most beautiful bit of code ever written by man (which is thankfully likely to get better now, since the control groups subsystem now has an active maintainer again). And then for some developers it is important that they can compare the runtime behaviour of many historic kernel versions in order to find bugs (git bisect). Since systemd requires kernels with basic control group support enabled, and this is a relatively recent feature addition to the kernel, this makes it difficult for them to use a newer distribution with all these old kernels that predate cgroups. Anyway, the summary is probably that what matters to developers is different from what matters to users and administrators.

I hope this explanation was useful for a reader or two! Thank you for your time!

Lennart Poettering: GUADEC 2012 CFP Ending Soon!

Tue, 10 Apr 2012 15:40:00 +0000

In case you haven't submitted your talk proposal for GUADEC 2012 in A Coruña, Spain yet, hurry: the deadline is on April 14th, i.e. this saturday! Read der Call for Participation! Submit a proposal!

Christopher Blizzard: I feel like I had a pretty good first week at the new job

Sat, 07 Apr 2012 19:53:57 +0000

Christopher Blizzard: running a large drupal or wordpress install? I want to talk to you.

Thu, 05 Apr 2012 21:38:22 +0000

Hi. If you’re running a large-ish Drupal (especially version 6) or a self-hosted WordPress install as a CMS and you’re interested in improving its performance, the company that I just joined is looking for people who might be willing to do some early testing with us on a new product.

If you’re interested, please drop me a line at my email address, Twitter, or Google+.

Christopher Blizzard: Day 0

Mon, 02 Apr 2012 19:44:29 +0000

Thomas Vander Stichele: Evolution backup recovery

Sun, 01 Apr 2012 14:36:12 +0000

I pretty much never drink and hack, and last Friday’s evening is a good reason why. I was having a rare beer and managed to spill part of it on my keyboard and desk. So I turned the keyboard around, started cleaning it as fast as I could, forgetting to actually unplug it. I called it a night because nothing good was going to come from that night anymore.

And on Saturday morning I noticed that my INBOX was gone. Hm, is it really gone? Yep, gone from my laptop too. Crap, must have deleted it on the server by accident while cleaning my keyboard…

And because my NAS is a little full lately, I haven’t been as diligent with backups as I normally have been. Hm, and the modest cache on my N900 isn’t very useful either…

Luckily, evolution on my work machine was shut down for some reason, so yay, it has a reasonably fresh cache of my INBOX!

Except that it’s not all that straightforward to actually get this cache back into Evolution. Just copying its contents to an existing or new folder doesn’t do anything. The files themselves are split up versions of the actual email, assumingly because the evo guys thought it would be faster to search header and body by splitting them off from the attachments and saving them separately, inventing their own caching format. Which is fine, but makes it impossible to actually restore a backup with…

After lots of Googling, I stumbled upon this tool that did the trick for me. A lot of hours wasted over a bunch of emails… But what would happen if I really lost my IMAP server mail ? Run this script by hand on all the folders ? Shudder…

Tom 'spot' Calloway: The Strong, the International Center for the History of Electronic Games, and the Promised Land

Fri, 30 Mar 2012 18:48:32 +0000

I've been out in Rochester, NY for most of this week. Red Hat has been partnering with RIT for the last year or so to generate, produce and teach Open Source courses (for an example, see ritfloss). I represented Red Hat at the RIT spring career fair, and the rest of the week, we've been meeting with various RIT students and faculty.

But to be honest, the coolest thing was what happened yesterday. One of the key advisors in RIT's open source initiatives is also the scholar in residence with The Strong museum, so he offered to take us over there and get a behind-the-scenes tour. The Strong National Museum of Play is dedicated entirely to "play", including tons of stuff on toys, books, comics, a working carousel and passenger train, and all sorts of kid friendly awesomeness.

All of that is good on its own, but the museum is also home to the International Center for the History of Electronic Games. Or as I now refer to it, the Promised Land. Jon-Paul Dyson, the Director, took me and Luke around the exhibits,then he took us into their archives.

The amount of stuff they have is just mindblowing. These pictures do not do it justice. Luke and I were so stunned that we're lucky we managed to take any pictures at all. Rows and rows of shelves. Shelves with video games in their cases, stacked tight, three sets deep. Loose items of all types. A Power Glove next to a Virtual Boy, besides an original Breakout cabinet.

Shelf after shelf of electronic gaming history. Every game I ever loved or ever wanted to love, here. Ken Williams's name badge from Sierra. A retired World of Warcraft server blade.

A wall of electronic (PC and console) gaming magazines. Every Nintendo Power, in order.

Each shelf? Three rows deep. This picture? Just a few of the shelves.

They had an RIT co-op who was gently and carefully playing a game for about 10 minutes and video recording it for archival purposes. That was a paid co-op, btw. He was a happy dude.

I'm sure I'm doing a terrible job describing this, but it was a mind-blowingly awesome experience. I've been a gamer for my entire meaningful sentient existence, and I never thought I'd see a collection like this. I wish I could have stayed there all day taking pictures of the stuff they had, but we only got a walkthrough. They're also collecting all of the gaming ephemera, everything from E3 swag to the original designer notes. The vast majority of their collection isn't on display (the stuff they do have on display is cool too, but it is just a drop in the ocean).

They had a large arcade's worth of cabinet games too, most of which seemed like they were in the process of receiving some love before going on display. A few pinball tables too.

I wish we had taken more pictures, with a better camera. I wish they would have left me in there to roam the stacks. After the tour, we played in their museum arcade until we ran out of tokens. Just a fantastic experience. I'm brainstorming on how Red Hat and FOSS can help them out, feel free to leave suggestions in the comments.

Dave Jones: Weekly Fedora kernel bug statistics – March 30 2012

Fri, 30 Mar 2012 18:09:02 +0000

	15	16	17	rawhide
Open:	344	526	53	142	(1065)
Opened since 2012-03-23	15	149	21	5
Closed since 2012-03-23	4	76	8	5
Changed since 2012-03-23	20	172	27	11

Weekly Fedora kernel bug statistics – March 30 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...

Related posts brought to you by Yet Another Related Posts Plugin.

Dave Jones: resolution on the i915/hibernate memory corruption bug

Fri, 30 Mar 2012 17:03:48 +0000

It seems Dave Airlie managed to track down and fix the bug that has been plagueing hibernate/i915 users for over a year.
My recent posts here showed some reports of memory corruption where we occasionally saw a strip of eight pixels. Sometimes 0×00000000, and sometimes 0x00aaaaaa.
In hindsight, it seems obvious. It’s a framebuffer cursor.

https://bugzilla.kernel.org/attachment.cgi?id=72739 should fix it.

There are still a whole bunch of other hibernate bugs that need fixing, but this at least is a huge step forward.

resolution on the i915/hibernate memory corruption bug is a post from: codemonkey.org.uk

i915, hibernate, and memory corruption. While going through some of the stranger unexplained bugs last...
Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
linked list debugging After looking through all the general protection bugs yesterday, today...

Related posts brought to you by Yet Another Related Posts Plugin.

Thomas Vander Stichele: git bash prompt

Thu, 29 Mar 2012 19:39:30 +0000

I’ve been having fun recently on a new project where I put myself through all sorts of pain by nesting git submodules into team submodules into platform submodules and so on. The goal here is to be able to tag a root repository and thus identify exact commit hashes of all the submodules to any level. This was an idea Andoni had when he was working on livetranscoding in response to a request of mine where I want to be able to use a single ‘tag’ to identify a complete deployment.

That’s been working better than I expected, and I even hacked git-submodule-tools so that I can do git rlog and get a recursive git log between two root version tags, and get a list of every commit between the master and all submodules. That’s pretty neat for writing out release notes.

However, the way I embedded submodules causes a bit of pain when going back and forth. One of my hackers once gave me a PS1 bash prompt that includes info of which git branch you’re on in your shell prompt. So today I decided to extend that a little, and I now have this:

(b:release-0.2.x d:deploy-pro-2012-03-29) [thomas@otto platform]$ ls Makefile platform puppet RELEASE-0.2.1 (b:release-0.2.x d:deploy-pro-2012-03-29) [thomas@otto platform]$ cd puppet/pro/ (s:puppet/pro b:release-0.2.x d:v0.2.1) [thomas@otto pro]$

This is showing me submodule name, branch, and description of the current commit.

If you want this for your prompting fun too, here’s the github repo

In the near future, simple portknocking for fun and profit with bash!

Lennart Poettering: /tmp or not /tmp?

Wed, 28 Mar 2012 12:04:00 +0000

A number of Linux distributions have recently switched (or started switching) to /tmp on tmpfs by default (ArchLinux, Debian among others). Other distributions have plans/are discussing doing the same (Ubuntu, OpenSUSE). Since we believe this is a good idea and it's good to keep the delta between the distributions minimal we are proposing the same for Fedora 18, too. On Solaris a similar change has already been implemented in 1994 (and other Unixes have made a similar change long ago, too). Yet, not all of our software is written in a way that it works nicely together with /tmp on tmpfs.

Another Fedora feature (for Fedora 17) changed the semantics of /tmp for many system services to make them more secure, by isolating the /tmp namespaces of the various services. Handling of temporary files in /tmp has been security sensitive since it has been introduced since it traditionally has been a world writable, shared namespace and unless all user code safely uses randomized file names it is vulnerable to DoS attacks and worse.

In this blog story I'd like to shed some light on proper usage of /tmp and what your Linux application should use for what purpose. We'll not discuss why /tmp on tmpfs is a good idea, for that refer to the Fedora feature page. Here we'll just discuss what /tmp should be used for and for what it shouldn't be, as well as what should be used instead. All that in order to make sure your application remains compatible with these new features introduced to many newer Linux distributions.

/tmp is (as the name suggests) an area where temporary files applications require during operation may be placed. Of course, temporary files differ very much in their properties:

They can be large, or very small
They might be used for sharing between users, or be private to users
They might need to be persistent across boots, or very volatile
They might need to be machine-local or shared on the network

Traditionally, /tmp has not only been the place where actual temporary files are stored, but some software used to place (and often still continues to place) communication primitives such as sockets, FIFOs, shared memory there as well. Notably X11, but many others too. Usage of world-writable shared namespaces for communication purposes has always been problematic, since to establish communication you need stable names, but stable names open the doors for DoS attacks. This can be corrected partially, by establishing protected per-app directories for certain services during early boot (like we do for X11), but this only fixes the problem partially, since this only works correctly if every package installation is followed by a reboot.

Besides /tmp there are various other places where temporary files (or other files that traditionally have been stored in /tmp) can be stored. Here's a quick overview of the candidates:

/tmp, POSIX suggests this is flushed as boot, FHS says that files do not need to be persistent between two runs of the application. Old files are often cleaned up automatically after a time ("aging"). Usually it is recommended to use $TMPDIR if it is set before falling back to /tmp directly. As mentioned, this is a tmpfs on many Linuxes/Unixes (and most likely will be for most soon), and hence should be used only for small files. It's generally a shared namespace, hence the only APIs for using it should be mkstemp(), mkdtemp() (and friends) to be entirely safe.^[1] Recently, improvements have been made to turn this shared namespace into a private namespace (see above), but that doesn't relieve developers from writing secure code that is also safe if /tmp is a shared namespace. Because /tmp is no longer necessarily a shared namespace it is generally unsuitable as a location for communication primitives. It is machine-private and local. It's usually fully featured (locking, ...). This directory is world writable and thus available for both privileged and unprivileged code.
/var/tmp, according to FHS "more persistent" than /tmp, and is less often cleaned up (it's persistent across reboots, for example). It's not on a tmpfs, but on a real disk, and hence can be used to store much larger files. The same namespace problems apply as with /tmp, hence also exclusively use mkstemp()/mkdtemp() for this directory. It is also automatically cleaned up by time. It is machine-private. It's not necessarily fully featured (no locking, ...). This directory is world writable and thus available for both privileged and unprivileged code. We suggest to also check $TMPDIR before falling back to /var/tmp. That way if $TMPDIR is set this overrides usage of both /tmp and /var/tmp.
/run (traditionally /var/run) where privileged daemons can store runtime data, such as communication primitives. This is where your daemon should place its sockets. It's guaranteed to be a shared namespace, but is only writable by privileged code and hence very safe to use. This file system is guaranteed to be a tmpfs and is hence automatically flushed at boots. No automatic clean-up is done beyond that. It is machine-private and local. It is fully-featured, and provides all functionality the local OS can provide (locking, sockets, ...).
$XDG_RUNTIME_DIR where unprivileged user software can store runtime data, such as communication primitives. This is similar to /run but for user applications. It's a user private namespace, and hence very safe to use. It's cleaned up automatically at logout and also is cleaned up by time via "aging". It is machine-private and fully featured. In GLib applications use g_get_user_runtime_dir() to query the path of this directory.
$XDG_CACHE_HOME where unprivileged user software can store non-essential data. It's a private namespace of the user. It might be shared between machines. It is not automatically cleaned up, and not fully featured (no locking, and so on, due to NFS). In GLib applications use g_get_user_cache_dir() to query this directory.
$XDG_DOWNLOAD_DIR where unprivileged user software can store downloads and downloads in progress. It should only be used for downloads, and is a private namespace fo the user, but might be shared between machines. It is not automatically cleaned up and not fully featured. In GLib applications use g_get_user_special_dir() to query the path of this directory.

Now that we have introduced the contestants, here's a rough guide how we suggest you (a Linux application developer) pick the right directory to use:

You need a place to put your socket (or other communication primitive) and your code runs privileged: use a subdirectory beneath /run. (Or beneath /var/run for extra compatibility.)
You need a place to put your socket (or other communication primitive) and your code runs unprivileged: use a subdirectory beneath $XDG_RUNTIME_DIR.
You need a place to put your larger downloads and downloads in progress and run unprivileged: use $XDG_DOWNLOAD_DIR.
You need a place to put cache files which should be persistent and run unprivileged: use $XDG_CACHE_HOME.
Nothing of the above applies and you need to place a small file that needs no persistency: use $TMPDIR with a fallback on /tmp. And use mkstemp(), and mkdtemp() and nothing homegrown.
Otherwise use $TMPDIR with a fallback on /var/tmp. Also use mkstemp()/mkdtemp().

Note that these rules above are only suggested by us. These rules take into account everything we know about this topic and avoid problems with current and future distributions, as far as we can see them. Please consider updating your projects to follow these rules, and keep them in mind if you write new code.

One thing we'd like to stress is that /tmp and /var/tmp more often than not are actually not the right choice for your usecase. There are valid uses of these directories, but quite often another directory might actually be the better place. So, be careful, consider the other options, but if you do go for /tmp or /var/tmp then at least make sure to use mkstemp()/mkdtemp().

Thank you for your interest!

Oh, and if you now complain that we don't understand Unix, and that we are morons and worse, then please read this again, and you might notice that this is just a best practice guide, not a specification we have written. Nothing that introduces anything new, just something that explains how things are.

If you want to complain about the tmp-on-tmpfs or ServicesPrivateTmp feature, then this is not the right place either, because this blog post is not really about that. Please direct this to fedora-devel instead. Thank you very much.

Footnotes

[1] Well, or to turn this around: unless you have a PhD in advanced Unixology and are not using mkstemp()/mkdtemp() but use /tmp nonetheless it's very likely you are writing vulnerable code.

Thomas Vander Stichele: Puppet pains

Tue, 27 Mar 2012 13:53:24 +0000

The jury is still out on puppet as far as I’m concerned.

On the one hand, of course I relish that feeling of ultimate power you are promised over all those machines… I appreciate the incremental improvements it lets you make, and have it give you the feeling that anything will be possible.

But sometimes, it is just so painful to deal with. Agent runs are incredibly slow. It really shouldn’t take over a minute for a simple configuration with four machines. Also, does it really need to be eating 400 MB of RAM while it does so ? And when running with the default included web server (is that webrick ?), I have to restart my puppetmaster for every single run because there is this one multiple definition that I can’t figure out that simply goes away when you restart, but comes back after an agent run:
err: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate definition: Class[Firewall::Drop] is already defined; cannot redefine at /etc/puppet/environments/testing/modules/manifests/firewall/drop.pp:19 on node esp

And sometimes it’s just painfully silly. I just spent two hours trying to figure out why my production machine couldn’t complete its puppet run.

All it was telling me was
Could not evaluate: 'test' is not executable

After a lot of googling, I stumbled on this ticket. And indeed, I had a file called ‘test’ in my /root directory.

I couldn’t agree with the reporter more:

I find it incredibly un-pragmatic to have policies fail to run whenever someone creates a file in root which matches the name of an executable I am running.

Alexander Larsson: Moar windows themes!

Tue, 27 Mar 2012 07:45:35 +0000

Last time I blogged about the Gtk 3 windows theme we had just landed the initial support. It kinda worked but didn’t look quite right. Since then the css machinery has gotten a bit more capable, and I recently found some time to work on this again.

Testing had revealed that the theme was totally broken on Windows XP. This had two main causes; first of all there was some bugs in the Win32 theme APIs on XP when rendering to surfaces with alpha, and secondly the css file used some windows theme parts that only existed in Vista and later. I added workarounds for the alpha bug and introduced a new css file that is used on XP (although most of the css is shared). So, now XP support is working.

I also went through all the widgets, fixing a lot of details in how they render and adding theming for some less common widgets.

Here is how gtk3-widget-factory looks now:

Windows 7

Windows XP

Not bad for 998 lines of css.

This is all in the new Gtk+ 3.4.0 release. We hope to have window binaries out for it soon.

Mark J Cox: Converting my Home Automation front end to Android

Sat, 24 Mar 2012 17:35:00 +0000

When I moved to a new house in 2001 I designed and installed my own home-brew home automation system in it to control things like the heating, lighting, alarm system, and more. The messaging system I picked was to use standard XMPP (Jabber) because most platforms have existing open source XMPP libraries so writing clients is easy. The back-end is just a load of XMPP bots written in Perl. Around the house I had a number of Fujitsu Point 1600 tablets running an interface I designed and wrote using Perl/Tk. The tablets are great, but they're starting to show their age with limited resolution of 800x600 and CPU speed making full-screen video not really possible.

So last year I obtained an Archos 101 10.1" android tablet with the plan being to replace the existing tablets with android powered ones. It was a good excuse to dust off the old skills and learn programming apps for Android. Converting the app was straightforward and tooks a couple of weekends, troubles with the tablet took quite a bit more work.

Finding a way to mount the Archos tablet on a wall proved tricky, the back of the device isn't perfectly flat and it has an annoying desk stand in the middle. I ended up using a PadTab for mounting, but having to custom modify it to handle not being in the centre of the device, and add thick sticky strips (normally used to dampen fans). The build quality of these tablets is pretty poor.

The display panel on the tablet isn't very good either and has limited viewing angles from three sides, so in order to be able to see the screen when mounted to the wall I had to turn it upside down. Android can happily handle a rotated display, the only downside is that the Archos logo is the wrong way up (a bit of black tape covers it up now).

I left the Archos mounted on the wall and running for a week, permanently attached to its charger. At the end of the week I noticed it wasn't sitting straight on the wall, and in fact the internal batteries had both swelled up and burst out of the case. I read online a few other stories from folks who had bought Archos tablets which had failed in the same way, I guess they're really not designed to be left on charge permanently (that's really bad design Archos, this could have easily caught fire!)

I figured I didn't really need to have batteries installed, the tablet is going to be permanently powered on anyway, and it would be safer to leave the house knowing there was no risk of exploding batteries. So I took the tablet apart and removed them. Without batteries the Archos starts its power up cycle, displays a logo, then gets to a certain part of the boot process and powers down. I guess it does a check on the state of the batteries and it fails. This presented a real problem and I gave up trying to use the tablet. Over the Christmas holidays I heard that you could flash an alternative OS, CyanogenMod, and that actually booted and ran just fine without batteries, but it wasn't stable and featured enough for running the Home Automation app I'd written.

So I decided to try to debug the Archos OS, so connected it to USB to get debug messages, and interestingly it powered up perfectly first time. Removing the USB connection caused it to lock up a few seconds later. Strange behaviour! I tried just connecting power to the USB port, and that worked too. So if you want to run your Archos 101 android tablet without internal batteries you can, but you need to splice your power cable and feed 5v to both the power socket and the USB socket.

So now I had a working tablet again I changed the power adaptor so it mounted neatly against the wall:

Here are a couple of pics of it in use:

Dave Jones: Weekly Fedora kernel bug statistics – March 23 2012

Fri, 23 Mar 2012 19:11:14 +0000

	15	16	17	rawhide
Open:	346	505	49	142	(1042)
Closed since 2012-03-16	5	68	11	5
Changed since 2012-03-16	13	407	21	4

This week saw a lot of activity in Fedora 16. We pushed out a rebase to 3.3. Yesterday I did a mass-update to bugzilla requesting people retest. (Which screwed up when I got a bugzilla proxy error, causing everything to be posted 3 times. Apologies to those who got all those mails).

The good news is we seem to be closing a lot of bugs quickly from this rebase.
Around 50 or so bugs got closed just since yesterday.
As with any rebase, some new bugs are showing up, but some of the more common ones (like the bluetooth oopses) I think we’ve got a handle on, and will get fixed in an update out next week.

Next week, 3.3.1 should also land, and that will likely coincide with Fedora 15 also getting a rebase to this kernel. It’s been a while since we’ve had three releases all on the same version. (I think last time was F12/F13/F14).

Despite all this activity, the overall bug count still remains very high. We still have no real answer for all the memory corruption problems caused by hibernate (which accounts for dozens of open bugs now, plus probably a bunch that we haven’t yet attributed to this problem).

Weekly Fedora kernel bug statistics – March 23 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...
Weekly Fedora kernel bug statistics. No time for a complete breakdown this week. (I’ll post...

Related posts brought to you by Yet Another Related Posts Plugin.

Pete Zaitcev: Confounding

Thu, 22 Mar 2012 03:37:14 +0000

16.1.9 JSON Format for ACLs

ACE flags and masks are members of a 32-bit quantity that is widely understood in its hexadecimal representations. The JSON data format does not support hexadecimal integers, however. For this reason, all hexadecimal integers in CDMI ACLs shall be represented as quoted strings containing a leading "0x".

"cdmi_acl" : [ { "acetype" : "0xnn", .....

If readability by humans is paramount, then why not use a bit string, like in ls(1)? If readability is not an issue, just transmit decimal.

They also have hexadecimals without "0x" prefix elsewhere in the spec.

Luis Villa: Joining the Open Source Initiative board of directors

Mon, 19 Mar 2012 23:09:55 +0000

In the past, I’ve been known to say that skeptical things about the Open Source Initiative’s role in the open source world – usually arguing that OSI was doing the basics (license approval, open source definition) respectably, but also had a lot of potential that wasn’t being taken advantage of. I’m excited to announce that I’m now putting my money where my mouth is, and joining the OSI board of directors.

“Hello, My Name is Open Source” by opensourceway, used under CC-BY-SA license

I’ll write more about my goals for OSI (and for my participation in it) in the coming months, once I’ve gotten a chance to actually meet with the rest of the board and better understand the projects that are already underway. But right now I think it’s very important to note how I became a member of the board, because I think it says something important about where OSI is going, and about why I agreed to invest my time and energy.

Specifically, at FOSDEM, OSI announced that it was beginning to shift in part to an affiliate model, where open source organizations like Mozilla, KDE, and others would have input into OSI’s processes and decisionmaking.¹ One of the first tangible outcomes of that process was to ask affiliate orgs to nominate board members. The result: Mozilla nominated me, and Eclipse nominated fellow new board member Mike Milinkovich. Because of this, our election is less about us,² and more about taking very concrete steps towards an OSI with deeper ties to the broader open source community. And that, I think, reflects what OSI has not always been, but could be – a place where the best of open source can talk and work together to move common interests forward.

Ask me how your organization can join!
Though obviously I expect we’ll be great :)

Pete Zaitcev: Bugception

Mon, 19 Mar 2012 20:00:24 +0000

[root@kvm-rei zaitcev]# rpm --rebuilddb error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch error: cannot open Packages index using db5 - (-30969)

I have a vague feeling that I encountered this before, but I do not remember how I dealt with it.

UPDATE: Needs rm /var/lib/rpm/__db* .

Thomas Vander Stichele: DAD hacking

Sat, 17 Mar 2012 23:06:59 +0000

On the bad side of life, I was planning to go to an awesome Calcotada in Lleida today, but I spent last night awake until 4:30 with an upset stomach, so I had to cancel and stay home feeling like shit.

On the good side of life, I really had no excuse left to not do a little long overdue hacking on Digital Audio Database.

I still use it regularly to listen to music, but the GNonLin-based player is just really not very stable. I should really just rewrite it using simply adder just like roughly ten years ago, but my brain won’t be able to do that. So instead I decided to clean up the web-based WebSockets using player I prototyped at OVC last year.

I started with some refactoring, clearly defining model/view/controller base classes and adapting the player and playerview classes to them.

WebSocket code seems to need an update every few months – I pulled the latest revision of txWebsocket on my fork, so my recent browsers actually play music again.

Since the last time I hacked on this, I actually added my 1500+ freshly ripped cd’s, in FLAC format – which browsers don’t actually support.

So, first off, I added an option for the scheduler – responsible for picking tracks, and picking audio files to represent them – to filter by extension. It’s not ideal, but it will do for now, and I punched that filter through the levels of abstraction in DAD. I now start it filtering on .mp3 and .oga, and so Chrome can play back all the tracks the scheduler throws at it.

The web-based player just loads tracks and timing info from the scheduler relative to page load time. I’ve been wanting to make that absolute for a while, so I did just that – the player server schedules tracks for epoch seconds now through websockets.

I had an entertaining half hour listening to the awesome echo effects obtained by having three chrome pages simultaneously playing the jukebox schedule – each page being slightly out of sync with the others.

As I’ll be wanting to use a smallish computer for music playback using a browser, I adapted the code to not use localhost any longer, but do everything with relative URL’s. Voila – the laptop now plays music too, a little bit more out of sync, and of course through its own speakers, adding to the eerie effect.

As an encore, I wanted to stumble my way through some jquery code, to which I’m a certified newbie. I want a nice background slideshow related to the current artist, and I pulled together echonest and bgstretcher-2 as an experiment.

That seems to work relatively well, except that the slideshow plugin doesn’t let you reload a new set of images to cycle through. And some of the other ones I tried instead after that seemed to have the same problem.

Oh well, it’s a start. If anyone knows of a good jquery background slideshow plugin that lets me update the list of url’s for images at any time, let me know!

Luis Villa: On the Importance of Per-File License Information

Sat, 17 Mar 2012 18:43:00 +0000

After the release of MPL 2, the first request for MPL 2.1 came from someone who didn’t want to put copyright headers in individual files. The issue has recently reared its head in Apache as well, and I recently was asked related questions by a GPL user as well.

The main reasons given for not using per-file headers are two-fold:

They’re awkward in short files. Many programming frameworks these days (most notably rails) are encouraging creation of many short files, so this is becoming a bigger problem than it was when per-file headers were first created.
They’re not considered very relevant in languages or frameworks that are library-centric, especially those with package managers that are heavily used (like ruby’s gems). Again, many modern language frameworks include tooling that encourages this approach, so some developers are thinking “why can’t I just express the license once per library?”

The case for per-file copyright headers is put well, and succinctly, by Larry Rosen:

“[O]ur goal is to pass on any important IP information that might be useful … in the place(s) [downstream licensees] are most likely to find it.”

Larry’s comment makes two assumptions that I want to flesh out and support.

First, Larry assumes that the place where people are “most likely to find” licensing information is in per-file headers. It is true that in the best case scenario in many modern languages/frameworks, library-level is a great place to put licenses – in normal use, they’ll get seen and understood. But lots of coding in the wild is not “normal use.” I review a lot of different codebases these days, and files get separated from their parent projects and directories all the time. And then you have to use fairly complex (and often expensive) tools to do what should be a simple task- figure out what the license is. So, yes, modern frameworks should in theory reduce the need for per-file licensing information – but in practice, that is often not the case.

Second, Larry assumes that you actually want people to use your code. Lots of publishers of open source code seem surprisingly unconcerned by this, unfortunately. The functional, practical benefits of open source all start with someone else reusing your code, so if you’re publishing open source code at all, you should be concerned about making it easy for people to use the code you publish. Again, putting licensing information in each file can help make this easier, by making it easier for people to figure out their rights and responsibilities. (This is particularly true if you want commercial uptake, since so many commercial users of open source are getting more conservative about using source code that is not properly labeled and licensed.)((Larry also perhaps assumes you want people to respect your license when using your code; that is a surprisingly complex topic that I will try to address some other day.))

So, yes: if you want people to find your licensing information, and to use your code, per-file headers are the way to go. They may not be ideal but they really are worth the effort.

Christopher Blizzard: opportunity

Sat, 17 Mar 2012 04:26:29 +0000

Today is my last day working at Mozilla. I’ve been involved with Mozilla, more often than not in a full time capacity, since its public inception in 1998. In that time I’ve worn a lot of hats for the organization. I’ve written code, maintained my own module, done driving and super-review. I sat on the original staff@mozilla.org list that steered the project, served on the board of the Foundation and drove a few early Mozilla releases. More recently I ran our developer evangelism & marketing group and did platform product management for Mozilla. To say that it’s been an important part of my life would be an understatement.

For those of you who might read this and work at Mozilla, I will miss you and love what you’ve done for the web. I just feel lucky that I’ve had the chance to be a part of it. And for those of you who use the web every day I feel lucky to have an active hand in the technology that you’ll be using tomorrow.

Everyone is asking what’s next for me, so I’ll mention that here too. I’m trying something new and starting in the next week or so I’m going to be joining a very small start-up that’s based in Palo Alto. I happened to stumble across an amazing team that’s doing great (and difficult!) work that deals with the intersection of systems, compilers, and web-scale problems. It’s not browser work so it will be a bit of a change for me. But I’m super excited to be able to help drive a product that will positively affect the lives of a large number of web developers and improve the web at the same time.

Dave Jones: Weekly Fedora kernel bug statistics – March 16 2012

Fri, 16 Mar 2012 18:48:56 +0000

	15	16	17	rawhide
Open:	349	509	31	144	(1033)
Closed since 2012-03-09	2	53	16	4
Changed since 2012-03-09	13	115	9	4

It’s pretty clear to see that f16 has been seeing most of the attention this week. (As it has been for most of the month).

Overall bug count creeping up, but some of the long standing bugs like the various corruption bugs mention in last weeks posts are slowly edging their way to closure. Andrea Arcangeli fixed bug in the transparent huge page code which explains a bunch of the page table corruption issues we saw reported. The i915/hibernate/memory corruption bug has at least some theories on a potential solution now, and we’ve been able to attribute another bunch of the ‘weird’ bugs to post-hibernate corruption now that we know what patterns to look for.

With 3.3 likely being released next week, we’ll be moving F16 to it (and F15 a week later). Hopefully we get to close out a bunch more of the longer standing bugs that haven’t been backported to -stable.

Weekly Fedora kernel bug statistics – March 16 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...
Weekly Fedora kernel bug statistics. No time for a complete breakdown this week. (I’ll post...
Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17 The 3.2 rebase for F15 got pushed to updates. Not...

Related posts brought to you by Yet Another Related Posts Plugin.

Caolan McNamara: shiny langtag library

Tue, 13 Mar 2012 13:15:34 +0000

liblangtag looks very nice. I wonder if there’s anything in my abandonware localehelper that might be useful to stuff in there. Maybe some of the locale to langtag mapping stuff.

Dave Jones: i915, hibernate, and memory corruption.

Mon, 12 Mar 2012 19:46:24 +0000

While going through some of the stranger unexplained bugs last week, I noticed that in a few cases, the hex pattern 0x00aaaaaa showed up a few times. Sometimes, these patterns aren’t obvious when you don’t know what you’re looking for.

After a discussion with Keith Packard about the ongoing memory corruption problems we’re seeing with i915 & hibernation, he pointed out that 0x00aaaaaa is likely a strip of grey pixels in ARGB format. Given that the pattern repeated through memory 8 times, it’s even more likely to be a strip of pixels.

The question still remains as to why these pixels are being scribbled into random memory. Keith’s current theory is that the GTT contains stale entries from before the hibernate, which on resume don’t point to the same memory they did before. Sounds plausible, and would explain a whole number of weird bugs that we’ve seen the last 9 months or so.

Petr Tesarik did some debugging on probably the same problem in June 2011.
There are also numerous other incidences of this corruption in other mailing lists/bug trackers.

i915, hibernate, and memory corruption. is a post from: codemonkey.org.uk

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Caolan McNamara: libreoffice help ported to clucene

Thu, 08 Mar 2012 12:21:41 +0000

From the things that make me happy department. Years ago our help documentation source was parsed with a bunch of java tools. At the time gcj was the only possibility for us in RHEL/Fedora and the build time for all localized langpacks that we included was about 26 hours in our build system.

Which was a bit depressing.

So I rewrote it in c++, taking super care to keep the same JavaHelp-derived format and so forth. Which brought build times down to about 10 hours.

Which made me happy.

At some stage though, then it was decided to then index our help with lucene, which brought back java as a build-time and run-time dependency for building help and searching it at run-time.

Which made me sad again, though openjdk was the default for us at this stage, so it wasn’t as much of a pain, though that’s why you have that perceptual lag when you first search for a term in help.

But now, for LibreOffice 3.6, Gert van Valkenhoef has ported our lucene code to clucene. helpcontent builds faster, and there’s no lag on searching for something in help.

Which made me happy.

Distro’s that want to use –with-system-clucene will need to build and install clucene’s contribs-lib

Dave Jones: linked list debugging

Tue, 06 Mar 2012 22:41:17 +0000

After looking through all the general protection bugs yesterday, today I spent some time trying to correlate the various reports we have of the linked list debugging being triggered

I added this code upstream in 2006, that gets called whenever a node is added/removed to linked lists, it checks that the prev->next and next->prev of the nodes points to the right places. It trips up surprisingly often. So much so, that we leave it permanently enabled in Fedora. The overhead is negligable, and it finds genuine bugs.

Sifting through the current reports, a few things jumped out at me.

784741,799229 & 799692: list_del corruption. prev->next should be ffff8801c2f41b18, but was (null)
These all look to be the same bug. The inode writeback list got corrupted. upstream discussion points at a commit that split the lock up as a potential cause. Investigation ongoing.

787044: list_del corruption, ffff88013008e520->next is LIST_POISON1 (dead000000100100)
Like the previous bugs, this is in the inode eviction path. This looks to be unrelated however, as it’s a different linked list (this time one private to jbd2).
Of note is that this happened on return from hibernate, where we know we have a number of memory corruption bugs. LIST_POISON showing up in objects on lists shouldn’t happen, because we take them off the list and fix up the pointers before we do the poisoning. Re-adding a poisoned object isn’t possible, as the pointers would be fixed up correctly at list_add. So it’s not clear to me how this got in this state.

722723: list_del corruption. prev->next should be ffffea0003f8d4f0, but was dead000000100100
Another case of the same dead object still on the list problem, but in a different code path. This time the vm lru page list.
The next Fedora 15/16 updates have some additional VM debugging enabled, which hopefully yields some clues to this sort of problem.

760642: list_add corruption. prev->next should be next (ffff880036835630), but was ffff880036875630. (prev=ffff880036835630).
769576: list_del corruption. next->prev should be ffff8801b068a6e0, but was fdff8801b068a6e0
788280: list_del corruption. prev->next should be ffff88003b0431a8, but was ffff8800390431a8
In all three of these cases, a single bit in the address is different from what was expected. This is usually indicative of a hardware fault of some kind (bad memory, poor cooling, insufficient power etc). It’s possible that something randomly set/unset some bit in memory, but usually random memory corruption happens on at least a byte level.

796109: list_del corruption. next->prev should be ffff88007f411168, but was 00ffffff00ffffff
This one jumped out at me, because it reminded me of bug 746482 from yesterday. Completely different bug, but the pattern looks similar. It looks like something wrote a zero byte to every fourth byte. Almost as if some code somewhere has its pointer arithmetic wrong, and instead of increasing byte at a time zeroing an array, it’s scribbled past the end of it by increasing by sizeof(int) instead of sizeof(char). Trying to find out what’s doing that writing however, is the hard part.

As with all commonly reported bugs, there’s a bunch of other list_debug reports, that don’t fit any particular profile.

linked list debugging is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...
Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17 The 3.2 rebase for F15 got pushed to updates. Not...

Related posts brought to you by Yet Another Related Posts Plugin.

Caolan McNamara: cross-compiling LibreOffice for windows (mingw32) under Fedora

Tue, 06 Mar 2012 11:55:37 +0000

Dave Tardon’s new howto cross-compile LibreOffice under Fedora to target mingw32 under Fedora, http://dtardon.fedorapeople.org/mingw/

Dave Jones: general protection faults, and other weird kernel addresses.

Mon, 05 Mar 2012 23:37:07 +0000

Spent some time today looking over all the general protection fault bugs we have open, making notes of the addresses they were faulting on, in the hope to spot some patterns.

Not much luck. They all look pretty unique kind of screwups.
652204: dereferencing RDI: 0405058300120024
Pretty much gibberish. That number could have come from anywhere.
710709: dereferencing RBX: 0101015001494454
This almost looks like something got OR’d with 0101010101010101, but not quite. No idea.
733608: dereferencing RDI: 00ff880005275660
This one is interesting. This looks like a valid address, but the top byte is 0. Almost as if something scribbled an erroneous 0 to the wrong address.
We have a bunch of bugs which fit this, usually after a hibernate. Still an unknown cause.
735995: dereferencing RAX: 0101141414140008
Very unusual address. Similar pattern to 710709
746482: dereferencing RDI: 00aaaaaa00aaaaaa
Now that’s just weird. Almost like we had a bunch of memory filled with 0xaa’s, and something scribbled zero’s to the MSB of every int.
771794: derferencing RDI: fbff88022ab26500
This would be a valid address if it wasn’t for a single bit that changed an 0xff to a 0xfb. Probably bad memory.
772444: dereferencing RAX: ff8801150f200000
This is a strange address. It looks like it’s been shifted to the left by 8 bits.
782751: derefencing RAX: 8660000000000000
w.t.f.
799983: RAX: f000f84dc0000f84
This almost looks like an address. One that’s had 12 bits of it ANDed away.

There were a few others that were dereferencing 0x6b6b6b6b6b6b6b6b, which is a sign of dereferencing memory after it’s been free’d (and poisoned).

So no real answers, but some more clues that something is scribbling 0′s over bits of memory.

general protection faults, and other weird kernel addresses. is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...
Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17 The 3.2 rebase for F15 got pushed to updates. Not...
Weekly Fedora kernel bug statistics. No time for a complete breakdown this week. (I’ll post...

Related posts brought to you by Yet Another Related Posts Plugin.

Thomas Vander Stichele: More adventures in puppet

Sun, 04 Mar 2012 22:32:31 +0000

After last week’s Linode incident I was getting a bit more worried about security than usual. That coincided with the fact that I found I couldn’t run puppet on one of my linodes, and some digging turned up that it was because /tmp was owned by uid:gid 1000:1000. Since I didn’t know the details of the breakin (and I hadn’t slept more than 4 hours for two nights, one of which involving a Flumotion DVB problem), I had no choice but to be paranoid about it. And it took me a good half hour to realize that I had inflicted this problem on myself – a botched rsync command (rsync arv . root@somehost:/tmp).

So I wasn’t hacked, but I still felt I needed to tighten security a bit. So I thought I’d go with something simple to deploy using puppet – port knocking.

Now, that would be pretty easy to do if I just deployed firewall rules in a single set. But I started deploying firewall rules using the puppetlabs firewall module, which allows me to group rules per service. So that’s the direction I wanted to head off into.

On saturday, I worked on remembering enough iptables to actually understand how port knocking works in a firewall. Among other things, I realized that our current port knocking is not ideal – it uses only two ports. They’re in descending order, so usually they would not be triggered by a normal port scan, but they would be triggered by one in reverse order. That is probably why most sources recommend using three ports, where the third port is between the first two, so they’re out of order.

So I wanted to start by getting the rules right, and understanding them. I started with this post, and found a few problems in it that I managed to work out. The fixed version is this:
UPLINK="p21p1" # # Comma seperated list of ports to protect with no spaces. SERVICES="22,3306" # # Location of iptables command IPTABLES='/sbin/iptables'


# in stage1, connects on 3456 get added to knock2 list

${IPTABLES} -N stage1

${IPTABLES} -A stage1 -m recent --remove --name knock

${IPTABLES} -A stage1 -p tcp --dport 3456 -m recent --set --name knock2
# in stage2, connects on 2345 get added to heaven list

${IPTABLES} -N stage2

${IPTABLES} -A stage2 -m recent --remove --name knock2

${IPTABLES} -A stage2 -p tcp --dport 2345 -m recent --set --name heaven
# at the door:

# - jump to stage2 with a shot at heaven if you're on list knock2

# - jump to stage1 with a shot at knock2 if you're on list knock

# - get on knock list if connecting t0 1234

${IPTABLES} -N door

${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2

${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock -j stage1

${IPTABLES} -A door -p tcp --dport 1234 -m recent --set --name knock
${IPTABLES} -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

${IPTABLES} -A INPUT -p tcp --match multiport --dport ${SERVICES}  -i ${UPLINK} -m recent --rcheck --seconds 5 --name heaven -j ACCEPT

${IPTABLES} -A INPUT -p tcp --syn -j door

# close everything else ${IPTABLES} -A INPUT -j REJECT --reject-with icmp-port-unreachable

And it gives me this iptables state:

So the next step was to reproduce these rules using puppet firewall rules.

Immediately I ran into the first problem – we need to add new chains, and there doesn’t seem to be a way to do that in the firewall resource. At the same time, it uses the recent iptables module, and none of that is implemented either. I spent a bunch of hours trying to add this, but since I don’t really know Ruby and I’ve only started using Puppet for real in the last two weeks, that wasn’t working out well. So then I thought, why not look in the bug tracker and see if anyone else tried to do this ? I ask my chains question on IRC, while I find a ticket about recent support. A minute later danblack replies on IRC with a link to a branch that supports creating chains – the same person that made the recent branch.

This must be a sign – the same person helping me with my problem in two different ways, with two branches? Today will be a git-merging to-the-death hacking session, fueled by the leftovers of yesterday’s mexicaganza leftovers.

I start with the branch that lets you create chains, which works well enough, bar some documentation issues. I create a new branch and merge this one on, ending up in a clean rebase.

Next is the recent branch. I merge that one on. I choose to merge in this case, because I hope it will be easier to make the fixes needed in both branches, but still pull everything together on my portknock branch, and merge in updates every time.

This branch has more issues – rake test doesn’t even pass. So I start digging through the failing testcases, adding print debugs and learning just enough ruby to be dangerous.

I slowly get better at fixing bugs. I create minimal .pp files in my /etc/puppet/manifests so I can test just one rule with e.g. puppet apply manifests/recent.pp

The firewall module hinges around being able to convert a rule to a hash as expressed in puppet, and back again, so that puppet can know that a rule is already present and does not need to be executed. I add a conversion unit test for each of the features that tests these basic operations, but I end up actually fixing the bugs by sprinkling print’s and testing with a single apply.

I learn to do service iptables restart; service iptables stop to reset my firewall and start cleanly. It takes me a while to realize when I botched the firewall so that I can’t even google (in my case, forgetting to have -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
) – not helped by the fact that for the last two weeks the network on my home desktop is really flaky, and simply stops working after some activity, forcing me to restart NetworkManager and reload network modules.

I start getting an intuition for how puppet’s basic resource model works. For example, if a second puppet run produces output, something’s wrong. I end up fixing lots of parsing bugs because of that – once I notice that a run tells me something like
notice: /Firewall[999 drop all other requests]/chain: chain changed '-p' to 'INPUT' notice: Firewall[999 drop all other requests](provider=iptables): Properties changed - updating rule
I know that, even though the result seems to work, I have some parsing bug, and I can attack that bug by adding another unit test and adding more prints for a simple rule.

I learn that, even though the run may seem clean, if the module didn’t figure out that it already had a rule (again, because of bogus parsing), it just adds the same rule again – another thing we don’t want. That gets fixed on a few branches too.

And then I get to the point where my puppet apply brings all the rules together – except it still does not work. And I notice one little missing rule: ${IPTABLES} -A INPUT -p tcp –syn -j door

And I learn about –syn, and –tcp-flags, and to my dismay, there is no support for tcp-flags anywhere. There is a ticket for TCP flags matching support, but nobody worked on it.

So I think, how hard can it be, with everything I’ve learned today? And I get onto it. It turns out it’s harder than expected. Before today, all firewall resource properties swallowed exactly one argument – for example, -p (proto). In the recent module, some properties are flags, and don’t have an argument, so I had to support that with some hacks.

The rule_to_hash function works by taking an iptables rule line, and stripping off the parameters from the back in reverse order one by one, but leaving the arguments there. At the end, it has a list of keys it saw, and hopefully, a string of arguments that match the keys, but in reverse order. (I would have done this by stripping the line of both parameter and argument(s) and putting those on a list, but that’s just me)

But the –tcp-flags parameter takes two arguments – a mask of flags, and a list of flags that needs to be set. So I hack it in by adding double quotes around it, so it looks the same way a –comment does (except –comment is always quoted in iptables –list-rules output), and handle it specially. But after some fidgeting, that works too!

And my final screenshot for the day:

So, today’s result:

Now, I have a working node that implements port knocking:
node 'ana' {


    $port1 = '1234'

    $port2 = '3456'

    $port3 = '2345'
    $dports = [22, 3306]
    $seconds = 5
    firewall { "000 accept all icmp requests":

      proto => "icmp",

      action => "accept",

    }
    firewall { "001 accept all established connections":

      proto => "all",

      state => ["RELATED", "ESTABLISHED"],

      action => "accept",

    }
    firewall { "999 drop all other requests":

      chain => "INPUT",

      proto => "tcp",

      action => "reject",

    }
    firewallchain { [':stage1:', ':stage2:', ':door:']:

    }
    # door

    firewall { "098 knock2 goes to stage2":

      chain => "door",

      recent_command => "rcheck",

      recent_name => "knock2",

      recent_seconds => $seconds,

      jump => "stage2",

      require => [

	Firewallchain[':door:'],

	Firewallchain[':stage2:'],

      ]

    }
    firewall { "099 knock goes to stage1":

      chain => "door",

      recent_command => "rcheck",

      recent_name => "knock",

      recent_seconds => $seconds,

      jump => "stage1",

      require => [

	Firewallchain[':door:'],

	Firewallchain[':stage1:'],

      ]

    }
    firewall { "100 knock on port $port1 sets knock":

      chain => "door",

      proto => 'tcp',

      recent_name => 'knock',

      recent_command => 'set',

      dport => $port1,

      require => [

	Firewallchain[':door:'],

      ]

    }
    # stage 1

    firewall { "101 stage1 remove knock":

      chain => "stage1",

      recent_name => "knock",

      recent_command => "remove",

      require => Firewallchain[':stage1:'],

    }
    firewall { "102 stage1 set knock2 on $port2":

      chain => "stage1",

      recent_name => "knock2",

      recent_command => "set",

      proto => "tcp",

      dport => $port2,

      require => Firewallchain[':stage1:'],

    }
    # stage 2

    firewall { "103 stage2 remove knock":

      chain => "stage2",

      recent_name => "knock",

      recent_command => "remove",

      require => Firewallchain[':stage2:'],

    }
    firewall { "104 stage2 set heaven on $port3":

      chain => "stage2",

      recent_name => "heaven",

      recent_command => "set",

      proto => "tcp",

      dport => $port3,

      require => Firewallchain[':stage2:'],

    }
    # let people in heaven

    firewall { "105 heaven let connections through":

      chain => "INPUT",

      proto => "tcp",

      recent_command => "rcheck",

      recent_name => "heaven",

      recent_seconds => $seconds,

      dport => $dports,

      action => accept,

      require => Firewallchain[':stage2:'],

    }

firewall { "106 connection initiation to door": # FIXME: specifying chain explicitly breaks insert_order ! chain => "INPUT", proto => "tcp", tcp_flags => "FIN,SYN,RST,ACK SYN", jump => "door", require => [ Firewallchain[':door:'], ] } }
and I can log in with
nc -w 1 ana 1234; nc -w 1 ana 3456; nc -w 1 ana 2345; ssh -A ana

Lessons learned today:

watch iptables -nvL is an absolutely excellent way of learning more about your firewall – you see your rules and the traffic on them in real time. It made it really easy to see for example the first nc command triggering the knock.
Puppet is reasonably hackable – I was learning quickly as I progressed through test and bug after test and bug.
I still don’t like ruby, and we may never be friends, but at least it’s something I’m capable of learning. Puppet might just end up being the trigger.

Tomorrow, I need to clean up the firewall rules into something reusable, and deploy it on the platform.

Dave Jones: Weekly Fedora kernel bug statistics – March 2 2012

Fri, 02 Mar 2012 17:46:44 +0000

	15	16	17	rawhide
Open:	344	485	22	145	(996)
Closed since 2012-02-24	33	94	8	8
Changed since 2012-02-24	17	125	13	17

Total bug count back under 1000 again. A slightly better picture than last week, though a number of really painful issues continue to dominate the bug reports. There seems to be a number of issues (or perhaps one widespread bug) that causes severe memory corruption. The forms it seems to take are corrupted linked lists (usually dcache, though not surprising given it’s going to be the majority of memory), corrupted page tables, or other similar “something scribbled over something in kernel memory”.

Some of the page table bugs look like an in-use page ended up on the free list. My working theory right now is that there’s a locking problem somewhere exposing a race, though code inspection hasn’t revealed anything yet. Annoyingly, I can’t reproduce any of these problems locally either.

Weekly Fedora kernel bug statistics – March 2 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics. No time for a complete breakdown this week. (I’ll post...
Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17 The 3.2 rebase for F15 got pushed to updates. Not...
Fedora 16 kernel bugzilla status report from 2012-02-10 to 2012-02-17 494 bugs open in total. (up from 378 last week)...

Related posts brought to you by Yet Another Related Posts Plugin.

Pete Zaitcev: Twitterfail 2

Fri, 02 Mar 2012 01:07:16 +0000

Just ran into a weird case on Twitter: If I tweet the following link to Failblog, the tweet disappears:

http://failblog.org/2012/02/23/epic-win-photos-win-wwf-win/

It disappears even if shortened with something like TinyURL. I noticed because a couple of people favourited my tweet before it disappeared, and then when it disappeared, Twitter's own "interactions" page became corrupted (one of their Javascript functions returned "undefined").

Caolan McNamara: syncfonts is handy

Wed, 29 Feb 2012 14:49:49 +0000

When debugging font related stuff its typical that the problem can only be triggered by a specific set of fonts. Here’s a rough-and-ready syncfonts script which when given the output of fc-list -v will try and install the fonts that are missing and remove the extraenous ones via yum, which works for the common case

Dave Jones: Some handy bugzilla scripts.

Tue, 28 Feb 2012 01:05:40 +0000

I got frustrated waiting for the bugzilla search query page to render every time I wanted to search for a fedora kernel bug.
Having to click ‘refresh components’ and wait for all the ajax’y stuff to pull down all the package names etc just sucked too much.
So I hacked up a simple shell script.
Over time, I’ve added a few other features to it too, to search for bugs in other places.

Use it like so..
$ bugsearch bad page state $ bugsearch lkml bad page state
Another script I’ve had around for a while is bz.
Ever had email like “hey, what’s up with bug 728531 ?”
$ bz 728531
and I’m done. It’s set up to look at Red Hat bugzilla by default (but takes fdo/gnome/korg as a 1st arg, and goes there to look instead).

Cheesy, but a timesaver.

Some handy bugzilla scripts. is a post from: codemonkey.org.uk

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Tom 'spot' Calloway: Red Hat Job Openings in the Fedora Universe

Mon, 27 Feb 2012 17:04:18 +0000

There are currently two job openings at Red Hat for people who wish to work on Fedora.

= Fedora Virtualization Maintainer =
Do you enjoy working with the Fedora community? Are you interested in working with the leading edge of the virtualization stack? We are looking for someone to maintain the core virtualization packages in Fedora!

Responsibilities include:
* Interract with the Fedora community on all things virt (Mailing lists, irc, FUDCon.)
* Virtualization Test Day organization
* Coordinate the Fedora feature process for virtualization features
* Maintain the fedora-virt mailing list
* Maintain the virtualization pages on the Fedora wiki
* Maintain the virtio-win drivers repository
* Maintain the virtualization preview repository for Fedora
* Primary package maintainer for the following Fedora packages:
qemu (qemu-kvm)
seabios
sgabios
gpxe (ipxe)
* Work with the kernel team on kvm specific bugs
* Assist in bugs for related virt packages

Qualified candidates will have experience with:
* C, C++, Linux (kernel and userspace), KVM, qemu

(This opening is not currently live on the Red Hat website as far as I know, so please email me your resume/CV along with a short note about why you think you'd be awesome for this opening. I am _not_ the hiring manager for this position, but it is Fedora focused.)

== Web Application Developer ==
https://careers.redhat.com/ext/detail?redhat9418

The Fedora Engineering team is looking for qualified candidates to assist us in creating and improving Free Software solutions for the Fedora Community. Fedora only uses Free Software solutions in all of its infrastructure and hosting, and you can help us in that effort. Our infrastructure is mostly driven using Python frameworks, delivering applications and contents over the web to the Fedora user and contributor communities. Travel requirements are minimal, although, participation at relevant conferences and events is expected. This position is a fantastic opportunity to develop new and interesting Free Software and Open Source solutions with a rich community of developers and users. Upstream contributions are an expected part of this position.

Existing team members in this role have created web applications like:
* Bodhi - https://admin.fedoraproject.org/updates
* Fedora Packages -https://community.dev.fedoraproject.org/packages
* Fedora Tagger - https://community.dev.fedoraproject.org/tagger

Requirements:
These skills are the sort of skills that are desirable, however, it is not necessary that any candidate possess them all.

* Python
* TurboGears
* SQL Databases (especially Postgresql)
* Xapien
* AMQP
* Linux
* Pyramid
* Javascript

A bachelors degree or greater in Computer Science is preferred for this role, although, not necessary if there is extensive expertise in the desired areas for this position.

Candidates would not necessarily be required to relocate for this position.
If you're interested, please apply on the website, and send me an email telling me why you'd be an awesome fit for this position. I am the hiring manager for this position, so you would get the added bonus of working for me. ;)

Thomas Vander Stichele: Collabora and Fluendo collaborate fluently!

Mon, 27 Feb 2012 12:01:14 +0000

Well, this sure has been a long time in the making.

Fluendo and Collabora have a checkered past which I won’t get into, but on paper it has always made sense for these two companies to collaborate and making GStreamer work commercially. One company specializes in products, the other in consulting (I’m sure you can figure out which is which), and complement each other perfectly to make GSstreamer more successful commercially.

I personally have always believed that we need to get GStreamer to other platforms and make them as easy to use as possible. Windows was an obvious target in the past, and now Android is another. There is a big difference between a successful open source project, and a commercially successful one. Flumotion’s Andoni Morales who came with me to the GStreamer 0.11 hackfest in Malaga is going to be working on this one SDK to rule them all.

Christian beat me to it in the blogosphere, but the word is now officially out! Feel free to read Fluendo’s press release.

Pete Zaitcev: The other reST

Sat, 25 Feb 2012 00:21:04 +0000

I was moonlighting a bit as a technical writer, and run into an odd issue. In Swift SAIO doc we have a piece that comes out like this:

[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = changeme

Why is "hash" red?

My reading of the documentation for reST suggests nothing. If I try to escape the dash with a backslash, color disappears, but the backslash leaks into the HTML.

To make matters worse, we cannot just spit on it all and re-code everything in MD, because Sphinx is quite well entrenched.

UPDATE: trying bug 797425. As it turned out, the problem is that Sphinx treats the document as Python code and highlights accordingly.

Dave Jones: Weekly Fedora kernel bug statistics.

Fri, 24 Feb 2012 21:35:27 +0000

No time for a complete breakdown this week. (I’ll post more about some of the more interesting bugs in a few days).

	F15	F16	F17	Rawhide
open:	373	541	11	141	(1063)
closed since 2012-02-24:	4	42	2	13
changed since 2012-02-24:	19	86	9	17

You can definitely see some things sticking out on the F16 changes. Ignoring the usual compat-wireless bugs, there are a lot of new oopses, and more worryingly a lot of ‘bad page map’ bugs. More on this next week.

Weekly Fedora kernel bug statistics. is a post from: codemonkey.org.uk

Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17 The 3.2 rebase for F15 got pushed to updates. Not...
Fedora rawhide bugzilla status report from 2012-02-03 to 2012-02-10 The rawhide kernel is continuing to rebase towards 3.3. (currently...
Fedora kernel bug status reports. Going to start trying a new thing. Every Friday, I’ll...

Related posts brought to you by Yet Another Related Posts Plugin.

Dave Jones: Fedora master branch statistics.

Fri, 24 Feb 2012 21:13:19 +0000

I did another checkout of master/ and run fedpkg prep over it for a few days.
I had to restart it a few times, after it fell over on a few packages that didn’t pass ‘fedpkg prep’ when run with an interactive session.
(This was enough of a problem that eventually I ran fedpkg prep

Some nerd stats.

132G of disk space used.
11249 packages.
5509839 files.

458671 .c files (LOC: 261851028)
56547 .cc files (LOC: 16995428)
224509 .cpp files (LOC: 86758698)
510207 .h files (LOC: 89604481)
109383 .py files (LOC: 26341574)
29405 .sh files (LOC: 22613899)
13473 .pl files (LOC: 2819156)
12094 .rb files (LOC: 2162266)
6734 .el files (LOC: 4451797)
5923 .hs files (LOC: 1198194)

Points of note:

People coding in C++ really like typing.
There might be twice as many ruby files as there are emacs lisp, but elisp has twice the linecount. (Probably all those brackets)

As an aside, the number of packages that contain files that have bad perms (000 ?!), or dangling symlinks surprised me.

Oh, and the total number of lines of text ? 2058188485. (It took almost four hours to count)

Fedora master branch statistics. is a post from: codemonkey.org.uk

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Mark J Cox: Enterprise Linux 5.7 to 5.8 risk report

Tue, 21 Feb 2012 14:52:00 +0000

Red Hat Enterprise Linux 5.8 was released today (February 2012), seven months since the release of 5.7 in July 2011. So let's use this opportunity to take a quick look back over the vulnerabilities and security updates made in that time, specifically for Red Hat Enterprise Linux 5 Server.

Red Hat Enterprise Linux 5 is coming up to its fifth year since release, and is supported for another five years, until 2017.

Errata count

The chart below illustrates the total number of security updates issued for Red Hat Enterprise Linux 5 Server if you had installed 5.7, up to and including the 5.8 release, broken down by severity. It's split into two columns, one for the packages you'd get if you did a default install, and the other if you installed every single package (which is unlikely as it would involve quite a bit of manual effort to select every one). For a given installation, the number of package updates and vulnerabilities that affected you will depend on exactly what packages you have installed or removed.

So, for a default install, from release of 5.7 up to and including 5.8, we shipped 42 advisories to address 118 vulnerabilities. 4 advisories were rated critical, 13 were important, and the remaining 25 were moderate and low.

Or, for all packages, from release of 5.7 up to and including 5.8, we shipped 71 advisories to address 177 vulnerabilities. 7 advisories were rated critical, 16 were important, and the remaining 48 were moderate and low.

Critical vulnerabilities

The 7 critical advisories addressed 20 critical vulnerabilities across 4 different packages:

An update to OpenJDK 6 Java Runtime Environment, (October 2011) where a web site hosting a malicious Java applet could potentially run arbitrary code as the user.

An update to the MIT krb5 telnet daemon (December 2011) where a remote attacker who can access the telnet port of a target machine could use this flaw to execute arbitrary code as root. Note that the krb5 telnet daemon is not installed or enabled by default, and the default firewall rules block remote access to the telnet port. This flaw did not affect the more commonly used telnet daemon distributed in the telnet-server package.

Updates to PHP and PHP 5.3 (February 2012) where a remote attacker could send a specially-crafted HTTP request to cause the PHP interpreter to crash or, possibly, execute arbitrary code. This flaw was caused by the fix for CVE-2011-4885.

Three updates to Firefox (August 2011, September 2011, November 2011) where a malicious web site could potentially run arbitrary code as the user running Firefox.

Updates to correct 19 out of the 20 critical vulnerabilities were available via Red Hat Network either the same day or the next calendar day after the issues were public. The update to krb5 took 2 calendar days because it was public on Christmas day.

Overall, for Red Hat Enterprise Linux 5 since release until 5.8, 98% of critical vulnerabilities have had an update available to address them available from the Red Hat Network either the same day or the next calendar day after the issue was public.

Other significant vulnerabilities

Although not in the definition of critical severity, also of interest during this period were a couple of remote denial of service flaws that were easily exploitable:

A flaw in BIND, CVE-2011-4313, fixed by RHSA-2011:1458 (bind) and RHSA-2011:1459 (bind97). A remote attacker could use this flaw to cause BIND to crash.

A flaw in Apache HTTP Server, CVE-2011-3192, fixed by RHSA-2011:1245. A remote attacker could use this flaw to cause httpd to use an excessive amount of memory and CPU time.

In addition, updates to Firefox, NSS, and Thunderbird were made to blacklist a compromised Certificate Authority.

Previous update releases

To compare these statistics with previous update releases we need to take into account that the time between each update release is different. So looking at a default installation and calculating the number of advisories per month gives the following chart:

This data is interesting to get a feel for the risk of running Enterprise Linux 5 Server, but isn't really useful for comparisons with other major versions, distributions, or operating systems -- for example, a default install of Red Hat Enterprise Linux 4AS did not include Firefox, but 5 Server does. You can use our public security measurement data and tools, and run your own custom metrics for any given Red Hat product, package set, timescales, and severity range of interest.

See also: 5.7, 5.6, 5.5, 5.4, 5.3, 5.2, and 5.1 risk reports.

Dave Jones: Fedora 16 kernel bugzilla status report from 2012-02-10 to 2012-02-17

Fri, 17 Feb 2012 20:56:00 +0000

494 bugs open in total. (up from 378 last week)
66 bugs closed.

Interesting closures:

Another couple dozen wireless driver dupes. The most commonly reported bug is still 768639: WARNING: at /builddir/build/BUILD/kernel-3.1.fc17/compat-wireless-2011-12-01/drivers/net/wireless/ath/ath9k/rc.c:697 ath_rc_get_highest_rix+0×158/0x1f0 [ath9k]()

Second most common source of dupe bugs this week was the i915 driver, which grew several new problems.
772886: WARNING: at drivers/gpu/drm/i915/intel_display.c:953 intel_disable_pipe+0×120/0×150 [i915]()
790701: WARNING: at drivers/gpu/drm/i915/intel_dp.c:344 intel_dp_check_edp+0×65/0xb0 [i915]()
790702: WARNING: at drivers/gpu/drm/i915/intel_dp.c:1006 ironlake_edp_panel_vdd_on+0×197/0x1a0 [i915]()

739499: kernel-3.1.0-0.rc6.git0.3.fc16.x86_64 won’t boot on EC2
Fix picked up in the 3.2 rebase.

We saw a few more dupes of the sysfs link remove warning.
This is queued up for the next update already.

In last week’s f16 report, under ‘totally weird shit’, I pointed at bug 787527: kernel BUG at mm/mmap.c:2378!. At the time I had no idea what was happening. Over the week, we got several more reports. Hugh Dickins chased this down to a locking bug in the transparent huge pages code. (upstream thread).
We’ll pull in the final fix for that in the next update.

We’ve had a number of bugs reported from the soft lockup detector firing. When this happens, the traces in a lot of cases don’t make a lot of sense.
The common thing seems to be that they are all using some form of virtualisation. Here’s one from vmware for example (though our first f17 kernel bug is the same problem, but in qemu). For now, booting guests with nosoftlockup is probably the best we can do. There is some work ongoing upstream to better handle this situation.
TODO: Go through all the rest of the soft lockup bugs and see if any of them are the same problem. (likely).

772649: Frequency not scaling on demand – Sandy Bridge
We’ve seen all kinds of power management disasters on sandybridge systems.
From the ongoing i915 rc6 fiasco, to BIOS bonghits that take away P-states when things get too hot.
I suspect it’s all related.

790097: your kernel is tainted by flags
We got so many tainted bug reports that we don’t care about automatically filed by abrt, we had the abrt guys put in a dialog explaining to users that it wasn’t going to file bugs.
So naturally, users have started filing them by hand. Derp.

Just like in F15, we got a bunch more reports of the sd_revalidate_disk bug.
The fix for which is going to be in next weeks update.

91 still-open bugs got filed, or changed in some way.. Of those, here’s some of the more interesting ones.

428555: Soft lockup while doing load_policy
A very old SELinux bug, where loading the policy takes a really long time.
A cond_resched would silence the soft lockup detector, but I’m really curious why it’s taking 22 seconds to load a policy.
Something really doesn’t add up here. AFAIK, this is the only report we’ve ever had of a policy load taking this long.

593035: mount.nfs: page allocation failure. order:4, mode:0xc0d0
The new NFS idmapper code should fix this problem, but is only just getting tested in f17.
Once it’s proven itself there, we’ll look at backporting whatever is necessary to 16. (f15 is likely to be EOL at that point).
This will require userspace updates, which is another reason it won’t be happening in f15.

There were a bunch more irqpoll bugs reported. Still no resolution on the automatic fallback-to-polling idea upstream.

assorted wireless:
746744: Can not connect to PEAP using Intel Corporation WiFi Link 5100
755370: ath9k stability issues
767855: Wifi performance issues (Tx aggregation enabled on ra=MAC)
768639: WARNING: at /builddir/build/BUILD/kernel-3.1.fc17/compat-wireless-2011-12-01/drivers/net/wireless/ath/ath9k/rc.c:697 ath_rc_get_highest_rix+0×158/0x1f0 [ath9k]()
770484: WARNING: at /builddir/build/BUILD/kernel-3.1.fc16/compat-wireless-3.2-rc6-3/drivers/net/wireless/iwlwifi/iwl-trans-pcie-tx.c:739 iwl_enqueue_hcmd+0x5c8/0x5f0 [iwlwifi]()
770595: WARNING: at /builddir/build/BUILD/kernel-3.1.fc16/compat-wireless-3.2-rc6-3/drivers/net/wireless/iwlwifi/iwl-trans-pcie-rx.c:461 iwl_irq_tasklet+0x3bd/0x7c0 [iwlwifi]()
773513: Wifi wireless network connection abruptly stop working
773652: [ath9k] randomly disconnects wireless[AR9285] — lenovo g475
785422: Wireless fails after kernel update to kernel 3.2.2.1 pae
785561: 3.2.5-3.fc16.x86_64/X53S/K53SV iwlwifi runs like a sloth compared to ath9k
785913: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/drivers/net/wireless/iwlwifi/iwl-agn-tx.c:396 iwlagn_tx_skb+0x98d/0xa10 [iwlwifi]()
786609: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/include/net/mac80211.h:3618 rate_control_send_low+0x23e/0×250 [mac80211]()
787649: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/drivers/net/wireless/brcm80211/brcmsmac/main.c:7998 brcms_c_wait_for_tx_completion+0×99/0xb0 [brcmsmac]()
788012: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/net/mac80211/driver-ops.h:10 ieee80211_bss_info_change_notify+0x28a/0×290 [mac80211]()
789605: rtl8192cu: After 5~6 minutes, wireless usb lancard doesn’t work (cannot connect internet).
789159: network connection failure
790810: ath5k port gets “hard blocked” when Wireless is disabled via NetworkManager
794710: ath9k: Cannot enable WiFi in gnome-shell (toggle button switches back to Off state)
790275: BUG: unable to handle kernel NULL pointer dereference at 0000000000000060 rtl92ce_get_desc()

ethernet:
625776: e1000e crashes with Intel 82574L
720207: Realtek rtl8188ce works slow: speed is around 1Mb/s
794788: Wake-On-LAN stopped working after upgrade from FC15 to FC16
781217: crash after unplugging DSL cable (atl1c?)

suspend/hibernate:
783032: Can’t suspend to RAM when IR dongle is allowed to wake
788433: Core i7 cannot pm-hibernate/pm-suspend/thaw properly
789699: suspend fails by instant resume
791149: System can’t be suspended with kernel 3.2.x
791267: System reboots immediately after hibernating with 3.2 kernels
794525: 3.2.6-3.fc16.x86_64 doesnt suspend
789708: Hibernating fails all time
767084: kernel crash after back from sleep

boot failures:
789536: Can’t boot into new kernel
789679: kernel 3.2.3-2, 3.2.5-3 won’t boot encrypted setup
791133: Fedora 16 doesn’t boot with 3.2.6-3.fc16.x86_64 kernel on my notebook

Misc oopses/warn_on’s/scary shit:
721127: Heavy disk I/O (MD RAID?) crashes or freezes Fedora 15
787862: WARNING: at fs/sysfs/inode.c:323 sysfs_hash_and_remove+0xa9/0xb0()
788706: WARNING: at block/genhd.c:1568 disk_clear_events+0×106/0×110()
791277: WARNING: at kernel/watchdog.c:241 watchdog_overflow_callback+0x9b/0xa6()
794692: WARNING: at fs/dcache.c:2485 prepend_path+0x18c/0x1a0()
789990: BUG: unable to handle kernel paging request at 000000011b02e000 vmap_page_range_noflush()
790013: BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 path_lookupat()
794531: BUG: unable to handle kernel NULL pointer dereference at (null) sock_init_data()
769576: WARNING: at lib/list_debug.c:56 __list_del_entry+0×82/0xd0()
770581: WARNING: at kernel/softirq.c:159 local_bh_enable_ip+0x7a/0xa0()
771794: kernel: general protection fault: 0000 [#1] SMP
772738: system crash
755334: Kernel freezes, loops audio, under moderate cpu load
788981: map_vm_area() BUG: unable to handle kernel paging request at 000000011b02e000
789993: BUG: Bad page state in process tar pfn:27d36
794639: BUG: Bad page map in process firefox pte:02126065 pmd:19f00067

btrfs:
789632: WARNING: at fs/btrfs/extent-tree.c:5985 btrfs_alloc_free_block+0×354/0×360 [btrfs]()
790297: kernel BUG at fs/btrfs/transaction.c:1337!
790232: untarring incredibly slow over NFS even worse with BTRFS export

Brightness changing seems broken:
702352: Brightness adjustment FN keys doesn’t work
784532: kernel-3.2.1-3.fc16.x86_64 seems to break settings-screen- brightness slider control so disappears
788675: acpi_video_device_lcd_get_level_current() BUG: unable to handle kernel NULL pointer dereference at 0000000000000009
789962: Cannot adjust the brightness of the display in my laptop by pressing the function key

Fedora 16 kernel bugzilla status report from 2012-02-10 to 2012-02-17 is a post from: codemonkey.org.uk

Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10 Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10 Fedora...
Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17 The 3.2 rebase for F15 got pushed to updates. Not...
Fedora 15 bugzilla status report from 2012-02-03 to 2012-02-10 The Fedora 15 kernel recently got rebased to 3.2 (named...

Related posts brought to you by Yet Another Related Posts Plugin.

Dave Jones: Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17

Fri, 17 Feb 2012 16:56:58 +0000

The 3.2 rebase for F15 got pushed to updates. Not a huge amount of post-update activity.

378 bugs open in total. (down from 389 last week)
9 bugs closed.

Interesting closures:
445757: name_count maxed, losing inode data messages in dmesg
Purely cosmetic, but very annoying for people who saw it. audit periodically caused these messages to be spewed:
audit: name_count maxed, losing inode data: dev=00:07, inode=735975
Audit was using a fixed length array to store inodes. The message above got printed whenever something caused enough filesystem activity to create 20 or more inodes in a single syscall. (Something like loading a module could create a whole bunch of sysfs nodes for eg).
The fix was to dynamically allocate them when the array fills up. (The array was also shrunk to 5, which should be the common case). Pretty straight-forward, but for a multitude of reasons this took nearly two years to get merged upstream. Finally fixed in 3.3-rc1. This doesn’t cleanly apply to 3.2, and as it’s purely cosmetic, we’ll wait until we rebase to pick this up in f15/f16.

790982: Kernel crash when unplugging a USB key
Another dupe of the SCSI crash in sd_revalidate_disk that I mentioned last week.
This got mentioned upstream after other distros also started seeing it. Jun’ichi Nomura & Tejun Heo seem to have arrived at a conclusion, so that problem should be put to bed soon. We’ll backport this to F15/16 if it doesn’t appear in -stable first.

20 still-open bugs got filed, or changed in some way.
Of those, here’s some of the more interesting ones.

717211, 718886 & 715137: WARNING: at net/sched/sch_generic.c:256 dev_watchdog+0xe2/0×147
We continue to see these watchdog timeouts being reported for r8169/atl1c/e1000e. No progress.

720005: possible threading issue on s390x
After last week’s f15 status report, Simon Farnsworth noticed this bug, and thought it may have been related to a threading bug in python’s os.fork. That may be a problem, but it seems there is at least one non-python related deadlock too. There may be more than one bug here being confused as a single problem.

729460: Periodically typed zeros to every text-field
This is one of those “wtf is even going on here?” bugs. No idea yet.
(Personally I’ve had nothing but problems every time I’ve tried one of those wireless keyboards. I don’t know why people persist)

735380: WARNING: at fs/inode.c:901 unlock_new_inode+0x2e/0x4a()
The F15 variant of the unlock_new_inode bug. This has been driving us crazy for months now. It seems that for some users, after they resume from hibernate, they hit a condition where we try to unlock an inode which was never locked.
It smells like memory corruption of some kind, but tracking it down has proved to be very difficult.
It’s made worse by the fact that none of the developers chasing it can reproduce it.
We’re going to throw this patch in the next build to see if it makes any difference at all, but it feels like shooting in the dark.

787054: DMA: Out of SW-IOMMU space for 16 bytes
Reported against 3.1. Not reproduced on 3.2 yet. User has hit an ath9k WARN, which may or may not be related. Too early to say.

789080: WARNING: at lib/kref.c:34
Some reference count got corrupted after resume from suspend.

789659: limited network bandwidth
User reported that his download speeds were chopped in half in 3.2.
Being worked out upstream on the netdev list
What’s interesting about this bug, is that this user was the only one who noticed.

Fedora 15 kernel bugzilla status report from 2012-02-10 to 2012-02-17 is a post from: codemonkey.org.uk

Fedora rawhide bugzilla status report from 2012-02-03 to 2012-02-10 The rawhide kernel is continuing to rebase towards 3.3. (currently...
Fedora 15 bugzilla status report from 2012-02-03 to 2012-02-10 The Fedora 15 kernel recently got rebased to 3.2 (named...
Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10 Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10 Fedora...

Related posts brought to you by Yet Another Related Posts Plugin.

Tom Tromey: Quick Multi-process Debugging Update

Fri, 17 Feb 2012 15:12:23 +0000

In my last post I mentioned that setting breakpoints is a pain when debugging multiple processes in GDB. While there are some bugs here (we’re actively working on them), it isn’t hard to make the basic case work. In fact, there’s nothing to it. Some background…

Starting with GDB 7.4, we changed how basic breakpoint specifiers (called “linespecs”) work. Previously, a linespec applied somewhat randomly to the first matching symbol found in your code. This behavior probably made sense in 1989, when all you had were statically linked executables; but nowadays it is much more common to have dozens of shared libraries, with the attendant name clashes.

So, instead of having GDB guess which symbol you meant, now a breakpoint just applies to all of them. Our idea is that we’ll start supplying ways to narrow down exactly which spots you meant to name, say by adding syntax like “break libwhatever.so:function“, or whatever.

Anyway, this new work also applies across inferiors. Here’s an example of debugging “make“, then setting a breakpoint on a function in libcpp (which itself is linked into a sub-process of gcc):

(gdb) b _cpp_lex_direct
Function "_cpp_lex_direct" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (_cpp_lex_direct) pending.
(gdb) run
Starting program: /usr/bin/make
gcc -g -o crasher crasher.c
[New inferior 8761]
[New process 8761]
process 8761 is executing new program: /usr/bin/gcc
[New inferior 8762]
[New process 8762]
process 8762 is executing new program: /usr/libexec/gcc/x86_64-redhat-linux/4.6.2/cc1

Breakpoint 1, 0x0000000000b156a0 in _cpp_lex_direct ()

The remaining issues have to do with breakpoint re-setting not doing the right thing with running inferiors. This causes some scary warnings when running, but I think for the time being you can just ignore those.

Well, I should say those are the known issues. This feature hasn’t had as much use as I would like (judging from the low bug rate — I can’t tell if that is a good insight or a horrible realization). So, try it out and report problems to GDB Bugzilla. We’ll be making it work for you.

Tom Tromey: Debugging multiple programs at once

Tue, 14 Feb 2012 22:04:29 +0000

Consider this Makefile:

all: runit

runit: crasher
	./crasher

crasher: crasher.c
	gcc -g -o crasher crasher.c

And, here is the program it is building:

int *x = 0;

int main ()
{
  *x = 52;
}

Now, if you run “make“, eventually you will see a crash. But how to debug the crash?

Well, obviously, this is a trivial example so you’d just debug the program. But what if you had a complex script involving extensive and obscure initialization? Say, in your test suite? The traditional answer is logging plus cut and paste into gdb; or perhaps hacking an invocation of gdb --args into your script. Nowadays you can do better, though.

Let’s start by debugging make:

$ gdb -quiet make
Reading symbols from /usr/bin/make...(no debugging symbols found)...done.
Missing separate debuginfos, use: debuginfo-install make-3.82-8.fc16.x86_64

Now set things up for multi-inferior debugging:

(gdb) set detach-on-fork off
(gdb) set target-async on
(gdb) set non-stop on
(gdb) set pagination off

(Yes, it is silly how many settings you have to tweak; and yes, we’re going to fix this.)

Now do it:

(gdb) run
Starting program: /usr/bin/make
gcc -g -o crasher crasher.c
[New inferior 9694]
[New process 9694]
process 9694 is executing new program: /usr/bin/gcc
[New inferior 9695]
[New process 9695]
process 9695 is executing new program: /usr/libexec/gcc/x86_64-redhat-linux/4.6.2/cc1
Missing separate debuginfos, use: debuginfo-install gcc-4.6.2-1.fc16.x86_64
[Inferior 3 (process 9695) exited normally]
[Inferior 9695 exited]
Missing separate debuginfos, use: debuginfo-install cpp-4.6.2-1.fc16.x86_64
(gdb) [New inferior 9696]
[New process 9696]
process 9696 is executing new program: /usr/bin/as
[Inferior 4 (process 9696) exited normally]
[Inferior 9696 exited]
[New inferior 9697]
[New process 9697]
process 9697 is executing new program: /usr/libexec/gcc/x86_64-redhat-linux/4.6.2/collect2
Missing separate debuginfos, use: debuginfo-install binutils-2.21.53.0.1-6.fc16.x86_64
[New inferior 9698]
[New process 9698]
process 9698 is executing new program: /usr/bin/ld.bfd
Missing separate debuginfos, use: debuginfo-install gcc-4.6.2-1.fc16.x86_64
[Inferior 6 (process 9698) exited normally]
[Inferior 9698 exited]
[Inferior 5 (process 9697) exited normally]
[Inferior 9697 exited]
[Inferior 2 (process 9694) exited normally]
[Inferior 9694 exited]
./crasher
[New inferior 9699]
[New process 9699]
process 9699 is executing new program: /tmp/crasher
Missing separate debuginfos, use: debuginfo-install binutils-2.21.53.0.1-6.fc16.x86_64

Program received signal SIGSEGV, Segmentation fault.
0x000000000040047f in main () at crasher.c:5
5      *x = 52;

Cool stuff. Now you can inspect the crashed program:

(gdb) info inferior
Num  Description       Executable
  7    process 9699      /tmp/crasher
* 1    process 9691      /usr/bin/make
(gdb) inferior 7
[Switching to inferior 7 [process 9699] (/tmp/crasher)]
[Switching to thread 7 (process 9699)]
#0  0x000000000040047f in main () at crasher.c:5
5      *x = 52;

There is still a lot of work to do here — it is still a bit too slow, setting breakpoints is still a pain, etc. These are all things we’re going to be cleaning up in the coming year.

Lennart Poettering: /etc/os-release

Mon, 13 Feb 2012 18:46:00 +0000

One of the new configuration files systemd introduced is /etc/os-release. It replaces the multitude of per-distribution release files^[1] with a single one. Yesterday we decided to drop support for systems lacking /etc/os-release in systemd since recently the majority of the big distributions adopted /etc/os-release and many small ones did, too^[2]. It's our hope that by dropping support for non-compliant distributions we gently put some pressure on the remaining hold-outs to adopt this scheme as well.

I'd like to take the opportunity to explain a bit what the new file offers, why application developers should care, and why the distributions should adopt it. Of course, this file is pretty much a triviality in many ways, but I guess it's still one that deserves explanation.

So, you ask why this all?

It relieves application developers who just want to know the distribution they are running on to check for a multitude of individual release files.
It provides both a "pretty" name (i.e. one to show to the user), and machine parsable version/OS identifiers (i.e. for use in build systems).
It is extensible, can easily learn new fields if needed. For example, since we want to print a welcome message in the color of your distribution at boot we make it possible to configure the ANSI color for that in the file.

FAQs

There's already the lsb_release tool for this, why don't you just use that? Well, it's a very strange interface: a shell script you have to invoke (and hence spawn asynchronously from your C code), and it's not written to be extensible. It's an optional package in many distributions, and nothing we'd be happy to invoke as part of early boot in order to show a welcome message. (In times with sub-second userspace boot times we really don't want to invoke a huge shell script for a triviality like showing the welcome message). The lsb_release tool to us appears to be an attempt of abstracting distribution checks, where standardization of distribution checks is needed. It's simply a badly designed interface. In our opinion, it has its use as an interface to determine the LSB version itself, but not for checking the distribution or version.

Why haven't you adopted one of the generic release files, such as Fedora's /etc/system-release? Well, they are much nicer than lsb_release, so much is true. However, they are not extensible and are not really parsable, if the distribution needs to be identified programmatically or a specific version needs to be verified.

Why didn't you call this file /etc/bikeshed instead? The name /etc/os-release sucks! In a way, I think you kind of answered your own question there already.

Does this mean my distribution can now drop our equivalent of /etc/fedora-release? Unlikely, too much code exists that still checks for the individual release files, and you probably shouldn't break that. This new file makes things easy for applications, not for distributions: applications can now rely on a single file only, and use it in a nice way. Distributions will have to continue to ship the old files unless they are willing to break compatibility here.

This is so useless! My application needs to be compatible with distros from 1998, so how could I ever make use of the new file? I will have to continue using the old ones! True, if you need compatibility with really old distributions you do. But for new code this might not be an issue, and in general new APIs are new APIs. So if you decide to depend on it, you add a dependency on it. However, even if you need to stay compatible it might make sense to check /etc/os-release first and just fall back to the old files if it doesn't exist. The least it does for you is that you don't need 25+ open() attempts on modern distributions, but just one.

You evil people are forcing my beloved distro $XYZ to adopt your awful systemd schemes. I hate you! You hate too much, my friend. Also, I am pretty sure it's not difficult to see the benefit of this new file independently of systemd, and it's truly useful on systems without systemd, too.

I hate what you people do, can I just ignore this? Well, you really need to work on your constant feelings of hate, my friend. But, to a certain degree yes, you can ignore this for a while longer. But already, there are a number of applications making use of this file. You lose compatibility with those. Also, you are kinda working towards the further balkanization of the Linux landscape, but maybe that's your intention?

You guys add a new file because you think there are already too many? You guys are so confused! None of the existing files is generic and extensible enough to do what we want it to do. Hence we had to introduce a new one. We acknowledge the irony, however.

The file is extensible? Awesome! I want a new field XYZ= in it! Sure, it's extensible, and we are happy if distributions extend it. Please prefix your keys with your distribution's name however. Or even better: talk to us and we might be able update the documentation and make your field standard, if you convince us that it makes sense.

Anyway, to summarize all this: if you work on an application that needs to identify the OS it is being built on or is being run on, please consider making use of this new file, we created it for you. If you work on a distribution, and your distribution doesn't support this file yet, please consider adopting this file, too.

If you are working on a small/embedded distribution, or a legacy-free distribution we encourage you to adopt only this file and not establish any other per-distro release file.

Read the documentation for /etc/os-release.

Footnotes

[1] Yes, multitude, there's at least: /etc/redhat-release, /etc/SuSE-release, /etc/debian_version, /etc/arch-release, /etc/gentoo-release, /etc/slackware-version, /etc/frugalware-release, /etc/altlinux-release, /etc/mandriva-release, /etc/meego-release, /etc/angstrom-version, /etc/mageia-release. And some distributions even have multiple, for example Fedora has already four different files.

[2] To our knowledge at least OpenSUSE, Fedora, ArchLinux, Angstrom, Frugalware have adopted this. (This list is not comprehensive, there are probably more.)

Dave Jones: Fedora rawhide bugzilla status report from 2012-02-03 to 2012-02-10

Fri, 10 Feb 2012 22:59:31 +0000

The rawhide kernel is continuing to rebase towards 3.3. (currently at -rc3)
There are 149 open bugs right now.

Things are pretty quiet in rawhide right now. Perhaps things will pick up after the F17 alpha/beta’s start seeing more use, but at the moment there’s considerably more bug activity going on in the releases than the development branch.

In the last week, 6 got closed.

772772: rt2860 should now be working fine.
703118: Dell ST2220T touch screens should work
787373: PowerPC compile failure fixed
626026: Some rcu_deference_check warnings should now be fixed.
788125: usrmove related fallout.
785295: Watchdog overflow no longer seems to be occuring.

16 bugs got filed/changed in the last week.

783211: Cache inconsistency when reading from a partition vs the parent block_device
647429: GFS2: [RFE] Implement trimfs ioctl
636287: GFS2: [RFE] Make GFS2 handle errors more gracefully
785939: iwlwifi is spewing garbage in the logs, sometimes hangs
785772: Logs filled by ICMPv6 RA: ndisc_router_discovery() failed to add default route
696219: No sound input using snd-hda-intel
735641: Oops in kernel-3.1.0-0.rc4.git0.0.fc16.i686 while plugging Sony digital camera
645877: possible circular locking at dquot_commit
788064: reproducible kernel BUG at fs/btrfs/inode.c:1668!
748159: SD card reader not detected on ASUS notebook
759213: Truncated core dumps generated when piped through custom hook
789017: [abrt] kernel: BUG: MAX_LOCKDEP_ENTRIES too low!
787281: [abrt] kernel: WARNING: at fs/sysfs/inode.c:323 sysfs_hash_and_remove+0xa9/0xb0()
784089: (AutoFS) [abrt] kernel: [ INFO: possible recursive locking detected ] lock(&(&dentry->d_lock)->rlock);
787319: [abrt] kernel: [ INFO: possible recursive locking detected ] snd_pcm_action_group() lock(&(&substream->self_group.lock)->rlock/1)
769747: [sdb] Asking for cache data failed

Not really enough samples to start seeing patterns. Going through the older bugs and closing out stale entries is something that needs doing.

Fedora rawhide bugzilla status report from 2012-02-03 to 2012-02-10 is a post from: codemonkey.org.uk

Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10 Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10 Fedora...
Fedora 15 bugzilla status report from 2012-02-03 to 2012-02-10 The Fedora 15 kernel recently got rebased to 3.2 (named...
Fedora kernel bug status reports. Going to start trying a new thing. Every Friday, I’ll...

Related posts brought to you by Yet Another Related Posts Plugin.

Dave Jones: Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10

Fri, 10 Feb 2012 22:31:56 +0000

Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10

Fedora 16 is currently shipping 3.2.3, the latest stable kernel.
We currently have 472 bugs open against it.

In the last week, we closed out 63 bugs, and saw changes to 91 bugs that are still open.

Interesting closures:
Looking over the list, a lot of duplicates jump out. Many of these were filed by different people, but in some cases, abrt is just being dumb.

Most commonly reported bugs in the last week:
768639: [abrt] kernel: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/drivers/net/wireless/ath/ath9k/rc.c:697 ath_rc_get_highest_rix+0×158/0x1f0 [ath9k]()

784692: hso: WARNING: at net/sched/sch_generic.c:255 dev_watchdog+0×247/0×250()
Another bunch of reports from the net/sched/sch_generic WARN_ON where the transmit queue locks up.
possibly related: 785806: e1000e Detected Hardware Unit Hang

Looking at the 26 closed bugs that weren’t duplicates, there’s a few wireless bugs that got fixed in an update, a handful of things that turned out to be bad hardware or user misconceptions, and generally pretty boring stuff.

The list of still open bugs is more interesting, especially from a ‘pattern spotting’ perspective.

Wireless:
We get a ton of wireless bug reports these days.
It seems like we get a lot more since we switched to using compat-wireless snapshots.
785409: After update to kernel 3.2.2-1 Atheros wireless fails to connect if using WEP Security
746744: Can not connect to PEAP using Intel Corporation WiFi Link 5100
785413: New kernel release has compatibility issues with my wireless card
784824: no radio devices for kernel 3.2.1
716988: Ralink RT2573 USB dongle randomly overheats
786566: Rapid decline in wireless performance with Atheros AR9002WB-1NG
785721: Unable to connect to WLAN using Broadcom kernel driver when WPA key is too long
767855: Wifi performance issues (Tx aggregation enabled on ra=MAC)
773513: Wifi wireless network connection abruptly stop working
758543: Wireless disconnects under load on Acer Aspire One 150Aw (Atheros AR242x / AR542x using ath5k)
785422: Wireless fails after kernel update to kernel 3.2.2.1 pae
770595: WARNING: at /builddir/build/BUILD/kernel-3.1.fc16/compat-wireless-3.2-rc6-3/drivers/net/wireless/iwlwifi/iwl-trans-pcie-rx.c:461 iwl_irq_tasklet+0x3bd/0x7c0 [iwlwifi]()
773205: WARNING: at /builddir/build/BUILD/kernel-3.1.fc16/compat-wireless-3.2-rc6-3/include/net/mac80211.h:3574 rs_get_rate+0x1c1/0x1d0 [iwlwifi]()
783822: WARNING: at /builddir/build/BUILD/kernel-3.1.fc16/compat-wireless-3.2-rc6-3/net/wireless/mlme.c:366 cfg80211_send_assoc_timeout+0xb8/0×150 [cfg80211]()
768639: WARNING: at /builddir/build/BUILD/kernel-3.1.fc17/compat-wireless-2011-12-01/drivers/net/wireless/ath/ath9k/rc.c:697 ath_rc_get_highest_rix+0×158/0x1f0 [ath9k]()
787649: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/drivers/net/wireless/brcm80211/brcmsmac/main.c:7998 brcms_c_wait_for_tx_completion+0×99/0xb0 [brcmsmac]()
786609: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/include/net/mac80211.h:3618 rate_control_send_low+0x23e/0×250 [mac80211]()
788012: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/net/mac80211/driver-ops.h:10 ieee80211_bss_info_change_notify+0x28a/0×290 [mac80211]()
789235: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/include/net/mac80211.h:3618 rate_control_send_low+0x23e/0×250 [mac80211]()
785913: WARNING: at /builddir/build/BUILD/kernel-3.2.fc16/compat-wireless-3.3-rc1-2/drivers/net/wireless/iwlwifi/iwl-agn-tx.c:396 iwlagn_tx_skb+0x98d/0xa10 [iwlwifi]()
788271: [ath9k] wireless connection drops off with newer kernels
773652: [ath9k] randomly disconnects wireless — lenovo g475
789159: network connection failure

There are a number of bugs that look all related, that take the form “irq happened, nobody cared”. Many of these seem to be using the ASM108x chipset.
Searching for irqpoll turns up a lot of these
There’s some work going on upstream to fall back to polling automatically. We’ll pick this up and backport it.

Suspend/Resume:
788433: Core i7 cannot pm-hibernate/pm-suspend/thaw properly
785384: hibernate hangs since kernel version 3.2.2-1.fc16.i686
754043: Thinkpad R61 sometimes fails to suspend (regression)
787467: Virt is very slow after suspend / resume

Sound:
785417: snd_hda_intel – no sound
786243: Subwoofer in ASUS G73 does not work
757375: microphone does not work in Samsung N110 netbook (regression from 2.6.38.6-26)
785329: setup_bdle() BUG: unable to handle kernel NULL pointer dereference at 0000000000000028
788978: snd_vortex_dev_free() BUG: unable to handle kernel NULL pointer dereference at 00000000000000f0

USB:
746914: bug in ehci_hcd module
789066: WARNING: at drivers/usb/host/xhci-ring.c:472 xhci_find_new_dequeue_state+0xb5/0x1f0(
787533: WARNING: at drivers/usb/host/ehci-hcd.c:1178 ehci_endpoint_reset+0xee/0×100()
This last one is interesting. We also saw this on F15.
From discussion with Alan Stern upstream, this appears to be hplip doing something wrong, and always trying to clear halt status on an endpoint, even if it hasn’t been set.

SCSI:
754518: oops in sd_revalidate_disk
SCSI patch goes unapplied upstream. Film at 11. The number of hotplug related crashes in SCSI continues to amaze. Things really started going to shit after circa 2.6.37

Soft lockups.
789002: BUG: soft lockup – CPU#0 stuck for 23s! [kworker/u:0:2600]
Looks like the ath9k chip locked up, and the ioread never completed. Nasty.
756542: BUG: soft lockup – CPU#2 stuck for 23s! [btrfs-endio-0:440]
788279: BUG: soft lockup – CPU#3 stuck for 23s! [flush-254:0:1124]
IO related stalls.
788938: BUG: soft lockup – CPU#3 stuck for 22s! [systemd-stdout-:563]
This one is weird. Systemd tried to open something, it blocked for 22 seconds.
788620: BUG: soft lockup – CPU#1 stuck for 30s! [kworker/1:1:9884]
atl1c locked up while checking link status. This seems to be a common thing for network drivers.

Networking:
759063: WARNING: at net/ipv4/tcp.c:1485 tcp_recvmsg+0x1bb/0x7f1()
Back traces and warnings from the network layer always give me the creeps.
Not entirely sure what’s happening here yet, or even if it’s reproducable.

Totally weird shit:
788981: map_vm_area() BUG: unable to handle kernel paging request at 000000011b02e000
788770: get_from_free_list() BUG: unable to handle kernel paging request at 01010154
743589: general protection fault: 0000 [#1] SMP : TAINTED ——-D
783335: kernel BUG at mm/huge_memory.c:1921!
787527: kernel BUG at mm/mmap.c:2378!
752175: WARNING: at block/genhd.c:1560 disk_clear_events+0xc6/0xfa()
788280: WARNING: at lib/list_debug.c:53 __list_del_entry+0xa1/0xd0()
787044: WARNING: at lib/list_debug.c:47 __list_del_entry+0×63/0xd0()
787171: WARNING: at fs/dcache.c:2458 prepend_path+0x18c/0x1a0()
788706: WARNING: at block/genhd.c:1568 disk_clear_events+0×106/0×110()

Power management:
675433: Thinkpad X201 overheats and shuts down; fan does not spin up to maximum although capable
786665: Sandybridge laptop seeing lots of wakeups
772730: Kernel panic after ACPI power event with x86_64 kernel
787333: On netbook Acer Aspire One 521 battery indicator is absent.

Virtualisation:
739499: kernel-3.1.0-0.rc6.git0.3.fc16.x86_64 won’t boot on EC2
769300: Strange xen kernel code when installing a vm
787403: WARNING: at arch/x86/xen/mmu.c:475 xen_make_pte+0×67/0xa0()

Misc:
789215: led_trigger_unregister() BUG: unable to handle kernel NULL pointer dereference at (null)
This looks like the first variant of the bug we saw in f15 bug 726983.

787655: Automatic switching from graphical environment to a virtual screen without requested
768153: Bluetooth usage freezes system.
787468: boot fails by timeout while activating RAIDs with many HDDs
788509: Can’t change brightness on Acer Aspire One 521.
786164: CD/DVD-Rom stop working after a while
758789: cgroup controller test failed on F16 PPC64
787483: Desktop and applications really slow when switching between users
770704: External hdd can’t be accessed via esata (/dev/sdb disappears on access)
751060: F16 won’t power off after shutdown
789282: In the time of high % iowait all users interface are not responding also not responding tty console.
736752: INFO: suspicious rcu_dereference_check() usage in IPoIB code
788316: Kernel breaks working modem
788626: kernel BUG: unable to handle kernel NULL pointer dereference at (null) nfs4 code
785033: Kernel panic (rcu_bh_state detected stall)
784532: kernel-3.2.1-3.fc16.x86_64 seems to break settings-screen- brightness slider control so disappears
754859: Load moduale be2net failed
788748: Webcam recognized but not working on sony vaio sa3
783561: pti_exit() BUG: unable to handle kernel NULL pointer dereference at (null)
788675: acpi_video_device_lcd_get_level_current() BUG: unable to handle kernel NULL pointer dereference at 0000000000000009
789068: xc4000_sleep() BUG: unable to handle kernel NULL pointer dereference at 0000000000000080
747927: [RV710] Xenta Wireless 2.4G mouse looses connection

Fedora 16 bugzilla status report from 2012-02-03 to 2012-02-10 is a post from: codemonkey.org.uk

Fedora 15 bugzilla status report from 2012-02-03 to 2012-02-10 The Fedora 15 kernel recently got rebased to 3.2 (named...
Fedora kernel bug status reports. Going to start trying a new thing. Every Friday, I’ll...

Related posts brought to you by Yet Another Related Posts Plugin.