For a long time now, fellow-Gentoo’ers have had to edit /etc/asound.conf or ~/.asoundrc to make programs that talk directly to ALSA go through PulseAudio. Most other distributions ship configuration that automatically probes to see if PulseAudio is running and use that if avaialble, else fall back to the actual hardware. We did that too, but the configuration wasn’t used, and when you did try to use it, broke in mysterious ways.

I finally got around to actually figuring out the problem and fixing it, so if you have custom configuration to do all this, you should now be able to remove it after emerge’ing media-plugins/alsa-plugins-1.0.25-r1 or later with the pulseaudio USE flag. With the next PulseAudio bump, we’ll be depending on this to make the out-of-the-box experience a lot more seamless.

This took much longer to get done than it should have, but we’ve finally caught up. :)

[Props to Mart Raudsepp (leio) for prodding me into doing this.]

Arun put an awesome article up, detailing how PulseAudio compares to Android's AudioFlinger in terms of power consumption and suchlike. Suffice to say, PulseAudio rocks, but go and read the whole thing, it's worth it.

Apparently, AudioFlinger is a great choice if you want to shorten your battery life.

I’ve been meaning to try this for a while, and we’ve heard a number of requests from the community as well. Recently, I got some time here at Collabora to give it a go — that is, to get PulseAudio running on an Android device and see how it compares with Android’s AudioFlinger.

The Contenders

Let’s introduce our contenders first. For those who don’t know, PulseAudio is pretty much a de-facto standard part of the Linux audio stack. It sits on top of ALSA which provides a unified way to talk to the audio hardware and provides a number of handy features that are useful on desktops and embedded devices. I won’t rehash all of these, but this includes a nice modular framework, a bunch of power saving features, flexible routing, and lots more. PulseAudio runs as a daemon, and clients usually use the libpulse library to communicate with it.

In the other corner, we have Android’s native audio system — AudioFlinger. AudioFlinger was written from scratch for Android. It provides an API for playback/recording as well as a control mechanism for implementing policy. It does not depend on ALSA, but instead allows for a sort of HAL that vendors can implement any way they choose. Applications generally play audio via layers built on top of AudioFlinger. Even if you write a native application, it would use OpenSL ES implementation which goes through AudioFlinger. The actual service runs as a thread of the mediaserver daemon, but this is merely an implementation detail.

Note: all my comments about AudioFlinger and Android in general are based on documentation and code for Android 4.0 (Ice Cream Sandwich).

The Arena

My test-bed for the tests was the Galaxy Nexus running Android 4.0 which we shall just abbreviate to ICS. I picked ICS since it is the current platform on which Google is building, and hopefully represents the latest and greatest in AudioFlinger development. The Galaxy Nexus runs a Texas Instruments OMAP4 processor, which is also really convenient since this chip has pretty good support for running stock Linux (read on to see how useful this was).

Preparations

The first step in getting PulseAudio on Android was deciding between using the Android NDK like a regular application or integrate into the base Android system. I chose the latter — even though this was a little more work initially, it made more sense in the long run since PulseAudio really belongs to the base-system.

The next task was to get the required dependencies ported to Android. Fortunately, a lot of the ground work for this was already done by some of the awesome folks at Collabora. Derek Foreman’s androgenizer tool is incredibly handy for converting an autotools-based build to Android–friendly makefiles. With Reynaldo Verdejo and Alessandro Decina’s prior work on GStreamer for Android as a reference, things got even easier.

The most painful bit was libltdl, which we use for dynamically loading modules. Once this was done, the other dependencies were quite straightforward to port over. As a bonus, the Android source already ships an optimised version of Speex which we use for resampling, and it was easy to reuse this as well.

As I mentioned earlier, vendors can choose how they implement their audio abstraction layer. On the Galaxy Nexus, this is built on top of standard ALSA drivers, and the HAL talks to the drivers via a minimalist tinyalsa library. My first hope was to use this, but there was a whole bunch of functions missing that PulseAudio needed. The next approach was to use salsa-lib, which is a stripped down version of the ALSA library written for embedded devices. This too had some missing functions, but these were fewer and easy to implement (and are now upstream).

Now if only life were that simple. :) I got PulseAudio running on the Galaxy Nexus with salsa-lib, and even got sound out of the HDMI port. Nothing from the speakers though (they’re driven by a TI twl6040 codec). Just to verify, I decided to port the full alsa-lib and alsa-utils packages to debug what’s happening (by this time, I’m familiar enough with androgenizer for all this to be a breeze). Still no luck. Finally, with some pointers from the kind folks at TI (thanks Liam!), I got current UCM configuration files for OMAP4 boards, and some work-in-progress patches to add UCM support to PulseAudio, and after a couple of minor fixes, wham! We have output. :)

(For those who don’t know about UCM — embedded chips are quite different from desktops and expose a huge amount of functionality via ALSA mixer controls. UCM is an effort to have a standard, meaningful way for applications and users to use these.)

In production, it might be handy to write light-weight UCM support for salsa-lib or just convert the UCM configuration into PulseAudio path/profile configuration (bonus points if it’s an automated tool). For our purposes, though, just using alsa-lib is good enough.

To make the comparison fair, I wrote a simple test program that reads raw PCM S16LE data from a file and plays it via the AudioTrack interface provided by AudioFlinger or the PulseAudio Asynchronous API. Tests were run with the brightness fixed, wifi off, and USB port connected to my laptop (for adb shell access).

All tests were run with the CPU frequency pegged at 350 MHz and with 44.1 and 48 kHz samples. Five readings were recorded, and the median value was finally taken.

Round 1: CPU

First, let’s take a look at how the two compare in terms of CPU usage. The numbers below are the percentage CPU usage taken as the sum of all threads of the audio server process and the audio thread in the client application using top (which is why the granularity is limited to an integer percentage).

44.1 kHz		48 kHz
AF	PA	AF	PA
1%	1%	2%	0%

At 44.1 kHz, the two are essentially the same. Both cases are causing resampling to occur (the native sample rate for the device is 48 kHz). Resampling is done using the Speex library, and we’re seeing minuscule amounts of CPU usage even at 350 MHz, so it’s clear that the NEON optimisations are really paying off here.

The astute reader would have noticed that since the device’ native sample rate is 48 kHz, the CPU usage for 48 kHz playback should be less than for 44.1 kHz. This is true with PulseAudio, but not with AudioFlinger! The reason for this little quirk is that AudioFlinger provides 44.1 kHz samples to the HAL (which means the stream is resampled there), and then the HAL needs to resample it again to 48 kHz to bring it to the device’ native rate. From what I can tell, this is a matter of convention with regards to what audio HALs should expect from AudioFlinger (do correct me if I’m mistaken about the rationale).

So round 1 leans slightly in favour of PulseAudio.

Round 2: Memory

Comparing the memory consumption of the server process is a bit meaningless, because the AudioFlinger daemon thread shares an address space with the rest of the mediaserver process. For the curious, the resident set size was: AudioFlinger — 6,796 KB, PulseAudio — 3,024 KB. Again, this doesn’t really mean much.

We can, however, compare the client process’ memory consumption. This is RSS in kilobytes, measured using top.

44.1 kHz		48 kHz
AF	PA	AF	PA
2600 kB	3020 kB	2604 kB	3020 kB

The memory consumption is comparable between the two, but leans in favour of AudioFlinger.

Round 3: Power

I didn’t have access to a power monitor, so I decided to use a couple of indirect metrics to compare power utilisation. The first of these is PowerTOP, which is actually a Linux desktop tool for monitoring various power metrics. Happily, someone had already ported PowerTOP to Android. The tool reports, among other things, the number of wakeups-from-idle per second for the processor as a whole, and on a per-process basis. Since there are multiple threads involved, and PowerTOP’s per-process measurements are somewhat cryptic to add up, I used the global wakeups-from-idle per second. The “Idle” value counts the number of wakeups when nothing is happening. The actual value is very likely so high because the device is connected to my laptop in USB debugging mode (lots of wakeups from USB, and the device is prevented from going into a full sleep).

	44.1 kHz		48 kHz
Idle	AF	PA	AF	PA
79.6	107.8	87.3	108.5	85.7

The second, similar, data point is the number of interrupts per second reported by vmstat. These corroborate the numbers above:

	44.1 kHz		48 kHz
Idle	AF	PA	AF	PA
190	266	215	284	207

PulseAudio’s power-saving features are clearly highlighted in this comparison. AudioFlinger causes about three times the number of wakeups per second that PulseAudio does. Things might actually be worse on older hardware with less optimised drivers than the Galaxy Nexus (I’d appreciate reports from running similar tests on a Nexus S or any other device with ALSA support to confirm this).

For those of you who aren’t familiar with PulseAudio, the reason we manage to get these savings is our timer-based scheduling mode. In this mode, we fill up the hardware buffer as much as possible and go to sleep (disabling ALSA interrupts while we’re at it, if possibe). We only wake up when the buffer is nearing empty, and fill it up again. More details can be found in this old blog post by Lennart.

Round 4: Latency

I’ve only had the Galaxy Nexus to actually try this out with, but I’m pretty certain I’m not the only person seeing latency issues on Android. On the Galaxy Nexus, for example, the best latency I can get appears to be 176 ms. This is pretty high for certain types of applications, particularly ones that generate tones based on user input.

With PulseAudio, where we dynamically adjust buffering based on what clients request, I was able to drive down the total buffering to approximately 20 ms (too much lower, and we started getting dropouts). There is likely room for improvement here, and it is something on my todo list, but even out-of-the-box, we’re doing quite well.

Round 5: Features

With the hard numbers out of the way, I’d like to talk a little bit about what else PulseAudio brings to the table. In addition to a playback/record API, AudioFlinger provides mechanism for enforcing various bits of policy such as volumes and setting the “active” device amongst others. PulseAudio exposes similar functionality, some as part of the client API and the rest via the core API exposed to modules.

From SoC vendors’ perspective, it is often necessary to support both Android and standard Linux on the same chip. Being able to focus only on good quality ALSA drivers and knowing that this will ensure quality on both these systems would be a definite advantage in this case.

The current Android system leaves power management to the audio HAL. This means that each vendor needs to implement this themselves. Letting PulseAudio manage the hardware based on requested latencies and policy gives us a single point of control, greatly simplifying the task of power-management and avoiding code duplication.

There are a number of features that PulseAudio provides that can be useful in the various scenarios where Android is used. For example, we support transparently streaming audio over the network, which could be a handy way of supporting playing audio from your phone on your TV completely transparently and out-of-the-box. We also support compressed formats (AC3, DTS, etc.) which the ongoing Android-on-your-TV efforts could likely take advantage of.

Edit: As someone pointed out on LWN, I missed one thing — AudioFlinger has an effect API that we do not yet have in PulseAudio. It’s something I’d definitely like to see added to PulseAudio in the future.

Ding! Ding! Ding!

That pretty much concludes the comparison of these two audio daemons. Since the Android-side code is somewhat under-documented, I’d welcome comments from readers who are familiar with the code and history of AudioFlinger.

I’m in the process of pushing all the patches I’ve had to write to the various upstream projects. A number of these are merely build system patches to integrate with the Android build system, and I’m hoping projects are open to these. Instructions on building this code will be available on the PulseAudio Android wiki page.

For future work, it would be interesting to write a wrapper on top of PulseAudio that exposes the AudioFlinger audio and policy APIs — this would basically let us run PulseAudio as a drop-in AudioFlinger replacement. In addition, there are potential performance benefits that can be derived from using Android-specific infrastructure such as Binder (for IPC) and ashmem (for transferring audio blocks as shared memory segments, something we support on desktops using the standard Linux SHM mechanism which is not available on Android).

If you’re an OEM who is interested in this work, you can get in touch with us — details are on the Collabora website.

I hope this is useful to some of you out there!

As a part of the Ubuntu Hardware Summit, I held a presentation on the topic “audio debugging techniques”, focused on HDA Intel cards. I also wrote down some notes for some of those slides. I share the slides and the notes with the hope that you will find the information useful if you run into troubles with your audio hardware.

Audio stack overview

The audio stack can seem a bit complex, but first look at the line all the way from the applications to the hardware. This is the optimal audio path. If the audio path is different, complexity will increase and you might run into undesired behaviour, such as one application blocking another from playing audio. There are valid exceptions though – we have a separate sound server for professional, low-latency audio. But that’s outside the scope of this presentation.

Let’s start from the top. On the top we have different kinds of audio applications, which talk to PulseAudio. GStreamer is a library to help media playback, it can for example decode ogg and mp3 files. PulseAudio mixes these audio streams and send them down to the kernel. The ALSA library and the ALSA kernel core do not do much here but send the audio pointers through. The HDA controller driver is responsible for talking directly to the hardware, and so it sets up all necessary DMA streams between the HDA controller and memory. The HDA controller driver also talks to the HDA codec driver, which is different for every codec vendor.

As some of you probably know, between the HDA controller – which is a part of the southbridge in most computers – and the HDA codec, a special HDA bus is used. This means that the only way we can talk to the codec is through the controller.

Controlling audio volume goes the same path. When you use your volume control application, it controls PulseAudio’s volume. PulseAudio in turn modifies the volume controls being exposed by the kernel, and the kernel in turn talks to the hardware to set volume control registers on the codec. There are two levels of abstraction here: first, the kernel might choose not to expose all of the hardware’s volume controls, and second, PulseAudio exposes only one big volume control which is the sum of some of the volume controls the kernel exposes. So there is filtering on two levels.

Audio stack overview – codec

Let us have a look at the HDA codec chip and how its internals are represented to the driver. The codec is constructed as a graph, and on this slide one of the more simple HDA codec graphs is shown (just because it would fit the screen). A while ago upstream made a small program to extract this graph from the codec and make a picture of it. Thanks to Keng-Yü, who works for Canonical in Taipei, this tool is available as a package in Ubuntu 11.10. Just install the “codecgraph” package.

In this graph we have nodes correspondings to DACs, ADCs, mixers, and pins. In this example we can see what pins are connected to which DACs by following the solid line. The dotted line shows a connection that is possible but not currently active.

As the Linux codec driver code grows more intelligent, we depend more and more on this information to be accurate. This way we do not hard code as much in the driver, so we can adapt to future codecs without having to rewrite much code.
The information coming from the codec is usually correct. One problem we have from time to time is though, is that sometimes chip vendors add features which they choose not to document in this graph (and not in any other way either). There is a mechanism called “processing coefficients” in the specification, where the vendor can add its own functionality without telling anyone. When that happens, and it is required to use these undocumented “processing coefficients” to enable all inputs and outputs, we usually run into difficult problems that require vendor support to resolve.

Also, in some cases the graph cannot describe the functionality needed, e g if some hardware is depending on special pins on the codec. We need to know about this when it happens, so we can support it in the driver. So if you are a hardware designer, my message is: Try to use the standard way of doing things as much as possible. Do this and it will work out of the box on Linux, and likely other operating systems as well. If you do anything special, you’re causing headache for driver writers, possibly causing a slower time to market.
An example of this would be how you control external amplifiers: you can use the EAPD pins, which is the standard way, and you can use GPIO pins, ACPI, or anything else, that will be more problematic and require special driver support.

Pin configuration default

We also depend on information from the writers of BIOS/UEFI, i e the computer’s firmware. As a hardware designer, you have the freedom to choose which pins of the codec that go to what physical jack. You might decide that you want a digital out, or you decide that this machine should not have that functionality, and then you leave that pin unconnected.
Then the firmware engineer needs to know this, and program this into the codec when the computer boots. This is done by setting the “Pin Configuration Default” register. This register tells us not only the device type (headphone, mic, etc), but also the location (Internal, External, Docking Station), the color, and the channel mapping (to use for surround functionality).

Several years ago, we did not read this register much, but these days, we depend on that for all new computers for setting up the codec correctly. So what do we do if this register is wrong? Well, if we work with hardware pre-release, there might be a chance we can feed this information back to the firmware writers so they can correct the problem. If the hardware is already released, we have to create a “quirk”. This means that the driver overrides the firmware’s pin value(s) and instead uses its own value.

Because this value is so important, I’ve written an application where you can try out different combinations of this register.

Mixer problems

One of the most common problems with getting audio up and running on Linux is to make sure the mixer is correct. Typical symptoms of this would be that some outputs are working where others are not, or that there is something wrong with the volume control.

Here are some initial checks of these problems. We do this at the two levels of mixer abstraction. First, let’s have a look at the PulseAudio volume control. You can do that in Gnome’s volume control application.

Also, PulseAudio controls the volume of mixers at the ALSA level. You can see how this works by starting the alsamixer program. In this program, you can also see additional sliders, which you can also use to verify that they are in the correct to enable sound. You start alsamixer from a terminal (in Ubuntu the quickest way to launch a terminal is the Ctrl-Alt-T shortcut).

Mixer control names

So let’s look at these two abstraction levels in more detail and how you can inspect what is actually going on. First, let’s look at the codec level. If you are familiar with the codec’s nodes and how they are connected, e g by running “codecgraph”, you can also find out which ALSA level controls that are connected to which nodes on the codec. This is done by inspecting the “codec proc” file. Every codec in the system has this file, and its name is made up of the sound card name, and the codec’s address on the HDA bus. In this file, you can also see a lot of other information about the codec.

So next, we will also take a look at PulseAudio’s abstraction of these controls. This is done by looking at the files in /usr/share/pulseaudio/alsa-mixer. In this case, if we look at /usr/share/pulseaudio/alsa-mixer/paths/analog-output-headphones.conf, you can e g find the sections [Element Master] and [Element Headphones]. That means that the ALSA-level controls “Master” and “Headphones” are being merged in PulseAudio’s volume control when the “Headphones” port has been selected.

So these two places are the keys to understanding what is going on when you have mixer problems.

PCM/Streaming problems

So up next is when you have problems with the streaming. That is usually shown as the audio is breaking up, crackling or glitching. Unfortunately these problems are typically quite hard to resolve.

Sometimes this can be a bug in PulseAudio, or in the driver. But more often the problem is on either the application side or the hardware side.

If an application is not submitting data to PulseAudio in time, the PulseAudio has no audio to play back, so therefore playback breaks up. Once some more data has reached PulseAudio, it starts playback again, and so playback is started and stopped repeatedly.

The other problem could be with bad position reports from the hardware. PulseAudio depends on being able to ask the hardware for its current position at all times, and this should be sample accurate. You can test this by trying to run PulseAudio with timer scheduling disabled, in this case PulseAudio will rely more on DMA interrupts and less on position reports. However, this will also make PulseAudio draw more power than necessary from the machine, so please avoid this if you can.

When I try to debug these problems I usually start with making a PulseAudio verbose log. It often takes some knowledge and experience to be able to analyze this log though.

Jack sensing

Over the last six months or so, one of the things I’ve been working with is trying to get better jack detection handling, throughout the audio stack.
“Jack sensing” in this context means what to do when something has been plugged in, or unplugged.

When this happens, an interrupt (IRQ) is triggered and control is passed to the HDA codec driver. The driver takes the first action itself. Now, this is an area, unfortunately, when things differ a lot between different drivers, mostly between different vendors, but also between different chips of the same vendor, or even between configurations of the same chip.

But as a general rule, and for the most common vendors – that means Realtek, IDT and Conexant – these rules are the ones that are followed:

• For headphones – when you plug them in, the Internal Speakers are muted. Remember, this is still all at the kernel level.
• For what we’re doing with Line Outs – it’s not completely standardised everywhere yet, but it seems upstream is leaning on having Headphones mute Line Outs and having Line Outs mute Internal Speakers by default. Some drivers also have a special control where the automute behaviour can be changed.
• For Microphones – the only rule here is that if we have only one internal microphone and one external microphone, the external microphone takes over when you plug it in, and the internal microphone regains control when you unplug. Should there be any other inputs, e g two external mic jacks, or a line in jack, no autoswitching is done at the kernel level.

After this has been done, a signal is sent to userspace. Hopefully – this also varies between vendors. We’ll get back to that. What’s new in Ubuntu 11.10, is that this signal is being picked up by PulseAudio. This is important, because it enables PulseAudio, to switch port for volume control. So this means, when you press your media keys (or use the sound menu) to control your volume, you control your headphone’s volume when you have headphones plugged in, and your speakers’ volume when your headphones are unplugged.

So this not working properly, is one of the more common problems. I have written a small tool that helps you to debug whether this issue is in hardware or software. This tool is called “hda-jack-sense-test”. This program sends the “get pin sense” command to each codec and outputs the results. I actually had use for it earlier this week, and confirmed that it was a hardware issue: although the headphones were unplugged, the “get pin sense” command returned that the headphones were being plugged in and unplugged all the time.

If you can confirm that things are working at this level, you can also look in “Sound settings” to see if the port (this is known as a “connector”) is automatically switched whenever headphones – or microphone – is plugged in. If it is not, the most common cause is that kernel driver does not notify userspace correctly about that change.

HDMI/DisplayPort Audio

One of the most common problem with HDMI these days are with newer chips supporting more than one output. These outputs could be HDMI, DisplayPort or DVI (with audio supported through a DVI to HDMI adapter). NVidia has supported four outputs for quite some time and Intel has supported three. But usually, not all of these are actually connected on the board.
Now, the problem is: How do we know what pin to output to? And the answer is, that there is no good way to figure that out until something is actually plugged in.

If you remember me talking about the pin config default earlier, you would say that maybe the graphics chip could mark the pins not connected to anything. If this was done, it would be a great start (and if they are, we make use of it to hide the outputs that are marked as not connected), but unfortunately, more often than not, these pins are set up as all pins connected and present. So if you write firmware for internal or external graphics cards, please do set up these pins.

So if we don’t know, what do we do? Well, here’s also work in progress at the userspace level. First, PulseAudio has to probe how many ports there are. Then we can use the new jack detection feature, to determine what has actually been plugged in. I’m currently working on redesigning the sound settings dialog so that the ports that are not plugged in will be actually hidden from the dialog, and I hope this will land in Ubuntu 12.04 which will be released in April next year.

And a final note, just so you don’t forget it: For NVidia and ATI, they both require proprietary video drivers to enable HDMI and DisplayPort audio. The ATI driver used to have support for some of the cards in its open source driver, but this feature was recently removed because they had some problems with it.
Intel has no proprietary drivers at all, so there it works with the standard open source driver.

pulseaudio: Do any of our followers have any internal contact with Adobe? Perhaps you can push them to fix http://t.co/lbvD0VJz (by Colin)

For those of you who were interested but couldn’t make it to the GStreamer Conference this year, the cool folks at Ubicast have got the talk videos up (can be streamed or downloaded).

Among these is my talk about recent developments in the PulseAudio world.

Longish day, but I did want to post something fun before going to sleep — I just pushed out patches to hook up the WebRTC folks’ analog gain control to PulseAudio. So your mic will automatically adjust the input level based on how loud you’re speaking. It’s quite quick to adapt if you’re too loud, but a bit slow when the input signal is too soft. This isn’t bad, since the former is a much bigger problem than the latter.

Also, we’ve switched to the WebRTC canceller as the default canceller (you can still choose the Speex canceller manually, though). Overall, the quality is pretty good. I’d do a demo, but it’s effectively had zero learning time in my tests, so we’re not too far from a stage where this is a feature that, if we’re doing it right you won’t notice it exists.

There lot’s of things, big and small that need to be added and tweaked, but this does go some way towards bringing a hassle-free VoIP experience on Linux closer to reality. Once again, kudos to the folks at Google for the great work and for opening up this code. Also a shout-out to fellow Collaboran Sjoerd Simons for bouncing ideas and giving me those much-needed respites from talking to myself. :)

As I’d blogged about last week, we had a couple of Audio BoF sessions last week. Here’s a summary of what was discussed. I’ve collected items in relevance order rather than chronological order to make for easier reading. I think I have everything covered, I’ll update this post if one of the attendees points out something I missed or got wrong.

• Surround: There were a number of topics that came up with regards to multichannel/surround support:

  • There seems to be some lack of uniformity of channel maps, particularly in non-HDA hardware. While it was agreed that this should be fixed, we need some directed testing and bug reports to actually be able to fix this.

  • Multichannel mixers, while theoretically supported, are not actually exposed by any current drivers. It was suggested that these could be exposed without breaking existing applications by having new MC mixers exposed with device names corresponding to the surround PCM device (like “surround51″).

  • We need a way to query channel maps on a given PCM. This will be implemented as a new ALSA API which could be called after the PCM is opened. (AI: Takashi)

  • It would be good to have a way to configure the channel map as well (if supported by the hardware?). The suggestion was to do this as was done in PulseAudio, where an arbitrary channel map could be specified. (NB: is there hardware that supports multiple/arbitrary channel maps? If yes, how do we handle this?)

• Routing: Unsurprisingly, we came back to the problem of building a simplified mixer graph for PulseAudio.

  • The current status is that ALSA builds a simplified mixer for use by userspace, and PulseAudio further simplifies this by means of some name-based guessing.

  • PulseAudio would ideally like a simplified version of the original mixer graph, but something more complete than what we get now

  • However, since PulseAudio has fairly unique requirements of what information it wants, it probably makes sense to have ALSA provide the entire graph and leave the simplification task to PulseAudio (discussion on this approach continues)

  • There was no consensus on who would do this or how this should be done (creating a new serialisation format, exposing what HDA provides, adding node metadata to ALSA mixer controls, or something else altogether)

  • As an interim step, it was agreed that it would be possible to provide ordering in the simplified ALSA mixer (that is, add metadata to the control to signal what control comes “before” it and what comes “after” it). This should go some way in making the PA mixer simplification logic simpler and more robust. (AI: Takashi)

• HDMI: A couple of things came up in discussion about the status of HDMI.

  • There was a question about the reliability of ELD information as this will be used more in future versions of PulseAudio. There did not appear to be conclusive evidence in either direction, so we will probably assume that it is reliable and deal with reliability problems as they arise.

  • It was mentioned that it might be desirable to only expose the ALSA device if a receiver is plugged in. This had been mooted earlier as something to do in PulseAudio as an alternative. One thing to consider with this approach is dealing with a device switch on the receiver side. Doing this without a notification to userspace was generally agreed to be a bad idea.

• Jack detection: The long-standing debate on exposing jacks as input devices or ALSA controls came to a conclusion, with the resolution being that jacks would be exposed as ALSA controls. This requires a change in the kernel (and potentially alsa-lib?) which should not be too complex. Actual buttons (such as volume/mute control) will continue to be input devices. Once this is done, the pending jack detection patches will be adapted to use the new interface. (AI: Takashi (patches are in a branch already!), David)

• UCM: Another long-standing issue was the merging of the ALSA UCM patches into PulseAudio. Most of the problems thus far had been due to an incomplete understanding of how ALSA and PA concepts mapped to each other. Some consensus was arrived at in this regard after a lengthy discussion:

  • As is the case now, every ALSA PCM maps to a PA sink

  • Each UCM verb maps to a PA card profile

  • Each combination of UCM devices that can be used concurrently maps to a PA port

  • Each UCM modifier is mapped to an intended role on the corresponding sink

  The code should (as is in the patches currently submitted) be part of the PA ALSA module, and there will be changes required to use the UCM-specified mixer list instead of PA’s guessing mechanism. (AI: ???)

  (NB: It was mentioned that PulseAudio needs to support multiple intended roles for a sink/source. This is actually already supported — the intended roles property is a whitespace-separated list of roles)

  (NB2: There was further discussion with the Linaro folks this week about the UCM bits, and there’s likely going to be an IRC/phone gathering to clarify things further in the near future)

• GStreamer latency settings: We currently do not actually use PulseAudio’s power saving features from GStreamer for several reasons. Suggestions to over come this were mooted. While no definite agreement was reached, one suggestion was to add a “powersave” profile to pulsesink that chose higher latency/buffer-time values. Players would need to set this if they are not using features that break when these values are raised.

• Corking: The statelessness of current the corking mechanism was discussed in one session, and between the PulseAudio developers later. The problem is that we need to be able to track cork/uncork reasons more closely. This would give us more metadata that is needed to make policy decisions without breaking streams. Particularly, for example, if PA corks a music stream due to an incoming call, then the user plays, then pauses music, and then the call ends, we must not uncork the music stream. We intend to deal with this with two changes:

  • We need to add a per-cause cork/uncork request count

  • We need to associate a “generation” with cork/uncork requests, so certain conditions (such as user intervention) can bump the generation counter, and uncork requests corresponding to old cork requests will be ignored

  This will make it possible to track the various bits of state we need to do the right thing for cases like the one mentioned before.

So that’s that — lots of things discussed, lots of things to do! Thanks to everyone who came for participating.

At the Kernel Summit in Prague last week Kay Sievers and I lead a session on developing shared userspace libraries, for kernel hackers. More and more userspace interfaces of the kernel (for example many which deal with storage, audio, resource management, security, file systems or a number of other subsystems) nowadays rely on a dedicated userspace component. As people who work primarily in the plumbing layer of the Linux OS we noticed over and over again that these libraries written by people who usually are at home on the kernel side of things make the same mistakes repeatedly, thus making life for the users of the libraries unnecessarily difficult. In our session we tried to point out a number of these things, and in particular places where the usual kernel hacking style translates badly into userspace shared library hacking. Our hope is that maybe a few kernel developers have a look at our list of recommendations and consider the points we are raising.

To make things easy we have put together an example skeleton library we dubbed libabc, whose README file includes all our points in terse form. It's available on kernel.org:

The git repository and the README.

This list of recommendations draws inspiration from David Zeuthen's and Ulrich Drepper's well known papers on the topic of writing shared libraries. In the README linked above we try to distill this wealth of information into a terse list of recommendations, with a couple of additions and with a strict focus on a kernel hacker background.

Please have a look, and even if you are not a kernel hacker there might be something useful to know in it, especially if you work on the lower layers of our stack.

If you have any questions or additions, just ping us, or comment below!

For those who are in Prague for GstConf, LinuxCon, ELCE, etc. — don’t forget we’ve a couple of interesting audio-related things happening:

• Today (Tuesday), at 4 pm, I’ll be talking about recent developments in PulseAudio
• Tomorrow (Wednesday), at 11am, we’re continuing the Audio BoF that I had mentioned earlier (since we ran out of time on Sunday)

If you’re around and interested, do drop in!

If you make it to Prague the coming week for the LinuxCon/ELCE/GStreamer/Kernel Summit/... superconference, make sure not to miss:

• The Linux Audio BoF with numerous Linux audio hackers, 5pm, on Sunday (23rd, i.e. today).
• Latest developments in PulseAudio by Arun Raghavan. 4pm, on Tuesday, GStreamer Summit
• Linux Kernel Developer Panel, a shared session of LinuxCon and ELCE. Panelists are Linus Torvalds, Alan Cox, Thomas Gleixner and Paul McKenney. Moderated by yours truly. 9:30am, on Wednesday
• systemd Administration in the Enterprise by Kay Sievers and yours truly. 4:15pm, on Wednesday, LinuxCon
• Integrating systemd: Booting Userspace in Less Than 1 Second by Koen Kooi. 11:15am, on Friday, ELCE

All of that at the Clarion Hotel. See you in Prague!

What is this? Well, I'm off to Prague tomorrow morning. I'm very much looking forward to this trip as there are a whole bunch of interesting talks going on over the three conferences I'll be visiting, plus I get to go to Prague, which has been on my "cities to visit" list for quite some time. Tick and tick.

Arun will be giving a PulseAudio talk and Lennart will be rambling on about init systems as is customary these days. Very much looking forward to both.

We've also had an IRC meeting about bluetooth support and policy stuff for in-car usage with some big car manufacturers which we'll follow up next week in person and there are also a lot of other audio folk in town so we'll hopefully kickstart the UCM discussions again with a view to merging into PA 2.0. Looking forward to catch up with Mark and Liam again on that front.

So with pretty much all the people invloved in the Linux audio field, this is a really good opportunity to make some good progress!

Here's to a successful trip!

Many thanks to Collabora who have helped me organise funding and also to Yocto Project (via Texas Instruments) who have very kindly sponsored my attendance of the LinuxCon/ELC-E part of the event. I look forward to finding out more about their project when I help out at their booth!

Yep, if we keep this up, it could even become a habit!

PulseAudio 1.1 is out. It’s mostly a bunch of bug fixes on top of 1.0. Most important of these are fixes for: a libpulse dependency on libsamplerate (if enabled) which would make our LGPL license invalid, broken Skype audio capture (because we changed from a 3 number version to 2 numbers), broken startup without a DBus session bus running, and not going crazy on USB disconnects.

This should be a very safe upgrade, so grab it while it’s hot!

Just a quick note to say that I've just pushed PulseAudio 1.1 out the door. Get it while it's hot!

This release fixes a couple issues people had with our two-point version number change and several other bits and bobs.

On it's way to Mageia Cauldron now and I should get around to backporting this sometime very soon for mga1 now that backports are open

pulseaudio: PulseAudio 1.1 just released as a stable update to 1.0. Get it while it's hot! (by Colin)

I’ve just pushed a bunch of patches by Pierre-Louis Bossart that can have a pretty decent CPU/power impact. These introduce the concept of an “alternate sample rate”.

Currently, PulseAudio runs all your devices at a default sample rate, which is set to 44.1 kHz on most systems (this can be configured). All streams running at different sample rates are resampled to this sample rate. Pierre’s patches add an alternate sample rate that we try to switch to under certain circumstances if it means that we can save on resampling cost. This would happen if the stream uses exactly the alternate sample rate, or some integral-or-so multiple of it.

The default value for the alternate sample rate is 48 kHz. So if you’re playing a movie off a DVD where the audio track is typically a 48 kHz stream, and your card supports it, we switch to 48 kHz and avoid resampling altogether. Similarly, while making voice calls, common sample rates are 8, 16, and 32 kHz. These can be resampled to 48 kHz much faster than to 44.1 kHz.

Now for the big caveat — this won’t work if there’s any other stream connected to your sink/source. So if your music player is playing (or even paused) when you get that voip call, we can’t update the rate. This situation can probably be improved by at least allowing corked streams have their sample rate change (so having some random stream connected but not playing — I’m looking at you, Flash! — won’t block rate updates altogether). Hopefully we’ll get this fixed before this feature is released in PulseAudio 2.0.

Thanks to Pierre for all his work on this, and to my company, Collabora, for giving me some time for upstream work!

Prague is an interesting place to be at this time of the year — next week it’s playing host to the Real Time Linux Workshop. The week after that, we have the Kernel Summit, GStreamer Conference, Embedded Linux Conference Europe and LinuxCon Europe. I’m going to be at the last 3, and there’s some great audio stuff happening!

On the evening of Oct 23rd, we’re having an Audio BoF to discuss pending issues that cut across the stack — ALSA, PulseAudio, GStreamer and any other similar bits that people have complaints about.

Then there’s GstConf, where there are going to be a bunch of awesome talks. Also included is a talk by yours truly about recent developments in the PulseAudio world.

At some point during that week, possibly Oct 26th morning, plans are afoot to have an ALSA BoF to discuss the state and future of the HDA driver.

There are also rumours of excellent beer that will need to be scrupulously verified. ;)

It’s going to be a pretty exciting week!

If you take Ubuntu Brainstorm’s word for it, one of the more popular wishes for Ubuntu, is to avoid having to adjust the volume slider up and down as you plug and unplug your headphones, but instead keep separate volumes stored for both.

Long story short, it’s a desirable feature, and we’re moving in that direction, but slowly, as the feature is more complex than it seems like at first glance.

The good news: in the upcoming Ubuntu Oneiric (11.10), this is actually working. The bad news: it isn’t working for everyone.

For external stuff, mainly USB and Bluetooth devices, this has been working for a quite a few releases now (although you might have to manually switch to your new card when you plug it in). So let’s restrict the discussion to internal sound cards, that on a typical laptop would control your internal speaker and your 3.5mm headphone jack. Here’s where Oneiric will make a positive difference for many of you (although, still far from all of you).

PulseAudio has the concept of “ports” (in your Gnome “Sound settings”, this is what’s labeled a “Connector”), and headphones and speakers would be different ports of the same card. As of Oneiric, every port has its volume stored independently, so when you switch ports, the volume will automatically change.
Now, this does not become really useful until this port can automatically switch back and forth when you plug and unplug your headphones. This feature is also now implemented in Oneiric, as you can read about in my previous blog post, PulseAudio with jack detection.

Things are not always that easy. Not everyone has just internal speakers and headphones, some have line outs instead, or all three. On the input side, some have internal mics, microphone jacks (often more than one), line ins, or any combination of those. In addition, people are different: some want headphones to automatically mute line outs, others don’t. That’s a typical case where different drivers expose very different behaviour: some do, some don’t, some have a setting you can control in alsamixer. Some drivers enable the user to have different volumes for different outputs, others don’t. Drivers label volume controls and jacks differently. Not every driver actually exposes the current jack sense state to userspace, either.

The bottom line: Is this working for you? Great! Is it not? You’re not alone. I’ll try to fix some of that up for Ubuntu 12.04, but there will – no doubt – be users who won’t have this functionality for a long time. At this point, the best you can do is to file a bug using the “ubuntu-bug audio” command, and hope for the best. Even if it might be too late for your hardware to be supported in 11.10, filing the bug sooner rather than later might help to get it into 12.04. However, manpower is always an issue, so even better would be if you could write a kernel patch yourself to fix it.

As Colin Guthrie reports, PulseAudio 1.0 is now out the door! There’s a lot of new things in the release, and we should be getting a much more regular release schedule going. Head over to the full release notes for more details.

A lot of people have contributed to this release and thanks to them all. Special props to Colin all the patch-herding, tireless help, and code ninjutsu!

p.s.: Gentoo packages are already available, of course. :)

PulseAudio 1.0 is out now. It's awesome. Get it while it is hot!

I'd like to thank Colin Guthrie and Arun Raghavan (and all the others involved) for getting this release out of the door!

pulseaudio: Want Twitter and more? Check out Planet PulseAudio: http://t.co/dJVa8RHp (by Colin)

It is with great pride that I announce PulseAudio 1.0!

It's been a long time coming and I'm very glad this is finally out of the door and I look forward to a much more streamlined release process in the future.

There are too many people to thank but in particular I'd like to thank Arun Raghavan, Tanu Kaskinen, David Henningsson, Maarten Bosmans, Daniel Mack, Jason Newton, Jyri Sarha, Lu Guanqun, Luiz Augusto von Dentz, Marc-André Lureau, Pierre-Louis Bossart, Siarhei Siamashka and of course Lennart Poettering.

There is more info over on the announce mail, so give it a read and also see our release notes.

Obviously there is still a huge amount to be done, both in the daemon itself, improving documentation and improving integration into the desktop environment itself. Any help is gratefully received!

So stay tuned for future improvements! And speaking of staying tuned, I'd also like to announce Planet PulseAudio. This is an aggregated feed of posts about PulseAudio. If you have a blog and write about PA, please get in touch and we can add your feed. The design is heavily borrowed from Planet GNOME so it should be familiar for some readers.

Packages are already available for Mageia Cauldron and backports for Mageia 1 will be available sometime soon. Hopefully someone will update the packages in Mandriva as I'm not actively doing stuff over there these days.

Happy listening!

PS I'm sure there will be a brown bag moment to come with a 1.0 release, but fingers crossed.... :p

pulseaudio: PulseAudio v1.0 is out! Go get it now: http://t.co/5f7YTdDm (by Colin)

In an unsurprising turn of events, Adobe completely fails to play well with modern Linux systems. Well done, guys. Well done, indeed.

p.s.: I was quite happy to see that the Google Talk plugin has proper PulseAudio support (thanks to the WebRTC née GIPS code, it looks like).

pulseaudio: Anyone who has an Adobe ID please vote for this bug: http://t.co/U1yQjgQ (by Colin)

This article will review the Creative X-fi surround usb first quickly in Windows XP and then in Linux. It will give you some basic performance ideas and hints for fixing problems and having more fun with it.

After my survey of the possibilities for USB surround sound in Linux, I ended up ordering a Creative X-fi Surround USB. As you could read there, there is a helpful webpage that includes e.g. ready made .asoundrc files.

Windows XP
On my netbook, without installing any special drivers, playing a song on an otherwise unoccupied Windows XP utilizes about 20% CPU, minus the almost 5 % that somehow seem to be there almost always...

I've read reports that there are clicking sounds, which I can imagine well with a heavy system load. I've heard them at first after plugging it in, but by now they disappeared. It sounds crisp on my Sennheiser headphones and the base is noticeably better than my onboard sound, obviously, but that doesn't say much.

Now let's see how it fares in Linux...

Reboot. Plug. Play. It could hardly be easier. All I had to do was to tell mplayer which device to use (mplayer -ao alsa:device=hw=S51). Then it happily started playing over my new external sound card. The sound is as excellent as under Windows. But better than Windows: Surround sound works out of the box in my Kubuntu 9.10 system.

The combination of AC-3 surround with VAAPI accelerated 720p movie on my netbook utilizes around 10-12 % CPU. There is no stuttering in Linux, even with heavier load. Just playing music hardly seems to have any impact -- maybe around 5 % CPU. I'll have to let the Phoronix suite benchmark all this sometime. In Windows even just playing music in VLC and increasing the volume setting with the knob on the device can create stutters.

Right now for some reason the slightly dusty Chromium version on this netbook keeps bringing the system into a hard stutter making characters appear slowly on the screen up to a second after I type and the mouse does not move smoothly. But the sound plays smoothly in the background. Just plain wonderful.

Resume

I can definitely recommend the sound card for USB (surround or stereo) sound in Linux. I'm not sure if I would use it for Windows. Let's see after I installed the drivers, which will probably drive me mad.

But be aware that there are clicking sounds when the card is first intialized. If you want to use it for e.g. system sounds, you may not be very happy with that. They also seem to appear during heavy disk access (apt-get dist-upgrade), even if I'm not playing any music. That's definitely weird...

The microphone input works flawlessly. I didn't have problems with over-amplification, though I can imagine that being an issue with the lack of hardware mixing. And I definitely have a sound card for surround sound HD movie nights now. And the crazy thing is I can do it all perfectly with my little netbook. Thank you, Linux. Thanks also to you, Mandar.

Post Scriptum
Some additional information for nerds: When the card is not in use, it produces not wakes in powertop, when in use, it produces about 200-300 wakes/s (no matter if surround or stereo sound).

Update
The clicking sound seems to be related to the power supply. The problem disappears in the same work load (e.g. heavy disk updatedb) if I plug the power adapter into my netbook.

Update2
As I almost expected the device is not anywhere as good in Windows as in Linux. The crystalizer does sound nice, but there are many glitches in the Windows driver. E.g.

• sometimes when you pause they playback, the result is a constant beep until you press play again, 
• during forwarding in movies, there are short beeps. 
• Once after removing the plug and putting it back inside, the entire system hung, only working again after removing it again and then showing and error message. 
• the driver uses much more CPU than in Linux and it's a 50 MB package
• there are no volume controls for rear, center, lfe and front channels, only 1 master control (in Linux you can fix this -- maybe MatrixMixer works for Windows?)
• the microphone input sounds slightly noise shaped, esp. in combination with the "crystalizer"

Hence I would make my recommendation more clear: I wouldn't recommend it for surround sound in Windows I think. It might still be your best option due to the lack of alternatives, though.

Update3 (9/2011)
Unfortunately I only just tried recording in Linux for the first time. And it seems it doesn't work at all. I got only noise. I've tried Line in and Mic in. There's no real setting for the record options anyway. (Which channel, etc.) So that's definitely a downside. I had never tried that in Linux before. btw. Check here to get the remote control and volume know working. It should come out of the box with kernel 2.6.37+.

And here's my .asoundrc so far:

pcm.!default {
type plug
slave.pcm "softvol" #make use of softvol
}

# create softvol master channel
# see http://alsa.opensrc.org/index.php/How_to_use_softvol_to_control_the_master_volume
pcm.softvol {
type softvol
slave {
pcm "dmixer2" #redirect the output to dmix (instead of "hw:0,0")
}
control {
name "Master" #override the PCM slider to set the softvol volume level globally
card S51
}
}

# create stereo dmixer, because using the 6 channel one causes stutter if the channels are empty
pcm.dmixer2 {
type dmix
ipc_key 2343
slave {
pcm "hw:S51"
channels 2
}
}

pcm.dmixer6 {
type dmix
ipc_key 2342
slave {
pcm "hw:S51"
rate 48000
channels 6
period_time 0
period_size 1024
buffer_time 0
buffer_size 4096
}
}

# reroute the channels because rear and sub/lfe are exchanged
pcm.mysurround {
type route
slave.pcm "dmixer6"
slave.channels 6

ttable.0.0 1
ttable.1.1 1
ttable.2.4 1
ttable.3.5 1
ttable.4.2 1
ttable.5.3 1
}

Jack detection in PulseAudio is now in Ubuntu 11.10. This means that PulseAudio will know whether you have plugged in your headphones, mic or HDMI cable, and be able to use that information. Most computers have automute already (i e, speakers mute when you plug in headphones), but this functionality is done entirely in the kernel. With PulseAudio now knowing about this, it can decide that your main volume control will adjust the headphones volume when you have headphones plugged in, and the speaker volume when you don’t.

HDMI adds one more twist to it. Due to hardware design, there are often several “false” ports accompanying the real one(s). And there is no way of knowing which one is right, except through jack detection. So you might see four different HDMI’s in the user interface, but only one is right, and with the jack detection, the right HDMI output will automatically be selected when you activate the HDMI device.
Speaking of the “Sound Preferences” user interface, I hope that we will have an improved user interface for Ubuntu 12.04, that can hide the HDMI devices that are unavailable.
(Also note that for NVidia and ATI cards, binary drivers are often needed to enable HDMI audio, as well as activating the display, through nvidia-settings or the “Displays” settings dialog.)

All of this won’t work for everyone from the start, as it will need support from the ALSA driver. However, for those who don’t have that support, things will not regress compared to the current handling. Hopefully I will be able to improve that situation for some of you in the 12.04 cycle.

Finally, a note on the upstream status of the patches needed for this functionality:

• The PulseAudio patches will hopefully be merged into PulseAudio, once PulseAudio 1.0 is out. Until then, you can grab the latest patch set here.
• A udev patch required to enable PulseAudio to read the input devices was rejected upstream.
• A kernel patch used to identify HDMI input devices is pending upstream review/approval.

I’m going to be at the Linux Plumbers’ Conference next week, speaking about the things we’ve been doing to make passthrough audio on Linux kick ass.

If you’re around and interested, do drop by!

I have a secret to confess. I’ve spent a great deal of time over the last few months talking to myself. I can’t say I haven’t enjoyed it — it turns out my capacity to entertain myself is far greater than initially suspected. But I hear you ask … why?

Here at Collabora, I’ve been building on Wim’s previous work on adding echo cancellation to PulseAudio. Thanks go to Intel for supporting us in continuing this work. Before too long, all this work will be trickling down to your favourite Linux distribution and all your friends will stop hating you.

First, a quick recap on what acoustic echo cancellation (AEC) is. If you already know this, you might want to skip this paragraph and the next. Say you’re on your laptop, and you receive a voice call from your friend. You don’t have a pair of headphones lying around, so you’re just going to use your laptop’s built-in speakers and mic. When your friend speaks, what she says is played out the speakers, but is also captured by the microphone and she gets to hear herself speak, albeit a short while (a few hundred milliseconds or more) later. This is called acoustic echo, and can be frustrating enough to make conversation nigh impossible. There are other types of echo for phone systems, but that’s not interesting to us at the moment.

This problem is common on pretty much all devices that you use to make phone calls. Astute readers will ask why they don’t actually face this problem on their phone. That’s because your phone (or, if you have a cheap phone, your phone company) has special software hidden away that removes the echo before sending your signal along to the other end. On laptops, which are general-purpose hardware, the job of echo cancellation is left to either your operating system (Windows XP onwards, for example) or your chat client (Skype, for example) to provide.

On Linux, we implement echo cancellation as a PulseAudio module (code-ninja Wim Taymans wrote this last year). We use the Speex DSP library to perform the actual echo cancellation. The code’s quite modular, so it’s not very hard to plug in alternate echo cancellers (we even include an alternate implementation, which isn’t quite as effective as Speex).

Recently, we plugged in some more bits from the Speex library to do noise suppression and digital gain control (so you can quit twiddling with your mic volume for the other end to be able to hear you). We also added a bunch of fixes to reduce CPU consumption significantly — this should be good enough to run on a netbook and reasonably recent ARM platforms.

While all this sounds nice, I think a demo would sound (haha!) nicer …

Without AEC:

(or download ogg, aac)

With AEC:

(or download ogg, aac)

This is a recording of a call between my laptop and N900. The laptop is playing audio out the speakers and recording with the built-in mic. What you hear is the conversation as heard on the N900.

All this echo cancelling goodness will come to a Linux distribution near you in the upcoming 1.0 release of PulseAudio. The next version of the GNOME IM client, Empathy (3.2), will actually make use of this functionality. In due time, we intend to make it so that all voice applications will end up using this functionality (so if you’re writing a VoIP application and don’t want to use this functionality, you need to set a special stream property to disable this — filter.suppress="echo-cancel").

For the impatient among you, you can try all this out by getting recent testing versions of PulseAudio (I know packages are available for Ubuntu, Debian, Gentoo and Mageia at least). To force your phone streams to use echo cancellation, just run pactl load-module module-echo-cancel, and you’re done.

There’s still some work to be done, refining quality and using other AEC implementations (in the short-term, the WebRTC one looks promising). Things don’t work at all if you’re using different devices for playback and capture (e.g. laptop speakers and webcam mic). These are things that will be addressed in coming weeks and months.

I’m in Berlin at the Desktop Summit, so you can drop me a note and we can meet if you want to yell about PulseAudio things that annoy you (or even, y’know, things you like).

pulseaudio: Make sure you come to the PA talk at the Desktop Summit 16:50 rm2002 #ds2011 (by Colin)

pulseaudio: PulseAudio BoF session planned for the #DS2011 http://wiki.desktopsummit.org/Workshops_%26_BoFs/2011/The_Audio_BoF (by Colin)

pulseaudio: Exited to see Windows supported in the upcoming v1.0 pre-release snapshot (by Maarten)

pulseaudio: PulseAudio Developers (and those users I like!) can tweet again from our IRC channel \o/ (by Colin)

pulseaudio: PulseAudio 0.9.23 Released! Grab it while it's hot: http://t.co/TWminho (Docs: http://t.co/Qx5ZBCG Changes: http://t.co/KktbFeC)

OK, so it's been about a year since I was last in this sleeply little town on the path to Zermatt and a lot has changed.

While last year it felt like I was the lone voice singing the praises of PulseAudio (although there were a few supporters!), but this year it feels like everything has gone 180° with pretty much everyone on board! This is a great result for me personally as I've been pretty much the only person working on KDE+PulseAudio integration, so I was very pleased to get this feedback. It's good to know that the hard work and effort you put in is appreciated. It's all too often that the people who appreciate your work are the silent majority (if you do a really good job, they don't know you've done anything as things Just Work™), while the vocal minority are quick to shout and judge and generally flame.

So I was off to an lovely start and I got down to hacking. What did I do this year? Well I continued some work on the interface I made last year called "Speaker Setup". I realised just a short while ago that there was no interface in KDE to be able to change the Source Ports (i.e. pick Mic vs. Line In on your laptop) so I set about extending speaker setup to cope with this. I added a Mic VU meter for good measure (mainly to use up the space with something vaguely useful!). I would ultimately like to do more with this UI but this would need more changes in PulseAudio itself (come listen to my talk in Berlin at the Desktop Summit if you want to know more about this!).

As well as this, I did some tweaks in Phonon to tidy some things up. Various bits and bobs within Phonon and the KCM had bit rotted a little, so minor tweaking saw that all brought up to speed.

I also spent some time hacking on PulseAudio itself, improving some earlier work related to adding Source Output volume controls to PA to take on peer review comments (for those of you unaware, this is capture stream volume control - PA has long supported "per-application" volume control but this only actually applied to outputs. It's not really very common for users to record multiple streams at the same time so support for per-capture stream volumes was never introduced. Now that PA supports Flat Volumes (a feature that always tries to use the hardware volume whenever possible to get the most efficient volume adjustment path), it makes sense to use this for inputs too. It also establishes a degree of symmetry to the API which has always felt a little weird in the past - especially if you are developing a VoIP app (the guys from Skype were a little confused about this disparity for example)). I also spent some time making some minor improvements to pavucontol (shh, don't tell the KDE guys but this is a GTK app!) as this is still my main debug tool when hacking on PA (I mainly improved it to deal more gracefully with errors - like when PA itself crashes and leaves behind the X11 root window's PULSE_SERVER property which results in an invalid argument error from the context with the result that the automatic reconnect mode doesn't work! - but also added some simple keyboard shortcuts that I generally miss when switching windows quickly). I also added support for Source Output volumes to KMix, but this will stay in my private branch until I've committed the PA code as the version check will currently match git master code even if it doesn't yet have the support needed!

I also started to look at Arun and Pierre's awesome work to support passthrough. As there is no reliable way to query receivers for the encodings they support (AC3, DTS etc.) we have to provide a way for users to specify this manually. I worked to rejig how PA stores various bits of information in internal databases to allow for arbitrary lengths of data to be stored rather than the fixed size blobs supported currently. This will pave the way to adding a protocol extension to set the formats for which support will have to be added to the Speaker Setup GUI somehow...

In addition, I also looked at VLC's PulseAudio output layer. I've known for a while that it's kind of lacking and Rémi from upstream VLC has become rather exasperated about the lack of good documentation we provide. I fully appreciate our docs are lacking (some mails on our mailing list today highlight that internal docs for module development are also severely lacking), but I was able to use what was out there to add what I think is quite robust support to VLC. As VLC is used as a Phonon backend by some distros, I felt this was an important task to work on during this KDE sprint.

All in all it was a pleasure to stay here again and meet some now familiar as well as some new people (especially Bart and Trever who are big PA fans!) I look forward to seeing several of them again in Berlin and hopefully next year here in Randa too!

Good news everyone! Mageia 1 is out!!!! Just as I travel to Randa for the KDE Multimedia Development Sprint, I hear that all the hard work put in by the various contributors (in all their forms: packagers, admins, translators, testers and artists) has come to fruition! Go read the official announcement and release notes and then download it!

I've not had nearly as much time to contribute as much as I would have liked to this release, due to various personal, work and upstream project commitments, but I know my good friends and colleagues have done a stellar job (and I've helped out when I can).

I should say that this shouldn't be expected as a ground breaking release. We're not using Gnome 3 or Systemd yet (both will most likely come in Mageia 2) as this release more signifies the establishing of all the various infrastructure needed to create a distro (build cluster, community management, mirror management etc.) especially the proper cleaning and rebuilding of all of the Mandriva packages thought to be essential or vaguely useful. This was a momentous task and one that I think has been achieved in good time.

Onwards and upwards! (to 2!)

 

The Linux Plumbers Conference 2011 in Santa Rosa, CA, USA is coming nearer (Sep. 7-9). Together with Kay Sievers I am running the Boot&Init track, and together with Mark Brown the Audio track.

For both tracks we still need proposals. So if you haven't submitted anything yet, please consider doing so. And that quickly. i.e. if you can arrange for it, last sunday would be best, since that was actually the final deadline. However, the submission form is still open, so if you submit something really, really quickly we'll ignore the absence of time travel and the calendar for a bit. So, go, submit something. Now.

What are we looking for? Well, here's what I just posted on the audio related mailing lists:

So, please consider submitting something if you haven't done so yet. We
are looking for all kinds of technical talks covering everything audio
plumbing related: audio drivers, audio APIs, sound servers, pro audio,
consumer audio. If you can propose something audio related -- like talks
on media controller routing, on audio for ASOC/Embedded, submit
something! If you care for low-latency audio, submit something. If you
care about the Linux audio stack in general, submit something.

LPC is probably the most relevant technical conference on the general
Linux platform, so be sure that if you want your project, your work,
your ideas to be heard then this is the right forum for everything
related to the Linux stack. And the Audio track covers everything in our
Audio Stack, regardless whether it is pro or consumer audio.

And here's what I posted to the init related lists:

So, please consider submitting something if you haven't done so yet. We
are looking for all kinds of technical talks covering everything from
the BIOS (i.e. CoreBoot and friends), over boot loaders (i.e. GRUB and
friends), to initramfs (i.e. Dracut and friends) and init systems
(i.e. systemd and friends). If you have something smart to say about any
of these areas or maybe about related tools (i.e. you wrote a fancy new
tool to measure boot performance) or fancy boot schemes in your
favourite Linux based OS (i.e. the new Meego zero second boot ;-)) then
don't hesitate to submit something on the LPC web site, in the Boot&Init
track!

And now, quickly, go to the LPC website and post your session proposal in the Audio resp. Boot&Init track! Thank you!

[tl;dr — if you're using GNOME or a GStreamer-based player, not using the Rhythmbox crossfading backend, and want to try to save ~0.5 W of power, jump to end of the post]

Lennart pointed to another blog post about actually putting PulseAudio’s power-saving capabilities to use on your system. The latter provides a hack-ish way to increase buffering in PulseAudio to the maximum possible, reducing the number of wakeups. I’m going to talk about that a bit.

Summarising the basic idea, we want music players to decode a large chunk of data and give it to PA so that we can then fill up ALSA’s hardware buffer, sleep till it’s almost completely consumed, fill it again, sleep, repeat. More details in this post from Lennart.

The native GNOME audio/video players don’t talk to PulseAudio directly — they use GStreamer, which has a pulsesink element that actually talks to PulseAudio. We could configure things so that we send a large amount (say 2 seconds’ worth) to PulseAudio, sleep, and then wake up periodically to push out more. Now in the audio player (say Rhythmbox), the user hits next, prev, or pause. We need to effect this change immediately, even though we’ve already sent out 2 seconds of data (it would suck if you hit pause and the actual pause happened 2 seconds later, wouldn’t it?). PulseAudio already solves because it can internally “rewind” the buffer and overwrite it if required. GStreamer can and does take advantage of this by sending pause and other control messages out of band from the data.

This all works well for relatively simple GStreamer pipelines. However, if you want to do something more complicated, like Rhythmbox’ crossfading backend, things start to break. PulseAudio doesn’t offer an API to do fades, and since we don’t do rewinds in GStreamer, we need to apply effects such as fades with a latency equal to the amount of buffering we’re asking PulseAudio to do. This makes for unhappy users.

Well, all is not as bleak as it seems. There was some discussion on the PA mailing list, and the need for a proper fade API (really, a generic effects API) is clear. There have even been attempts to solve this in GStreamer.

But you want to save 0.5 W of power now! Okay, if you’re not using the Rhythmbox crossfading backend (or are okay with disabling it), this will make Rhythmbox, Banshee, pre-3.0 Totem (and really any GNOMEy player that uses gconfaudiosink, which will soon be replaced by gsettingsaudiosink, I guess), you can run this on the command line:

gconftool-2 --type string \
    --set /system/gstreamer/0.10/default/musicaudiosink \
    "pulsesink latency-time=100000 buffer-time=2000000"

On my machine, this brings down the number of wakeups per second because of alsa-sink to ~2.7 (corresponding nicely to the ~350ms of hardware buffer that I have). With Totem 3.0, this may or may not work, depending on whether your distribution gives gconfaudiosink a higher rank than pulseaudiosink.

This is clearly just a stop-gap till we can get things done the Right Way™ at the system level, so really, if things break, you get to keep the pieces. If you need to, you can undo this change by running the same command without the latency-time=… and buffer-time=… bits. That said, if something does break, do leave a comment below so I can add it to the list of things that we need to test the final solution with.

D. Jansen has put up a blog story including some power saving results when running PulseAudio on modern HDA drivers. This shows off some work Pierre-Louis Bossart from Intel did on the HDA drivers which now enables the timer-based scheduling code in PulseAudio I added quite some time ago to come to its full potential. You can save half a Watt and reduce wakeups while playing audio to 1 wakeup/s.

Previously there was little public profiling data available about the benefits PA brings you for low-power devices. Thanks to Dennis' data there's now public data available that hopefully explains why PA is the best choice for low-power devices as well as desktops. Hopefully this cleans up some misconceptions.

Pierre-Louis, thanks for your work!

Update: Arun Raghavan has posted a follow-up to this.

Here is a group photo of the attendees at the ASoC and Embedded Linux Audio Conference in Edinburgh :

(Skip to the update)
Ok, so with some help from Pierre-Louis from Intel I've managed to get it working and do some performance/power tests. But let me start at the beginning: Recently, pulseaudio not only switched to a more power efficient (and otherwise) timing system, as far as I understand a callback API. It also provided the infrastructure to use ALSA devices without causing any interrupts ("period wakeup disabling"), so you CPU can stay longer in standby mode (e.g. "C6 residency"), saving you power and avoiding playback glitches at the same time. See here and here or more background information. With kernel 2.6.38 the first driver (snd-hda-intel) supports this infrastructure out of the box, the snd-hda-intel driver. This combination is what I tested for power efficiency...
Read more »

I’ve been a little busy lately with new assignments and travel. I just returned from two back-to-back conferences, in two different countries, in less than one week.

The first was the ASoC and Embedded Linux Audio Conference in Edinburgh. This was a gathering of about 30-40 multimedia developers from all over the world representing almost every company involved in this space and I was honored to be invited. After introductions, the 2 days worth of in-depth technical discussions got started with a description of the ASoC DSP support, focused on the description and charts shown here -> http://omapedia.org/wiki/Audio_Drive_Arch   We then shifted to OS integration topics that mainly centered around Android’s need for a different ALSA user space lib, with more business favorable license terms. Several options were discussed. The UCM talk was next and with an initial brief overview and then enumerating the needs moving forward, including PulseAudio work. Next were discussions on QoS, run-time coefficients, firmware and events. The meeting drew to a close with a summary, questions time and assignments review. I drew the assignment for looking into getting an install script and  git set up for the UCM profile config files, something we will need for Linaro.  All in all, a great technical conference that was well prepared and useful. Also, it was a fantastic opportunity to meet everyone I have been interacting with in email and IRC, plus many new contacts. I look forward to attending the next one!

The second was the Linux Audio Conference in Dublin, aka, LAC2011. I worked with the planners of this conference to get the PulseAudio folks, including Lennart Poettering,  a place to work. The space we were allocated was large enough to allow others to come and listen and ask a few questions. I want to apologize to the people that waited so patiently to ask questions.  We had such an active discussion between the developers that they could hardly get a word in edge wise. I learned a lot and hopefully contributed to the overall discussion, direction and schedule. The rest of the conference was more general purpose for all things Linux Audio including: sound generation, sequencing, pro audio, and composition. I have followed this for many years now and it was really interesting to see how it continues to evolve. Even though it was a Linux conference, there were constant references to Apple and iPad integration with the applications used, even a session on running PulseAudio on Mac OS X. I think that mobile Linux has an opportunity to reach these folks.

Even though the trip was jam-packed, it was a perfect technical requirements review for going into the next development cycle for Linaro.  They were both worth the hectic travel schedule.

The next generation desktop has arrived. I am running it as I type this, and so should you. So, go, get it!

If you are in Berlin on Friday you should also attend our GNOME 3.0 Release Party. It's at the world famous c-base, in the remains of an alien spaceship that crashed into Berlin 4.5 billion years ago (no kidding!). We've got Ubuntu's Daniel Holbach as DJ, and a few folks from the GNOME community will do a talk or two (including that annoying dude who created Avahi, PulseAudio and systemd). We even got Mirko Boehm from the KDE side to say a few things. And there are going to be GNOME 3 goodies! How awesome is that? See the wiki page for further details.

And here's your homework until Friday: Try out GNOME 3.0!

The best way to learn code is to work on it. I did just that implementing a feature to add tsched buffer size to the configuration file. Late in the process, I discovered that it was already possible to pass it to the ALSA module, so I abandoned the additional code. Since it stumped me, I figured the configuration of PulseAudio might be somewhat confusing and could use a blog entry, hence this.

First, if you have not yet read PerfectSetup, start with that.  You can find it here: http://www.pulseaudio.org/wiki/PerfectSetup

There are 3 places to change the configuration parameters and behavior of PulseAudio.

system.pa/default.pa – startup script to specify module loading, etc, module parameters are passed here, more on that later

client.conf – does just what is says, config file for clients

daemon.conf – configuration specific to the sound server daemon

On Ubuntu, these files are installed in /etc/pulse. The system.pa startup script is used when PulseAudio is used in system-wide mode. The other, daemon.conf is used when the sound server is started in user mode.

The values listed in these files are default values and are commented out. Obviously, uncommenting the parameter will not change anything unless the assignment is value changed.

The startup script can be used to pass module specific parameters. This is done by uncommenting the line in the file. For example, passing a new tsched buffer size to the ALSA module would look like this:

load-module module-alsa-sink tsched_buffer_size=XYZ

Here are all the module specific parameters for the ALSA module (from module-alsa-card.c) :

“name=<name for the card/sink/source, to be prefixed> “
“card_name=<name for the card> “
“card_properties=<properties for the card> “
“sink_name=<name for the sink> “
“sink_properties=<properties for the sink> “
“source_name=<name for the source> “
“source_properties=<properties for the source> “
“namereg_fail=<pa_namereg_register() fail parameter value> “
“device_id=<ALSA card index> “
“format=<sample format> “
“rate=<sample rate> “
“fragments=<number of fragments> “
“fragment_size=<fragment size> “
“mmap=<enable memory mapping?> “
“tsched=<enable system timer based scheduling mode?> “
“tsched_buffer_size=<buffer size when using timer based scheduling> “
“tsched_buffer_watermark=<lower fill watermark> “
“profile=<profile name> “
“ignore_dB=<ignore dB information from the device?> “
“sync_volume=<syncronize sw and hw voluchanges in IO-thread?> “
“profile_set=<profile set configuration file> “

This is just a quick reminder that the Call for Papers for the Desktop Summit ends on Friday. So get your thinking caps on and write some abstracts! I've submitted my own proposal which would discuss the UI layers which expose PulseAudio in both KDE and GNOME, how they differ and what is missing (hopefully a lot less will be missing by the time the summit comes around in August!!).

I've included my abstract below just in case you are interested.

PulseAudio integration has come a long way. The times when detractors derided it for crashing or simply not working are mostly in the past (there are always exceptions!) and the various distributions now have solid integration and configuration options available. ALSA has also come a long way to support the timer-based scheduling that PulseAudio uses by default.

Today, a major hurdle is UI and Desktop Environment integration. In this talk I intend to look at the current UIs in both GNOME and KDE and the kind of interfaces that are missing and are still needed and what options still need to be exposed from the underlying ALSA level. I'll look at the routing logic chosen under the (more exposed) KDE and the (more minimal) GNOME interfaces and how we support that at the PA level. I'll also look at how we should be configuring some of the more advanced features of PA in a way that can fit in neatly to the DE with their own, native UI.

It has been a few weeks, but in my defense, I have been pretty busy testing and debugging in this new environment. As an assignee to the Linaro project, my first task was to figure out if increasing the buffer size would enable the ARM cpu to drop into a lower power state. Code investigation, community feedback and prototyping said probably not, but I followed through. Below is a chart showing all the testing done, including the recent work to go back through the tests with ALSA 1.0.24 in the stack to enable checking for  period disable. Enjoy!

Here are (now improved) quick instructions to create a working build sandbox for PulseAudio development on Ubuntu 10.10 Maverick on an i386 platform. I highly recommend reading all this post completely before starting this process.

Here are the steps to make PulseAudio that builds and works for Ubuntu:

1. Install or update to Maverick and accept defaults, this process will probably work fine with other Ubuntu versions, so if you are not on Maverick, give it a try.
2. Update all packages with Update Manager
3. Install dpkg-dev -> sudo apt-get install dpkg-dev

If you are using source from git:

1. Install git if you haven’t already -> sudo apt-get install git
2. From your home directory, get the latest PulseAudio source -> git clone git://git.0pointer.de/pulseaudio.git
3. Git will create the PulseAudio repository in a directory called pulseaudio, when the clone has completed, change to that directory -> cd ~/pulseaudio

If you are using Ubuntu maverick source:

1. Install the source -> apt-get source pulseaudio

Configure and build the source:

1. Install the prereq packages -> sudo apt-get build-dep pulseaudio
2. To check the environment and set up the basics, run the bootstrap script -> ./bootstrap.sh
3. Next, run configure -> ./configure
4. Run make to build the executables

Note: the executables will be in the src directory. Change to that directory and run specifying that path so that you will not run the system programs. For example, to run pulseaudio with verbose log messages, cd to src and run -> ./pulseaudio -n -F ./default.pa -p ./.libs -vvvv

There are some very helpful hints at: http://colin.guthr.ie/2010/09/compiling-and-running-pulseaudio-from-git/ The steps on this page are slightly different for Ubuntu, but worth reading to better understand the process.  Also, this page does a really nice job of explaining how to run pulseaudio for test/debug.

Here are my notes from the experimental process of learning how to get sound out of a Linaro image (Headless, ALIP, Netbook) and a BeagleBoard.

• Plug amplified speakers into the 1/8″ jack closest to the s-video port – they must be amplified, signal is line level only

• ALSA – Playback hardware devices:
  • card0: omap3beagle
  • device0: TWL4030
  • No regular simple controls like “Master” defined for BeagleBoard
• Sounds installed in /usr/share/sounds
• Test with aplay
• By default, all sound outputs for ALSA are set to 0 and/or muted

• Configure with alsamixer
  • Unmute HeadsetL and HeadsetR, turn volume up for “DAC2 Analog” ~50 and “DAC2 Digital Fine” ~50-75, and “Headset” to ~ 67 for best results, these are the only controls that change line-level output via the onboard jack
  • All controls listed must be turned up or no level is present at the jack
• Pulseaudio – Successful playback of .wav with paplay, one dropout, quality fairly good

OK, so this question crops up quite often, so I figured it's worth a post. People often ask how to enable 5.1 digital output with PulseAudio. In the past the answer is typically "you can't because S/PDIF only accepts stereo PCM or passthrough data" and while this is true, we can relatively easily hook up an AC3 encoder which is helpfully provided in the ALSA plugins package.

While the "a52" plugin is part of the official ALSA plugins source, some distros do not compile it by default. It's available in Mandriva out of the box, but on Ubuntu for example you'll need to compile it yourself or find someone who provides a custom build for you (feel free to suggest a PPA in the comments below). Someone did tell me about pretty good instructions about building the plugin yourself.

So after building the plugin all you need to do is add a snippet of ALSA configuration to setup the a52 plugin with a named PCM that PulseAudio checks for and then everything should work automatically.

Here is the snippet. I'd recommend adding it to your ~/.asoundrc file (just create it if it doesn't exist; ~/ is just a shortcut to your home directory).

pcm.a52 {
  @args [CARD]
  @args.CARD {
    type string
  }
  type rate
  slave {
    pcm {
      type a52
      bitrate 448
      channels 6
      card $CARD
    }
    rate 48000 #required somehow, otherwise nothing happens in PulseAudio
  }
}

(just as a note, the previously linked article has an error when is suggests using echo to add this to your ~/.asoundrc as it doesn't double escape the $CARD)

Once this is in place, then PulseAudio will automatically probe it and provide you the option of selecting a 5.1 Digitial Output profile. You can use pavucontrol's Configuration tab to pick your profile (or gnome-volume-control or the Speaker Setup tab in KDE).

One other thing you may have to do to get this working properly is change the default sample rate used by PA to be 48kHz. To do this, just edit daemon.conf in either /etc/pulse or ~/.pulse and include the line:

default-sample-rate = 48000

 

What about Passthrough?

At present Passthrough support isn't available in the latest released version of PA (it is supported in git master). Even when this is available it will be exclusive (as will any other codec support such as MP3 for Bluetooth and RAOP etc.) and as such mixing will not be available which leads to a whole set of UI and configuration problems, so in many ways using a software AC3 encoding system like the A52 plugin here has many advantages over using passthrough. Obviously there is processing and mixing overhead but for many people (myself included) this is perfectly acceptable.

One odd thing about the setup is that some applications will consider this an "Analog[ue]" output (e.g. in XBMC I have to select Analog Output and choose 5.1 speakers to get everything working correctly, but at least it works

PS, yes the title is a Wu-Tang reference

So I'm just back from Nürmberg, German where I was fortunate enough to be invited to the speak at the OpenSuse Developer Summit. It has to be said, everything went really well. The first thing that struck me about this event is that it was quite inclusive. Although, obviously, the point was to discuss, promote and develop Suse there were folk there from other distros including Debian, Fedora and, of course Mandriva/Mageia (well, me!). This was very nice to see.

I was fortunate enough to be invited by Scott Reeves to speak about PulseAudio and specifically the KDE Integration work recently undertook. Scott works mostly on the SLED side of things (Enterprise Desktop) and PulseAudio falls within his remit. I'm pleased to say that the next version of OpenSuse's KDE flavour will follow the lead of Mandriva, Fedora and, most recently Kubuntu, and default to use PulseAudio under KDE. This is IMO a great step forward and will help continue the acceptance and appreciation of PulseAudio and I'm very happy to have played a part in making that possible.

Our talk itself went very well and seemed to be well received. It was also a pleasant surprise to finally meet both Takashi Iwai and Lennart Poettering who attended my talk and generally milled around. I've managed to miss meeting Lennart at various events over the years and while we've worked quite closely on PulseAudio in the past (to the extent he trusts me enough with managing the PA project while he works on the systemd project), it was nice to actually "talk" about PulseAudio! The subsequent BoF (Birds of a Feather) session on systemd was also very interesting (even if there was a little bit of a sticking point on defining preload vs. readahead which IMO was really labouring semantics to a degree and prevented getting to the more interesting stuff!). Takashi is another one of the few people who work on Linux Audio and it was nice to chat to him after exchanging several emails over the years on the ALSA mailing list.

The rest of the conference was also interesting. As someone who sits rather on the fence between KDE and Gnome it was interesting to see the developments in Gnome 3 demonstrated. There were various other titbits of interesting things too, but I wont bore the world with the details!

On the Saturday, I took the tram to the south of Nürmberg and visited their New Museum of Art and Design. It was very enjoyable. Almost deserted, which made it a rather nice experience, but also sad to see such a place under appreciated. Still it was quite early when I went, so maybe it hots up in the afternoon! My favourite photographs were by Gabriele Basilico and depicted a series of chairs with interesting seats: lattice, round holes, metal bars etc. and then accompanying photographs of a bottom that had clearly been sitting in said chair for a while. That quite tickled me An installation peace by Jeppe Hein was also very smart. Using light, mirrors and sensors to create interactive "rooms", including and empty one that just set off an alarm when you stepped into it (no doubt just to make people jump!). I then walked up through the centre checking out the churches (which would be a lot more interesting if they didn't have all that religious stuff inside them!) and the castle. A thoroughly pleasant day out!

When it was time to return home, my only regret was that I didn't get a chance to say goodbye to Scott, so as luck would have it I bumped into him at the airport in the queue going through security. It turned out he was on my flight to Amsterdam anyway so we were able to have a little blether on the way there.

What came as even more of a coincidence was the fact that I bumped into my mate Dom who happened to sit in front of me on the connecting flight to Edinburgh! He'd just been off climbing Kilimanjaro, which is arguably slightly more interesting than my week but he got his phone stolen at the "security" check in the airport (and the subsequent delay that it caused actually meant he was on my flight rather than an earlier one) so that evened things up!

But all in all, thanks to Novell, OpenSuse and Scott for inviting me along!

Confession time. I think I cocked up. Someone was asking me about why the HDMI was the default sound device in Phonon when used with PulseAudio. I patiently explained that this was a bug a while back but I fixed it.I diligently went to look up the relevant commit refs and then realised that I hadn't actually committed the fix. Whoops. It seems I fixed that bug during a late part of the Mandriva development cycle and only applied the code changes as patches on top of the source rather than committing them properly. I think I did this because I wanted wider testing before deeming them "ready" for upstream inclusion but then basically forgot. The other reason was perhaps that I was ashamed of myself for using a bubble sort - I really need to make an ordered pa_hashmap for this as I've found myself needing it a couple times now and will definitely need it in the future too. Another item on my todo list!

So apologies for that. I know some distro maintainers look at the Mandriva packages subversion for patches etc. so I'm sure some folks will have these fixes already, but it was a mistake not to push them upstream sooner.

Anyway, fixes are pushed to upstream master and stable-queue and will be part of the (hopefully) upcoming bugfix release.

pulseaudio: Want to know how to build & run PA from git without messing up your system? Here is a handy guide http://is.gd/fuhm8 (by Colin)

A few budding developers have asked me recently about this and to make life easy, I decided to write up this guide! There are some gotchas to look out for so please read carefully!

Are we ready? OK, lets being!

The shell output shown below will include my machine's name, "jimmy". My bash prompt also shows the current git branch thanks to the git-prompt package in Mandriva (you can enable it manually by following this guide). Note that for various reasons I wont go into in this guide, the development version of PA is currently 0.9.19, this is despite the current released version being 0.9.21. Its due to how the git tree is organised, and I'm hoping to fix this soon. Edit: Git master is now tracking PA 1.0 (not for any specific milestone of 1.0, but just because a 3-point version number is kinda annoying. Essential the version policy is now decided and all should be working fine now. There may still be a 0.9.23 based of the current stable-queue branch, but the next release from master will be 1.0.

May the Source Be With You

The first job is to clone our code repository. You first have to pick where you want to keep your development version. In the example below I've decided to use a folder under my home directory called "padev"

[colin@jimmy ~]$ git clone git://git.0pointer.de/pulseaudio.git padev
Cloning into padev...
remote: Counting objects: 39578, done.
remote: Compressing objects: 100% (6760/6760), done.
remote: Total 39578 (delta 32779), reused 39578 (delta 32779)
Receiving objects: 100% (39578/39578), 14.56 MiB | 1.97 MiB/s, done.
Resolving deltas: 100% (32779/32779), done.
[colin@jimmy ~]$

Now you should have a ~/padev/ folder containing the code.

Compile Time

Now it's time to compile the code, but before we do this, we have to prepare the checkout for compilation. Handily there is a script provided to make this easy for us.

[colin@jimmy ~]$ cd padev/
[colin@jimmy padev (master)]$ ./bootstrap.sh -V
+ case $(uname) in
++ uname
...
+ make clean
make: *** No rule to make target `clean'.  Stop.
[colin@jimmy padev (master)]$

There is an error about not being able to "make clean" here but you can safely ignore that.

Next we'll create a build directory. This is not mandatory, but it helps keep temporary build files etc. separate from the code in the checkout (there are special git commands to delete such files but all the same, I feel this is cleaner). After creating the build directory, we change to it and run the configure script.

[colin@jimmy padev (master)]$ mkdir build
[colin@jimmy padev (master)]$ cd build/
[colin@jimmy build (master)]$ ../configure --prefix=$(pwd)
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
...
config.status: executing po/stamp-it commands

---{ pulseaudio 0.9.19-562-g395da }---

prefix:                        /home/colin/padev/build
sysconfdir:                    ${prefix}/etc
localstatedir:                 ${prefix}/var
System Runtime Path:           ${prefix}/var/run/pulse
System State Path:             ${prefix}/var/lib/pulse
System Config Path:            ${prefix}/var/lib/pulse
Compiler:                      gcc -std=gnu99
CFLAGS:                        -g -O2 -Wall -W -Wextra -pipe -Wno-long-long -Winline -Wvla -Wno-overlength-strings -Wunsafe-loop-optimizations -Wundef -Wformat=2 -Wlogical-op -Wsign-compare -Wformat-security -Wmissing-include-dirs -Wformat-nonliteral -Wold-style-definition -Wpointer-arith -Winit-self -Wdeclaration-after-statement -Wfloat-equal -Wmissing-prototypes -Wstrict-prototypes -Wredundant-decls -Wmissing-declarations -Wmissing-noreturn -Wshadow -Wendif-labels -Wcast-align -Wstrict-aliasing=2 -Wwrite-strings -Wno-unused-parameter -ffast-math -Wp,-D_FORTIFY_SOURCE=2 -fno-common -fdiagnostics-show-option -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include

Have X11:                      yes
Enable OSS Output:             yes
Enable OSS Wrapper:            yes
Enable CoreAudio:              no
Enable Alsa:                   yes
Enable Solaris:                no
Enable GLib 2.0:               yes
Enable Gtk+ 2.0:               yes
Enable GConf:                  yes
Enable Avahi:                  yes
Enable Jack:                   yes
Enable Async DNS:              yes
Enable LIRC:                   yes
Enable DBUS:                   yes
Enable HAL:                    yes
Enable udev:                   yes
Enable HAL->udev compat:       no
Enable BlueZ:                  yes
Enable TCP Wrappers:           yes
Enable libsamplerate:          yes
Enable IPv6:                   yes
Enable OpenSSL (for Airtunes): yes
Enable tdb:                    yes
Enable gdbm:                   no
Enable simple database:        no
Enable fftw:                   yes

System User:                   pulse
System Group:                  pulse
Access Group:                  pulse-access
Enable per-user EsounD socket: yes
Force preopen:                 no
Preopened modules:             all
[colin@jimmy build (master)]$

You should pay particular attention to the --prefix argument passed to configure. Rather than "installing" this version of PulseAudio, we'll just run it from the source tree. This is both quicker and saves any potential conflict with your system-installed PulseAudio packages.

You should also pay attention to the table at the end which lists the available support. In order for automatic card detection to work properly with your build, you really should ensure that udev support in particular is available. If it does not print a "yes" line in the output then you probably do not have the "udev-devel" package for your distro installed.

Let's Build

OK, so you're ready to build! But not quite. Due to an upstream bug, the translations for .desktop files are not written if the destination folder does not exist, so let's create it manually

[colin@jimmy build (master)]$ mkdir -p src/daemon
[colin@jimmy build (master)]$

OK, with all the prep work done, we really do build it.

[colin@jimmy build (master)]$ make
echo 0.9.19-562-g395da > ../.version-t && mv ../.version-t ../.version
make  all-recursive
...
file=`echo zh_TW | sed 's,.*/,,'`.gmo \
 && rm -f $file && /usr/bin/msgfmt -o $file ../../po/zh_TW.po
make[2]: Leaving directory `/home/colin/padev/build/po'
make[2]: Entering directory `/home/colin/padev/build'
make[2]: Leaving directory `/home/colin/padev/build'
make[1]: Leaving directory `/home/colin/padev/build'
[colin@jimmy build (master)]$

Now that it's built successfully we can run it, but we'll do a little bit of preparation first. As ALSA currently lacks UCM (Use Case Management) (although this is due to be added soon), PulseAudio supports a fairly robust "probing" system to determine how your sound hardware works. In order to run these probes it has to know where to look for the "mixer profile" definitions. As we are running from the build tree, we'll cheat a little and use a symlink so that our development build can find the files.

[colin@jimmy build (master)]$ mkdir -p share/pulseaudio
[colin@jimmy build (master)]$ ln -s ../../../src/modules/alsa/mixer share/pulseaudio/alsa-mixer
[colin@jimmy build (master)]$

In order to run some test tools, we also need to manually create some symlinks that are normally done as part of the install process.

[colin@jimmy build (master)]$ ln -s pacat src/paplay
[colin@jimmy build (master)]$ ln -s pacat src/parec
[colin@jimmy build (master)]$

Run, Forest, Run!

Now that things are built and some symlinks are in place, we can run our nice shiny development version. You should first ensure that the system-installed PulseAudio daemon is not running. In order to do this, you should disable autospawn by doing:

echo "autospawn=no" >> ~/.pulse/client.conf

Once this is done, you should reboot. PulseAudio will likely still start when you log in to X11 by virtue of the start-pulseaudio-x11 script that is run at login, but some systems that rely on PA autospawn may not initialise correctly (e.g. under KDE knotify and kmix may start in 'ALSA mode'. This is generally not a problem, but you should be aware of the consequences.

So if your system PA has been run, simply execute:

pulseaudio -k

To kill the currently running daemon. You can then start your development version via:

[colin@jimmy build (master)]$ src/pulseaudio -n -F src/default.pa -p $(pwd)/src/.libs/ -vvvv
W: main.c: Couldn't canonicalize binary path, cannot self execute.
I: main.c: setrlimit(RLIMIT_NICE, (31, 31)) failed: Operation not permitted
I: main.c: setrlimit(RLIMIT_RTPRIO, (9, 9)) failed: Operation not permitted
D: core-rtclock.c: Timer slack is set to 50 us.
D: core-util.c: RealtimeKit worked.
I: core-util.c: Successfully gained nice level -11.
I: main.c: This is PulseAudio 0.9.19-562-g395da
...

This will produce a lot of debug output, so you should leave that terminal running. The command line arguments are as follows: "-n" says "do not process the (system) default.pa". This is generally only needed if you have a ~/.pulse/default.pa file, but it does no harm to include it always. "-F src/default.pa" says to "process the script src/default.pa" and "-p $(pwd)/src/.libs" tells PA where to look for it's modules (i.e. from your build tree).

Note that the state files saved by PulseAudio in ~/.pulse/ folder will very likely NOT conflict with your system PA's files. This is because our development PA build does not know the right path to look for /var/lib/dbus/machine-id. Because of this, the prefix used on files will default to the host name of your machine, not the string of apparently random numbers and letters that you may see in there already. If you cross reference, the output from cat /var/lib/dbus/machine-id will show the same number as used here. We do this to ensure we can have separate preferences for different machines when your home directory is shared over e.g. NFS - the machine-id is more stable than the hostname which is why we prefer that as a prefix.

Running a Client App

So now that everything is running, you should be able to run a client application. As the build tree comes with some utilities you can run them directly from there:

[colin@jimmy build (master)]$ src/paplay -vv /usr/share/sounds/ia_ora-startup.wav
Opening a playback stream with sample specification 's16le 2ch 22050Hz' and channel map 'front-left,front-right'.
Connection established.
Stream successfully created.
Buffer metrics: maxlength=4194304, tlength=176400, prebuf=174640, minreq=1764
Using sample spec 's16le 2ch 22050Hz', channel map 'front-left,front-right'.
Connected to device alsa_output.pci-0000_00_1b.0.analog-stereo (0, not suspended).
Stream started.
Stream underrun.
Playback stream drained.: 1007045 usec.
Draining connection to server.

If, however you want to run a more advanced client application, then you need to employ a little hack to tell your system to use your development version of libpulse. This is quite trivial:

[colin@jimmy build (master)]$ export LD_LIBRARY_PATH=$(pwd)/src/.libs
[colin@jimmy build (master)]$ mplayer /path/to/your/fav.mp3
...

Easy eh? That's all you need to do to run PulseAudio from git. You can now easily try out patches, write your own modules and help contribute! Happy hacking.

Some notes on Overlinking

So, just before I sign off, I thought it was best to mention overlinking. PulseAudio itself uses a shared library that is used by both client and server. This library is "libpulsecommon-0.9.x.so". Client applications should NOT link to this file directly - instead libpulse will load it in for you. This can lead to some strange results. e.g. consider the following output:

[colin@jimmy ~]$ ldd `which paplay` | grep pulse
	libpulse.so.0 => /usr/lib64/libpulse.so.0 (0x00007f5bd9fc7000)
	libpulsecommon-0.9.21.so => /usr/lib64/libpulsecommon-0.9.21.so (0x00007f5bd9d76000)
[colin@jimmy ~]$ ldd `which mplayer` | grep pulse
	libpulse.so.0 => /usr/lib64/libpulse.so.0 (0x00007f617c1f4000)
	libpulsecommon-0.9.21.so => /usr/lib64/libpulsecommon-0.9.21.so (0x00007f6175f08000)

This looks more or less the same right? Not quite. Compare this to:

[colin@jimmy ~]$ objdump -p `which paplay` | grep pulse
  NEEDED               libpulse.so.0
  NEEDED               libpulsecommon-0.9.21.so
  required from libpulse.so.0:
[colin@jimmy ~]$ objdump -p `which mplayer` | grep pulse
  NEEDED               libpulse.so.0
  required from libpulse.so.0:

So what you can see here is that my paplay really does need both, but mplayer actually only does not directly need libpulsecommon-0.9.21.so. But what does this mean to you when running things? Well, due to the fact that PulseAudio has this kind of circular dependancy internally, we cannot use the --no-undefined or --as-needed build options. This means that the PulseAudio package is Over linked. This is why the paplay utility needs libpulsecommond-0.9.21.so directly, unlike mplayer, which does not.

So if you try and use the above guide and ultimately run the system provided paplay utility, you'll find you run into problems. This is because the system libpulsecommon-0.9.21.so will be used, not your freshly compiled version (which could have a completely different version number - e.g. libpulsecommon-0.9.19.so!).

While we don't need to run the system paplay (as we have built our own version), it's easy to forget this quirk and break things. If you want to be sure, you can place a symlink in your build folder to fool the system into loading your libpulsecommon, even when the versions don't "match". As this is an overlinking problem, there is little danger in doing this hack:

[colin@jimmy build (master)]$ ln -s libpulsecommon-0.9.19.so src/.libs/libpulsecommon-0.9.21.so
[colin@jimmy build (master)]$

This puts the necessary symlink in place to make my dev build (0.9.19) replace the system build (0.9.21). This only has effect with the LD_LIBRARY_PATH variable set, so it wont interfere with anything on your system.

Depending on your your distro packages things, the problems of overlinking may be present in more than just the paplay utility. So check this out and use objdump -p to confirm the client application you want to run is linked correctly and use the symlink hack if needed.

Well it's taken me a little time to commit this work, but here it is. This is the fruits of my labour from the KDE Multimedia Sprint earlier this year.

As well as taking part in various discussions, I was able to spend some time cooking up a UI to control the configuration of PulseAudio and the various cards attached.When adding PulseAudio support to the various parts of KDE that need it (Phonon, KMix), there was an important part of the puzzle missing: a card profile selector, and a sink/source port selector. I had always intended to include this functionality somewhere, but the KMix framework didn't really allow for it neatly (I could have created a separate dialog of course but it didn't quite feel right).

The eagle eyed readers may have seen a sneak preview of this feature when looking at the KDE-specific help page I wrote up on the PulseAudio website. So I give you this: the Speaker Setup tab in System Settings.

The little icon in the middle is your user icon, so you'll see this differently unless your a weird stalker type and have set your icon to mine :s

The various drop downs allow fully control over all cards that are attached to the system. The buttons on the main pane allow you to test each speaker separately. In order to test the speakers, libcanberra is used. libcanberra is an implementation of the Free Desktop Sound Theme specification. It allows this test to be implemented with minimum hassle and I'm not personally interested in reinventing the wheel, hence the use of this library and the additional dependency.  Some people dislike using libcanberra in KDE (as was apparent from some discussions at the Multimedia Sprint), but I believe the reasons were often personal ones and not related to the usefulness of the code. If someone really wants to factor this out, they can but I have no interest in doing so and will prefer to use the existing implementation whenever possible (and this would include any future implementation I may or may not do with regards to support the Sound Theme Specification in KDE (or maybe Qt) directly)

When a 5.1 surround system is presented, the GUI is obviously a bit more advanced:

For those naysayers, I've tried my best to ensure that the compile will work fine without PulseAudio installed. It should also degrade gracefully when it is compiled with PA support, but PA is not configured to be used at runtime. The whole tab will simply not be available.

There is still one small problem left in that I don't handle disconnection/reconnection yet. This causes the GUI to crash if the PA server is stopped and restarted. This is not typically something that happens, but it's still something I will fix shortly all the same.

This code is now in trunk (r1154776) so feel free to try it out and report other bugs etc. This GUI is also included in Mandriva Cooker (I did want to include it prior to 2010.1 release, but the timing didn't work out - tho' it probably would have been OK considering the delays that cropped up in the release process). I expect this functionality to be included in any updated/backported versions of KDE for 2010.1.

For reference, in case you didn't spot it yourself, this GUI was heavily inspired by the gnome-speaker-setup utility.

Well it's not really like that, but I guess those involved can think about it a bit like that at times!. For some background, Phonon is a Multimedia framework that was included in Qt 4 as far as I understand it, it was developed outside Qt, but was adopted (please correct me if my history is incorrect here). It was designed to give application developers easy access to media playback systems, be it MP3 music or new fangled WebM video! Rather than implement any of the complex stuff itself, Phonon hands off the actual decoding and playback parts to existing media frameworks. Originally Qt wrote a GStreamer "backend" for Phonon and this was the only available backend on Linux in the early stages (others were available for other platforms too). I personally think that GStreamer was a good choice. I think it is a very powerful system, but it's not for the feint hearted. I wont begin to pretend that I understand it (although I have hacked my way through some GST code!), but the principle of it's operation seemed to fit the needs of the Phonon project very nicely.

Sadly the GStreamer backend was plagued with some problems. I wont go into the details (primary because I don't really know them all!) but many KDE developers felt that it just wasn't appropriate and decided to write a Xine backend for Phonon instead (NOTE: This is not actually correct. Please read comment by Kevin Kofler below). This matured quickly, but it was always had it's own set of problems that resulted in many hacks to be introduced at the application level. This was very bad as it meant that those of us (Mandriva included) who stuck with and helped fix the GStreamer backend were sometimes left a little out in the cold due to these hacks. Ultimately this story has concluded with the Xine backend being more or less abandoned now, and renewed focus being put on the new VLC backend. VLC, or more accurately "VideoLAN", is a multimedia framework, very similar to GStreamer. It has a very well maintained code base and support for numerous codecs and formats. It is also totally cross platform, working with Windows, OSX and of course Linux and several embedded OSs to boot. Having a single Phonon backend with uniform capabilities and support across all platforms is very desirable, so progress with this is quite rapid and an official release of the Phonon-VLC backend will be available in a matter of weeks.

But this is all about Phonon, not Qt Multimedia or Qt Mobility. So where does this all fit in? Well, Phonon has ultimately not really been developed much by Qt. That's not to say they have totally forgotten about it. I am assured that bugs reported will be looked at, time permitting, and that patches from downstream will be tested and merged, but it's fairly clear that it will not be actively developed further and no new features will come from the Qt side. Their new babies are Qt Multimedia and Qt Mobility.

Qt Multimedia, and arguably to a larger extent, Qt Mobility has been developed with a strong focus on mobile usage. Now that Qt is owned by Nokia this is not surprising! So what was the motivation for this and how does it differ from Phonon? Well, like Phonon it also makes use of existing projects to provide the core media decode capabilities. It will continue to use GStreamer for most "linuxy" systems and others for e.g. Windows and OSX. It differs however with regards to focus. It encompasses several additional features regarding the various needs for a typical mobile environment, such as still image and video capture (think cameras on a mobile phone).

There are several other key differences too but I wouldn't like to go into detail just now (I'm not really familiar enough with either Qt MM or Qt Mob to comment in any great depth), but suffice to say that this general abandonment of innovation in Phonon in favour of a new project has left several people in the KDE multimedia community feeling rather uncertain about what to do next; which horse to back!

Here at the KDE Multimedia Sprint in Randa, Switzerland, we were lucky enough to discuss the various issues with the Qt Multimedia/Mobility guys in Brisbane, Australia via a video conference. Well it was supposed to be a video conference, but despite it being between two groups of multimedia geeks, video completely failed us (various firewall issues for one solution and the lack of admin rights at the remote end for another!), meant we had to make do with a normal phone call! Such is life! Anyway, they were kind enough to give us an overview of Qt Multimedia/Mobility and let us know their approximate roadmap.

The outcome of this meeting was more or less that KDE Multimedia focus for the short term will remain with Phonon but the Qt guys will try and be more open about their plans and try and work better with the community longer term. At present, this is an uphill struggle as the VLC project has been very active in the KDE community for the last little while and seems to fit the needs in both the short and relatively long term of KDE, albeit in a way that is not currently exposed via Phonon API. This previous lack of communication is something that will be hard to overcome, but using a Qt integrated and supported solution definitely has advantages in the future - if they really mean it this time! The fact that the code is also used on Nokia mobile platforms give a fairly good reassurance that this will indeed be the case at least!

The rather obvious question of "Why not extend Phonon?" was also raised. While it was difficult to hear exactly the responses, the general reasoning seemed to be that there were some design mistakes in Phonon that basically meant that each individual feature that needed to be added, actually needed to be done at the backend level first (for each and every backend) and then exposed out through the API. With the Qt Mobility approach, more of the core features and functionality can be implemented once, with it only reaching out the the underlying platform specific system for specific and defined operations.

We enquired about the possibility of having a single Qt Multimedia backend written for Phonon, that allowed the Phonon API to continue in use at the application level while using the underlying systems/framework provided by Qt Mobility and/or Multimedia and deprecating all other Phonon backends. This is attractive in the sense that applications to not need to change horses mid stream and can gradually move over to using a pure Qt Multimedia API if indeed that is determined to be the most desirable outcome. That said it is also unattractive in some ways too. This adds yet another layer of abstraction to a system that many people argue is a layer of abstraction too far in itself! i.e. Application uses Phonon which uses Qt Multimedia which uses GStreamer; Why not just Application uses GStreamer? (there are of course many reasons to add a layer of abstraction; cross platform support being a primary one; but two layers is arguably not the best situation, especially on mobile platforms where wasted CPU cycles really hurt). That said, it would still be an acceptable stepping stone for most people. Regardless, there was no actual commitment to this eventuality from the Qt side. It was seen as a good idea but it's likely not something that there will be time (aka budget) for developing in the near term, which is understandable.

Many of the problems with Phonon still remain however. Qt Mobility will still require the setup of different underlying support libraries on different platforms. Just as before, GStreamer will be required for it to work on Linux and downstream (distributions and, to a lesser extent, users) will need to ensure that this is all configured and working correctly. As with the original Phonon, this backend will be different on different platforms, which is not ideal for the downstream.

So in the short to medium term (e.g. the next one to two years at least), Phonon will continue to be the primary media framework on KDE, and development on the Phonon-VLC backend will be seen as the best way forward to provide a standard experience across all platforms. GStreamer as a phonon backend will also continue, although the mplayer and xine backends should be considered deprecated.

With regards to Qt Multimedia and Mobility development itself, the Qt guys will try to be more open; opening a private mailing list to more access and generally being more communicative.

Those of you who regularly read this blog will be no doubt wondering where the PulseAudio connection is. Well, from what I gather, this is a primary difference of Qt Mobility and Qt Multimedia. Qt Mutlimedia seems to use ALSA directly and thus is not ideally suited for mobile situations (PulseAudio's timer based scheduling now being pretty much universally accepted as being the preferred approach on mobile platforms). Qt Mobility does actually use PulseAudio indirectly via GStreamer, but it does not seem to do anything special with regards to the "buffer-time" or "latency-time" attributes of the pulsesink (not that pulsesink is actually referred to directly in the appropriate place anyway from what I can tell). These attributes map directly to the tlength and minreq attributes of the buffer metrics in a PulseAudio stream. These are very important when it comes to mobile environments as the general aim is to provide as high a latency as possible. Higher latency means lower power usage (and for those wondering, this does not necessarily affect A/V sync on video playback - it's just about how much data you pump into the audio buffers before sleeping until you know it'll be almost empty). For a system that deals with audio playback on a mobile device, this is very important.

Now Phonon is also similarly ignorant to these properties when using PulseAudio. The the help of folk from Intel we're going to look at increasing the defaults in Phonon when used with GStreamer pulsesink to see if higher latencies would bring benefits without any drawbacks at the application level (I don't think any of the uses of Phonon will have problems with this approach), but we'll also have to consider how best to expose this via the Phonon API to allow the application level to state more explicitly if this matters. When capture APIs become possible on Phonon this will become important. The whole question of how much thought has gone into this kind of power saving is Qt Mobility is still very much at the forefront of my mind. I hope to be able to discuss things in more depth with the developers in due course, hopefully influencing them to extend the API in a similar was as described above.

More generally, those of us in the KDE community will try to get involved with Qt Multimedia/Mobility but until such times as it is easily configurable on the Desktop or there is a Phonon backend based on it, it will be hard to get involved too deeply.

Overall I think everyone has an open mind, but with the current focus on VLC and the functionality it provides, it will be the most interesting bits for most KDE Multimedia guys for the short term.

With regards to all of the above, I hope I've made fair and levelled comments. If you feel this is not the case, please tell me so in the comments below. I will happily retract or rephrase or reconsider any point if a suitable argument is made. I do not intend to represent anyone else in the KDE community in a way they do not feel is accurate so I'll be happy to both comment and edit the article with suitable annotation if this is deemed necessary.

Edits:

1. 27th May: Clarification of my incorrect statement of when/why Xine backend was introduced. Thanks to those who pointed it out.

It's an annoying topic. Not many USB surround sound cards exist anyway, even less of those are good ones. But after an hour of searching the web and reading various articles, I've finally found a USB surround sound card that works well in Linux:

Terratec Aureon 5.1 USB MK-II

Usually the problem is that terratec's drivers are not too good. Luckily, the Linux drivers are often excellent due to the open source community's investments in them. And here the case is similar: Terratec was smart enough to use the standard USB interface, so they need not write drivers for any OS and the Linux USB Audio driver works perfectly - already for a long time.

But German reviews on Amazon suggest that the digital output does not work with surround sound, but only stereo. This means it's just an option if you want analogue sound.

"Enforce USB bandwidth allocation (EXPERIMENTAL)" must be deactivated in the kernel.

If you still need some help with the setup, e.g. for games, try this page.

If you want a really good USB stereo sound card with digital out for Linux (cost " 20 €"), and you're up for some work, here's a German guide how to build your own.

But according to German Amazon reviews and this website, the

Creative SoundBlaster X-Fi Surround 5.1 USB

is also supported in Linux - at least for stereo. A German guide is in the Ubuntu forums, but it seems it should work with any current Linux.

Both cards have driver issues. It seems the Creative card doesn't work well in Vista x64 (no/bad subwoofer sound). I think in the end I will decide for the Creative card anyway. It's much newer and probably has better sound than the much older Terratec box.

pulseaudio: Spinal Tap FTW! We're trying to decide on a standard for software amplification and the current thinking is 11! (dB that is!) (by Colin)

pulseaudio: Latest PulseAudio+Phonon+Amarok+KMix support info+screencast: http://colin.guthr.ie/2010/04/slide-around-the-sound/ (by Colin)

Just a quick update on various KDE+PulseAudio changes I've made recently. This is more of an update from previous articles than anything ground breaking in it's own right although there is a nifty (IMO) new feature now available.

Firstly, it was pointed out to me that the use of a QEventLoop was not the best approach when connecting to PulseAudio from a KDE app. I used this technique in Phonon and while it seemed to work fine in testing it did trigger some problems in some unlikely places: namely this bug as reported against Okular! So I changed how the connection test to PulseAudio is run to make it blocking. This has so far appeared to fix the problems \o/

Now on to something interesting I've recently added support in Phonon for "hijacking" the volume change requests to allow them to be pushed directly to PulseAudio, totally avoiding the backend itself. In additional it is possible to listen to changes from PulseAudio and then emit the relevant Phonon signal to let applications know the volume has changed. This allows volumes changed in a PulseAudio mixer (pavucontrol and now KMix too!) to be reflected in the Application GUI (should it present it to the user).

Phonon provides it's own VolumeSlider widget that applications can use. I had to make some changes to this to avoid some complicated race conditions. As volume changes in PulseAudio are asynchronous, the change request may be followed some time later by the actual update notification. For this reason the it was not possible to use the "traditional" blockSignals(true) approach. I settled on two flags to achieve the necessary protection and it seems to be working well so far in my testing.

Then I tested Amarok and found it didn't work at all This is because Amarok has it's own volume slider widget and does not listen for any volume changes from the Phonon layer. I was able to make the necessary changes to support this without too much trouble, but in doing so I had to change the mute behaviour slightly. Due to (I'm told) legacy reasons, when the volume is muted, Amarok would also set the volume to 0. This was reflected in the GUI by the slider showing 0. While it would be possible to keep this behaviour it's not really desirable these days (the OSD always said "Volume: 0% (muted)" when muted which made it rather redundant to show the %age!) when mute is supported properly. If GUI feedback of mute status on the slider itself is desired (the image does change to a mute symbol already) then it would likely be more appropriate to change the slider colour to grey or similar (which is also how other media players - e.g. VLC represent things).

Anyway, with these issues fixed and suitable support for the feedback race conditions added to the Amarok slider code too, I've now got a rather nicely integrated solution

Here is a quick screencast I recorded to show you how all these changes: Phonon, KMix & Amarok all work together

(direct link)

As always, these latest changes are in Mandriva Cooker. The Amarok changes are not in the 2010.1 Beta 1 just recently released, but they will be part of Beta 2