See, slowing audio properly requires a lot of CPU power. On a desktop Mac, there’s plenty of power to spare these days. Unfortunately—this is not the case in the mobile world. Because Capo is both graphics- and audio-intensive, it eats up a lot of CPU power.

When version 1.0 released, Capo was eating up about 80% CPU while playing audio in the foreground (i.e. with graphics visible) on a 3rd Generation iPod touch, and maybe 75-80% on an iPhone 4. The CPU/RAM improvements on the iPhone 4 were offset by the increase in pixel density pushing the GPU a little harder. C’est la vie.

If I wanted to do anything else with the app, I’d have to fit it into approximately 20% of leftover CPU.

Capo’s effects are both graphics and CPU-hungry, so I had my work cut out for me to add them in the next version. This was especially challenging, because I wanted to keep the live-updating spectrums in the effects control backgrounds as I have on the desktop version. Go big or go home, right?

Luckily for me, there were optimizations added to the slowing engine I use, as well as some further optimization work on my part. That 20% margin increased a little bit more, so I was able to shoehorn an initial implementation of the effects UI into the mix.

The initial approach simply sucked—it caused various skips in the audio, and the graphics were very choppy. It was especially noticeable when you’d scroll the effects controls while they were enabled.

The first culprit was my effects implementations. I built my own Equalizer, but it was taking about 15% of the app’s runtime according to Instruments. I had to take care of this, and re-wrote the tight loop using NEON intrinsics. That took it down to 1.5-2%—sweet!

I also had to re-write my FFT-based vocal reduction to use the Accelerate version of FFT rather than the implementation I previously used. I don’t recall the specifics, but I think the result was approximately 10x faster—the performance optimization team at Apple really kicked ass with their FFT implementation!!

Unfortunately, there was still about 30% of runtime being allocated to my live-updating spectrum drawing. I initially implemented the spectrum drawing using Core Animation and Quartz. That had to be thrown out, and I instead re-built the live spectrum views using OpenGL ES 2. A few custom shaders and other black magic made short work of that—reducing the runtime to about half of what it was before—15-18%. It cost me the prettier anti-aliased curves, but the speed boost was worth it.

In the end, I’m proud of what I shipped today. I know that the progress isn’t as quick as I’d hoped on the iOS side of Capo, but I think I truly nailed this effects implementation. Because of this extended effort, Capo will continue to be the best tool for learning your music by ear on your iPhone, iPod, or iPad.

I can’t wait to find out what else I can shoehorn into that “last 20%” in the future…

Since iTunes 10.4 was released recently with Cocoa under the hood, I thought I should take a look at the dragging situation again. I wrote a little test application that dumped the NSPasteBoard types coming in from a drag, and obtained the following list:

As far as I can tell, the com.apple.itunes.metadata type on the pasteboard is new in iTunes 10.4. Digging deeper, I managed to dump out the NSString contents of this item on the pasteboard:

This looks a whole lot like the information that was present in the old CorePasteboardFlavorType item. You can iterate through the list of tracks in the dictionary, and there’s your file’s location URL!

Still, there’s a problem…

While it’s great that we can now (slightly more) easily obtain the track’s location, it’s quite odd that the com.apple.pasteboard.promised-file-url item on the pasteboard isn’t enough to get the dock icon to recognize the drag as a supported file. Capo happily accepts .m4a files, but this drag is apparently not good enough for the Dock to accept it. Developers shouldn’t have to implement a service to accept song drags from iTunes.

Using the dictionary on the pasteboard could also be fragile over time. If someone on the iTunes team decides to manipulate the “schema” inside the dictionary in a future version of iTunes, there will be some painful transition required. Luckily for us, there appears to be some kind of versioning present in the dictionary, so we could cope with multiple versions of iTunes drag formats in the future. (Provided the schema-changer in question is smart enough to update the versioning…)

What about the old solution?

It still works, for now. There’s no need to go out and rush to fix this issue today: the old-style item on the pasteboard is still present. Going forward, you just have to add the new pasteboard item UTI, in addition to the one we declared to wrap CorePasteboardType 0x6974756E in your service’s NSSendTypes.

Believe it or not, Capo for iOS was almost exclusively developed in Xcode 4. Unfortunately, Apple did not allow app submissions with the beta tools, so I was forced to bounce back and forth between Xcode 3 and 4 for Capo version 1.0. But I digress.

I’ve finally got around to moving the Capo project to Xcode 4 just the other day, but I hit some bumps in the process. I created a workspace containing the projects for Capo, and its companion frameworks, but couldn’t get a solid ‘Copy Files’ build phase to work properly.

You see, when you drag a framework from the project navigator into your ‘Copy Files’ build phase, it appears to use a pre-baked relative reference to the framework’s path. Unfortunately, that means the following things are baked into the path:

• The build configuration (Debug vs Release)
• The derived data folder name

Both of those items are moving targets, depending on what scheme/action you’re building for, and where you’re building your workspace. The derived data location for a workspace is based on a hash of the location of the workspace itself, so that could theoretically change on a separate machine, or even on the same machine between two different folders!

Ideally, the framework targets would be specified for the Copy Files build phase in a way that makes sense no matter what scheme you’re building, or where you’re building it on your machine. Furthermore, it’d be nice to get Debug versions of your frameworks when debugging your app, wouldn’t it?!

Of course, I’ve filed a bug to document this issue: rdar://9722234.

So, my solution to the problem in the meantime is to use a script build phase. I created a script in Capo’s project directory called copyFrameworks.sh, and it looks like this:

Be sure that you run chmod+x on that file in the terminal, so it executes.

Now, for my new Run Script build phase, I use this:

That handles both the copy step, and also the subsequent removal of the frameworks’ headers. Note that I don’t use the “Input Files” list in the script to specify the list of frameworks. I know it could be done, but I’ve just not gotten around to it.

Hope this helps!

alias mvdiff="mvim -R - &>/dev/null"

Now, whenever you want to view diffs of changes in your favorite SCM, it’s as simple as doing something like this:

hg diff | mvdiff

I can’t seem to wrap my head around the fancy diff tools that are out there, and unified diffs + syntax highlighting seem to work best for my eyes.

Hope this helps!

Subclassing UIButton isn’t entirely straightforward, so I figured I’d share my approach for getting this to work. Here’s the code (Note: Requires the iOS 4.0+ SDK, due to the lack of an ivar!):

There are a few things you need to know about my approach, since I hit some road-blocks while subclassing UIButton on my own.

For starters, you must call setImage:forState: in your chosen initializer, rather than overriding imageForState:. Things won’t work if you try and go that route. I think this is because UIButton is accessing the underlying data structure that holds its per-state UIImage instances directly.

Second, it’s necessary that you call layoutSubviews to get the button to update its image when you modify its state programmatically. Calling setNeedsDisplay: does nothing, because UIButton likely doesn’t do anything in drawRect:.

Finally, you may notice that I asserted against the UIControlStateApplication masks. This is partially because I wanted a record of why I chose the values in the .h file, and partially because I can get some kind of warning in the future if the masks go away (or change.)

Hopefully this helps some of you iOS developers that may be struggling with custom UIButtons!