SpectroGraph 2.0 for iTunes (OBSOLETE?)

by Alexander Thomas (aka Dr. Lex)
Mail

Note: third-party visual plugins are broken since iTunes 12.6. In typical Apple fashion, this happened without any explanation. However, SoundSpectrum still makes working plug-ins, and they have also shed some light on what actually happened. Apparently plug-ins can again be used in OS X Sierra or newer. Sure enough, I have so far managed to make a new build of SpectroGraph that shows up in iTunes in (High) Sierra, but it shows only a black window.
Any help here would be greatly appreciated: contact me if you know how to make a working visual plug-in. Most helpful would be a skeleton plug-in XCode project that demonstrates how to render something very basic with OpenGL, for instance just a rectangle with R,G,B colour from 3 arbitrary values out of the spectrum data.

What is it?

Screenshot

SpectroGraph is a visual plug-in for iTunes for Mac OS X. Visual plug-ins were introduced in version 1.1 of iTunes. The plug-in started life as a simple modification of Apple's iTunesXPlugIn sample code, but has grown beyond that in newer versions. Of course it's completely free!

The plug-in displays a plain spectrogram (hence the name) of the music. A spectrogram is a graph of the frequency content of a sound in function of time. The left channel is plotted in red, the right channel in green. This means that a mono sound will show up in pure yellow (because in RGB space, red + green = yellow).
I made this plug-in because I needed something which represented the actual spectrum of the sound as realistically as possible. Amidst the existing visual plug-ins for iTunes, I only found funny gizmos which produce a fancy display which moves to the music. That's nice to look at, but not handy if for some reason you need an accurate display of the frequency content of the music.

The immediate reason why I made this plug-in, was to be able to see hidden images in the frequency content of certain songs. Some songs by Aphex Twin contain hidden images (for instance, the ‘Windowlicker’ EP and especially the ‘Equation’ song), but they may look distorted because he used a harmonic frequency scale while this plug-in has the usual linear scale. The song ‘The Warning’ from the Year Zero album by Nine Inch Nails also contains a hidden image, originally to be found in a pre-release version of ‘My Violent Heart’ (it is hard to see in Spectrograph though). But even with a lot of ‘ordinary’ songs, the frequency display is quite interesting and fun to look at. To get the best renditions of hidden images in songs, you should use the original CD or a high-bitrate encoding.

Features

You can change the look of the spectrogram either by accessing the ‘Options’ dialog (in iTunes versions prior to 7, you can use the button in the top-right corner when the plug-in is active, from version 7 on you need to use the ‘View’ menu), or by using the keyboard. You can choose between horizontal and vertical scanning or scrolling, color and grayscale display, anti-banding, inverting of the spectrogram and change the maximum drawing speed. You can also force the plug-in to re-start drawing at the left or top at any moment.

Mind that the maximum drawing speed is not only limited by the software setting, but also by how much processor time the plug-in gets. If you run heavy background tasks or stretch the display all over the screen, playback may become slow and irregular. Run the plug-in in full screen mode for optimum performance.

Version 2.0 is a major rewrite of the plug-in. The drawing routines have been converted from the archaic QuickDraw to OpenGL. This was necessary to make the plug-in compatible with OS X Leopard, but it has a lot of nice side efects. The quality of the display is improved and the drawing speed is much more stable. Moreover, it allows to do more fancy things, for instance scrolling the whole image (and more). The only drawback is that a recent graphics card is required to get similar maximum drawing speeds as with the older version. On older Macs which can't run Leopard anyway, I recommend using version 1.2.1 because the new version won't even be able to run at 'normal' speed.

Note 1: SpectroGraph runs extremely slow in iTunes 10.0. It runs faster in iTunes 10.0.1 and newer, but the maximum speed is still limited to about one third of what would be required to view all detail in the spectrogram. There's nothing I can do about this, iTunes 10 has a built-in limit on how many chunks of spectrum data it can deliver per second. If you want SpectroGraph to run fast, reinstall or do not upgrade from iTunes 9 unless you really need any of iTunes 10's new functionality. Or, ask Apple to either drop the data frame rate limit or at least increase it to 180 frames per second. You can also file a bug report and refer to bug #8393710.

Note 2: the plug-in will display nothing if a file is played which is not MP3, AAC, or uncompressed audio. This is a ‘feature’ of iTunes, and it's also the case with all other visual plug-ins (they might display something, but will not react to the music). If you want the visual plug-ins to react to other files too, the only thing you can do is ask Apple and hope they'll listen. By the way, the equalizer, audio enhancer and sound check features also do not work for other formats, so you might want to ask to fix that too while you're at it.

Download

Mac Universal

Download SpectroGraph 2.0 (gzip compressed tar archive)

The download includes an mp3 file which will produce an interesting result when viewed with the plug-in!

If you are unable to run iTunes in 32-bit mode, I do have a build of the unfinished 64-bit conversion of the plug-in that was made by Ivan Wick a while ago. Some things about it are broken, for instance the settings dialog and some of the shortcut keys, but it is mostly functional. Of course, the “use at your own risk” warning applies doubly to this.
Download unfinished 64-bit build (ZIP archive)

To install, unzip the file, and drag the ‘SpectroGraph.bundle’ into the folder “Library/iTunes/iTunes Plug-ins”, either in your home directory (the plug-in will only be available to you) or in your root directory (the plug-in will then be available to all users). Then re-open iTunes and choose "SpectroGraph" from the 'Visuals' menu.

Remember, this plug-in is Mac OS X only. I didn't have the time to work on a Windows version yet, and given the fact that I have near zero experience in programming for Windows, it's unlikely that I will do so in the near future. If you are a Windows developer with some spare time, download the source code below and get crackin'.

SpectroGraph works in Mac OS X Lion, but only if iTunes runs in 32-bit mode (this can be set through the ‘Get Info’ window in the Finder). Unfortunately, upgrading SpectroGraph to 64-bit proves to be more difficult than a simple recompile, so it may take a while before a 64-bit compatible version is ready.

This version will only work in OS X 10.4.x or newer. You can download older versions of SpectroGraph to use on older systems.
Download SpectroGraph 1.2.1 (gzip compressed tar archive) (universal binary: will only work in OS X 10.4.x)
Download SpectroGraph 1.1 (ZIP archive) (works from OS X 10.3.0/iTunes 4.0 on)
Download SpectroGraph 1.0 (StuffIt archive) (works almost anywhere)

Version History

Version 1.0: initial release.
Version 1.1: made the source code available and added 'anti-banding'. The latter is an attempt to reduce the banding artefacts which often occur with deep bass sounds. It provides a cleaner output and is on by default, but can be toggled with the 'b' key or in the settings dialog.
Version 1.2: converted to universal binary, added speed limiter.
Version 1.2.1: fixed an overflow bug in the speed limiter which could cause the plugin to stop drawing.
Version 2.0: complete rewrite, using OpenGL for rendering, added scrolling.

Ideas for improvements

If you didn't already notice it from the huge delay between version 1.2.1 and 2.0: I have very little time to work on this plug-in. Moreover, the following pile-up of issues have been eating away at the little motivation I had left:

2012/01/11: Good news! Someone took the effort in his spare time to rewrite SpectroGraph with the latest iTunes visuals SDK, making it 64-bit compatible. Moreover, this seems to allow a higher update rate as well. When this new version has been polished, I will make it available here and the source will be available on GitHub.

On my TODO list are the following points, in decreasing order of importance/feasibility:

  1. Add 64-bit support for iTunes in OS X Lion. Spectrograph works in Lion, but only if iTunes is launched in 32-bit mode.
  2. Add some kind of grid to allow a more quantitative analysis of the data, and the possibility to ‘zoom in’ on certain ranges of frequencies.
  3. Create a Windows version. With the conversion to OpenGL, this has become much more viable. However, there's still a lot of work to do.
  4. Implement my own FFT and logarithmic filter bank, and calculate them from waveform data. This would allow higher resolutions, a proper fix for the banding issue, and offer the option of a true logarithmic frequency analysis. The big problem is that it requires full continuous waveform data. Until iTunes 10 it was still theoretically possible (albeit very problematic and error-prone) to reconstruct it from the visualiser data frames. However, due to the forced cap on the update rate, it has become completely impossible.
  5. Start from scratch and write a stand-alone program that can capture audio from any application (in a similar fashion as Audio Hijack) and visualise it. This would solve every problem currently caused by limitations of the iTunes plug-in framework, but I really don't have the time to start a project like this. Although I could recycle a lot of the current SG code, finding out how to obtain raw audio from OS X' innards is a total mystery for me at this point.

If you think you can tackle one or more of these problems on your own, you can simply download the source code and give it a shot. Remember, this source code is provided on a “share your improvements”-basis: if you release an updated version of the plug-in, you should also release the updated source code. The project will only open in XCode 2.4 or newer, but you can easily reconstruct the project in older versions by taking the iTunesPlugin sample source code from the Apple site and replacing its source code files with the ones from this project.

Download source code (bzip2 compressed tar archive)

Honto.com had created a custom plug-in, based on the source code of this one. The display is mono, but it supported multiple color schemes and constant drawing speed. The webpage seems to be gone though.

The author of this software is not responsible for possible damage of any kind caused by the use of this program. There is no warranty of fitness for any purpose. Use at your own risk!
This software is free and may not be sold. No explicit permission of the author is required for reproduction and distribution of the software. The compiled SpectroGraph bundle may only be distributed if the original files, i.e. the ReadMe file and the SpectroGraph.mp3 sample, are included unmodified.