I've been using arduinos in my artistic practice for a couple of years now. It's a great tool that has enabled me to realize projects that I couldn't have done without it. It has also brought me together with people from many different fields using the arduino in anything from interactive art installations to autonomous submarines. We even started a band here in Reykjavik called "Arduino Bandið" that only use arduino based technology like MakerBots, musical automates, physical computing, synths, DIY dolly rigs etc to create music and visuals. A lot of fun :)
It is easy to generate some really nice 8 bit LoFi pulse waves and glitchy punk console sounds with the bare bones arduino. But it is not really made for working with sound since it is an inexpensive micro controller and not a DSP engine. But there are shields (plug in extension boards) available that enables you to work with high resolution audio directly.
Another way is to do all the DSP externally and use the arduino with different sensors to build your own custom "midi controller". In my experiments with making a space controlled instrument I've been using photo cells, infrared proximity sensors and the more precise ultrasonic proximity sensors and then doing the actual synthesis and processing in Pd (Pure data) and Pduino.
@dewster: have you checked out Raspberry Pi?