I’ve been looking at several ways of picking apart a still image and converting it to sounds or music. Although programs like Metasynth already do this, the literal interpretation of image width / height as time / frequency, and of pixel intensity as amplitude can quite often result in some harsh sounds. The application as a filter is nicer, and could be easy to implement in MAX too.
To cross-reference with the emotive connections that are often made with sounds (blue, red, bright, dark etc) i want to try and derive literal colour names from the image, and then apply these to a range of sounds. So how do i do this?
Stage 1 is to get the image into MAX. I import an image directly into a jitter matrix for display, then into a second matrix with a fixed resolution for analysis. Using a 32 x 32 matrix results in a pretty, pixelated image like this:
Then it’s easy enough to get a RGB using the ‘getcell’ message. So, a bit of blue sky would have a higher ‘blue’ value than ‘red’ or ‘green’. So, how to associate that with a word? I could store away RGB values for each colour i want to identify, then compare the colour sampled to see if it matches?
Well, stage 2 is to convert this image from an RGB colourspace to an HSL one. HSL stands for Hue, Saturation and Luminance – meaning one single value (hue) is used to describe the colour. That can be mapped to a standard MAX ‘swatch’ object which looks like this:
Moving across the X axis you can see how each colour in the visible spectrum is represented. The Y axis represents Luminance.
Anyway, from left to right we get a variety of colours starting a red, going through green, blue and back to red. I’ve identified 12 colours across the spectrum which i’m going to use as identifiers for sounds:
So as each cell in the matrix is examined, is is cross-referenced with the X axis on the swatch object, it’s position here being mapped to the list of colours.
Stage 3. The H value is scaled and sent to a ‘nodes’ object which calculates a list of weighted values in 2d space… we create a node in the X axis for each colour, mapping this to a table which describes the colour!
There’s probably a much easier way of doing all this. If you’re reading this and you can see it – feel free to tell me!