The Camera Color Sound system uses a camera to grab an image. The Python program language selects the appropriate region(s) of the image, and then process that data to generate the information to be presented to the user. Next Python processes this information to generate the actual sounds the user will hear. This information is then transmitted to earphones using bluetooth, or some other appropriate means. This symple systems enables much interesting and useful research.


The Hardware

The hardward consists of a camera, a computer, earphones plus connections and circuit boards. A possible system is shown below. The computer is a Raspberry Pi Zero W (Adafruit 3400), the camera is a Raspberry Pi camera (Adafruit 5657), and the circuit board is two boards borrowed from the Hex computer. This system could be mounted on a hat, such as the beret shown below (Amazon: Gajaous British Military Berets for Men and Women) or a belt buckel, or some other convenient place. This system is only for illustration.

The Software

The last complete system was Thor, written in Python 2.7. The later systems written in 3.x had problems with dependencies and other things. So most of the examples we give will probably be from the Thor system. I'd be happy to chat. -- Please email me using the email link at the top of the Biiboards.com web page. In this section it is assumed the reader has a working knowledge of the Python computer language.

Camera Software

Adafruit has a nice tutorial on how to get the camera software up and running at

https://picamera.readthedocs.io/en/release-1.13/

Below is a sequence to generate sound from the camera image. First, the nine pixels at the center of the image are averaged, giving three numbers, such as (Red_value, Green_value, Blue_value), where each of these values run typically from 0 to 255 or 0 to 1, or ... . Check your camera. I am assuming the camera values were given in RGB, 0 to 1. If not, colorsys gives conversions between the many color systems; (search the at top of the colorsys reference for RGB).

https://docs.python.org/3/library/colorsys.htm
https://www.tutorialspoint.com/colorsys-module-in-python
https://en.wikipedia.org/wiki/Color_model

Color models get very complicated! After playing with a number of these models, I found that RGB is the easiest for my ear to distinguish.

The piano musical notes given under the Alphabets section above are numbered from 00 to 89. Where the 00 and 89 keys are silence, ie playing no note. Also the 00 key is usually saved as two separate and identical files, 00 and 0, and 01 saved as 01 and 1 ... . This makes calculating and playing the keys much easier.

I used the 40 piano notes between C3 and D#6 for many complex reasons, (play time, stereo effect, clarity, harmonic effects, and I don't really remember all the reasons). This is a very interesting and extensive research area.

Anyway, if you take the 0 to 1 Red_Value and multiply by 40 (the number of notes I used) and add 0.99999999, and then INT that number (take the whole part) you get the number of the note corresponding to that Red_Value.

Now play that note, I used Pygame, which is an awesome set of programs. Here are some references to web pages. In general Sound is used for short clips, like piano notes, and Music is for long clips like playing a whole song. Also Adafruit has much interesting hardware.

https://www.pygame.org/docs/ref/mixer.html
A general summary of Pygame

https://www.pygame.org/docs/ref/music.html
Discusses Music and the mixer

https://pythonprogramming.net/adding-sounds-music-pygame/
A discussion and example of Music and sound

https://blubbervision.neocities.org/pygame/ref/mixer
Searches the internet for examples

Here is an example of playing a chord with a 14 millisecond delay between the first note and the rest.

def Sound_Chord(self, note_1=28, note_2=32, note_3=35, note_4=89):
snd1 = pygame.mixer.Sound('./Piano/' + str(note_1) + '.aiff')
snd2 = pygame.mixer.Sound('./Piano/' + str(note_2) + '.aiff')
snd3 = pygame.mixer.Sound('./Piano/' + str(note_3) + '.aiff')
snd4 = pygame.mixer.Sound('./Piano/' + str(note_4) + '.aiff')
snd1.play()
time.sleep(0.015)
snd2.play()
snd3.play()
snd4.play()
return()

To play the RGB sequence, place delays between the first three notes, so each note plays separately, and delete the snd4 line. You may wish to overlap the notes, I did not play with this.

All of the above is conceptual. What really matters is what the actual user wants !

Three quick items

A convenient power source is the PowerBoost 1000 Charger or the PowerBoost 500 Charger (Adafruit 2465 or 1944). Plugged into a 5 V USB power source, these chargers will charge a Lithium Ion Polymer Battery (Adafruit 1781 or 328 or 258 or 353 ...) and also power the computer. Unplugged from a 5 V source, the battery will power the computer. These units also have a convenient control pin, EN, which will disconnect the PowerBoost from the computer.

At my house in the winter time the air becomes very dry and static charges will destroy Raspberry Pi computers and other things. To solve this problem I bought an anti-static mat. (I have an Rosewill Anti-Static Mat with Grounde Wire, size 23.6 by23.5 inches with grounding cable. There are many good anti-static mats on the market.)

Ground Loops, Ground Loops, Ground Loops. If it really won't work, try looking for ground loops. This happens when the connecions for a signal or ground or power supply makes a loop. Induced voltages in the loop can work havoic on circuits. If it's a problem, find a good article and read it. Does anyone have some recommended articles? I'd be glad to list them.

Jump to Top of This Web Page