AI_Jam

AI_Jam is an audio-visual interactive web-based installation which people can use it as an instrument to compose sound together with AI simultaneously. Seeking a stronger and closer bonding and finding the harmony between humans and AI to create a direct interaction for making music is the concept here. The background drumbeat is generated by AI (Google Magenta) and the beat pattern can be switched by the audience’s preferences. While the beat is playing in the background, the audience can freely drag the mouse to different positions on the screen to generate sound (or use *Wii-nstrument simultaneously). By turning on the “color-tracking mode”, instead of mouse-dragging , the audience can move a certain assigned color object (yellow in this code/case) detected by the webcam(tracking.js) in order to generate sound (Tone.js). Imagine, the audience is just like a lead guitar player by dragging the mouse or moving the color object to generate the melody, while Artificial Intelligence as a drummer takes parts in the rhythm. It attempts to lower the difficulty of playing the instruments and realizing the dream of jamming with AI without any sophisticated skillset. The whole piece is done with P5.JS (https://p5js.org/).

*Wii-nstrument is an instrument set composed with 4 Wii-nunchuk mainly designed for real-time/live audio-visual performance as well as interactive installations.