Kovács Balázs / Xrc

[ english / magyar ]

Projektek:

Szövegek:

CV

works/2021-Websonify.txt

Webaudio-based sonification solutions that I once created for the Újegyetem alternative higher education initiative.



https://makker.hu/szonifikacio/

These are currently based on parametric mapping sonification, including real-time linear and interactive approaches. I recommend the attached presentation as background information:
https://makker.hu/szonifikacio/prez.html

Linear solutions

For practical reasons, linear solutions do not allow changing the order of reading, but the user can influence its speed, input data, voice-playing parameters, etc. Linearity here is not a constraint, but a convenient solution compared to interactive mapping: the user can pay more attention to the output results and feedback control. The first two of my examples are not really visual, in the sense that their input data are texts entered by the user; however, their sound representation is based on the x/y coordinates and size of the letters, i.e. visual information. Both Letter Games assign sound parameters to the letters of the entered text, but the first one allows the user greater control over the individual parameters, while the other one is more autonomous in this regard.

Betűjáték 1 és a Betűjáték 2

Also linear playback, but with image data, is implemented by the two image sequencers, which follow small pieces of the loaded images along the x/y coordinates (raster scanning), and use the resulting pixel information (a color channel or hue values) as the basis for the sound. The first image sequencer uses piano pitches for this, and the second uses sine waves.

Image sequencer 1:

Image sequencer 2:

Interactive solutions

Instead of linearly reading back the sequence of information, the user can be entrusted with the exploration of this data - in this case, the use of the tools is much more intuitive, but at the same time it also requires user activity, which reduces a certain type of attention. I have designed each of them so that instead of existing images, you can also upload your own files.

The Image Player monitors the position of the mouse/pointer, and if it is over the used image, it returns the corresponding image information:

The Drawing Sequencer analyzes the shapes hand-drawn by the user and plays the data associated with them in a way that assigns different tones to different colors. It is also possible to load photos. (Technical information: you need to enlarge the page to see it better:)

Instead of visual information, the Smarnisonifier again uses alphanumeric values ​​and their data: sounds can be played by searching and moving the node-based structure created for the Smarni workshop. In the following case, you can work with data containing the keywords and connections of the two workshop presentations: