Musical Browser

After a tough year I finished my graduation project . The result is a browser which translates html and style properties into sounds. Unfortunately it is not possible to make the current application webbased. A lot of research needs to be done to increase the accuracy of the sonification of the web. In the future I hope to post some video’s to demonstrate the working, for now the sonification can be heard here.

Documentation

The full paper

Abstract

Is it possible to make an interpretable sonification of the web? To answer this question the Musical Browser was build, The Musical Browser makes it possible to explore the web, and sent the HTML to the parser. The parser translates this model into data which a sound synthesizer understands. The synthesizer software turns this model into sound. This separation made it easy to experiment with the sonification.

Several tests were conducted with custom created, preselected and random webpages. These test provided a granular approach to interpretation. From webpages with a distinct property, (very dark webpage) to more average webpages (colorful webpage). For the last test predictions were made about webpages after hearing the sonification of these webpages.

The result is that the musical browser’s sonification is interpretable for the HTML tags and some style properties of a webpage. But the sonification is not sufficient to make an exact prediction of the visual display of the webpage.

OpenSoundControl list from Processing to Max/MSP

During my graduation project I needed to get a 2 dimensional list from Processing to Max/MSP. I knew my best option was Open Sound Control. First you need to CNMAT objects for Max/MSP, these objects make Max/MSP work with OSC. Processing needs the oscP5 library

OSC does not make it possible to send multidimensional arrays, therefore this only works with multidimensional arrays with the same data type. Now the code, processing sends an Integer array of an index and 4 values.

Processing Code

import oscP5.*;
import netP5.*;

OscP5 oscP5;
NetAddress myRemoteLocation;

void setup()
{
  size(400,400);
  frameRate(25);
  /* start oscP5, listening for incoming messages at port 12000 */
  oscP5 = new OscP5(this,5555);

  myRemoteLocation = new NetAddress("127.0.0.1",5555);
}

void draw()
{
  background(0);
}

void mousePressed()
{
  /* in the following different ways of creating osc messages are shown by example */
  OscMessage myMessage = new OscMessage( "processing" );

  myMessage.add(new int[] { 1, 2000, 1000, 5, 5, 2, 400, 500, 5, 6, 3, 200, 560, 5, 6 }); 

  /* send the message */
  oscP5.send(myMessage, myRemoteLocation);
}

Max/MSP part

Now in Max/MSP we need to update the objects to their maximum buffersize and their maximum listsize. To extract the data we need to slice the list to remove the “processing” String and then divide the list in chunks of 5, the last step is to unpack these object to get to the data.

max msp patch screenshot

As you can see you need to set the buffersize for the OpenSoundControl object in bytes. For the zl object you can set the maximum list size.

update ( 09-03-2010 )
You can download a sample application with a processing patch as well as a max/MSP patch.

Musical Browser

The last episode of my Media Technology ( at the Leiden University ) study has begun. I will post regularly about my progress. But now the project.

As a front-end developer i am highly interested in web markup, and the idea came to me to use this markup to generate sound/ music. The ultimate goal is to make this sound/ music resemble the style and semantics of the webpage. In short a musical browser.

Thesis
Is it possible to make music from websites?

Idea summary
There are millions and millions of websites available. Every website has been built with a unique structure, design en behavior. Wouldn’t it be cool to use all these information to make music? Websites are made of HTML, JavaScript, flash and other techniques one technique may be less accessible then the other. It might even be possible to extract the style information of websites and play with it.
Of course the term music is subjective, especially from people to people. However I will try to get some rhythm and melody out of the websites. Another interesting thing I may encounter is the difference between manmade pages and machine made pages, like feeds and stock exchange information.
The goal is to build an interesting installation which allows users to interact with websites in an uncommon way, and let them be creative. The output might be horrible sounds which will not please the human ear or the next future symphony. However I will be very pleased if my thesis will be proved, and there is music in websites to be found.