ITP / General

Always On Project Ideas

Where I’m At is a sonic graffiti app that tags locations with audio. I’ve been working on it, and I’m currently getting nearby locations with Foursquare, saving audio files to my remote server.

Every location has only the most recent audio file associated with it, so there is some competition to tag the place with your sound. Like if you’re a graffiti artist and your work gets painted over. Maybe there can be a way to access history, but it will be an advanced feature.

To do list:

  • Ability to pull in audiofiles from the server, and data from the database (so far I’m just posting to it).
  • Visualize the nearby locations on a map with pins representing recent audio.
  • User logins (optional) will get contacts and create a list of friends who you can follow, in addition to browsing by what’s around you.
  • Send push notification when you’re nearby a sound somebody has left.
  • Hold up phone to your ear to hear a random recording, walkie talkie style, of a sound from nearby. Like WeChat or ChitChat. So when you get a push notification you can listen privately without plugging in headphones, or broadcast it.
  • Recognize if music is playing and/or tag songs via Spotify API… I wonder if I should focus more about tagging/listening to recorded music, less on recording/sharing audio?
  • What should the visual language be for an audio app?
  • How do I find the closest sounds for any given location? In the long run, I would need a better data structure, because I’d want to be able quickly search just for the locations that are nearby, and prepare their corresponding audio.
  • How can I visualize the frequency spectrum as audio is being recording/played back? The Media API doesn’t provide access but I’d like to figure this out.

More importantly, how would people want to use this? I’m still researching location-based audio, and I’d like to focus on one aspect that will help me make decisions about what to do next, how to prioritize features, the visual design etc.

1. This could be for sharing audio recordings with strangers and/or friends. I’ve been looking at Chit Chat for inspiration.

2. This could be for sharing location-based music – it tags music that’s playing like Shazam or lets you seach for a song and share it with your location. Example: Soundtracking.

3. If this was an app for creating and publishing site-specific Audio Tours, I could do both those things, and might have a clearer picture of who would use this. There would be tour guides, and tourists. It could allow users to trigger songs, or audio recordings, and might also need to orient based on the compass. There are already some apps for this, like toursphere, locacious and locatify.

Research for Thesis (excerpt from my private thesis blog)

TITLE: OnLine Open Source DAW.

 

SYNOPSIS: OLOS-DAW is a browser-based music production environment where everything—both the music and the underlying code—can be remixed. It is an open source, educational resource for communities to form around the intersection of music, code and the web.

DESCRIPTION: OLOS is a digital audio workstation, similar to Garage Band, but with a more modular architecture inspired by Max/MSP.

Web browsers now ship with many of the building blocks and optimizations of these advanced music production environments.

A community is forming around web audio and could benefit from a centralized place to share code. OLOS would provide that resource, and a spec for Web Audio modules. Modules could include synthesizers, effects, samplers, and algorithms, each with one or many UI modules.

Anybody who wants to make music can play with OLOS’ many different instruments and interfaces, or chain them together in a DAW. Creations can be saved, shared, and remixed.

Modules and songs be public/private, rated, curated.

 

RESEARCH: I think that a lot of the components have been done before, but I don’t think it’s been put together in quite this way.

Examples of online DAW’s:

 

Examples of online music/coding environments:

Attempts to unify Web Audio community with a spec:

  • Web Audio Components (why did this fizzle out in 2013? Will the project be revived? Follow up with everyone involved and figure out how I could help revive)
  • Open Music

Research into Web Audio community:

  • What would be most useful for Web Audio Community?
  • Who do I want to contribute the first modules?

Research into potential music creators:

  • How could this be used in the classroom?

Examples of other online code/music communities – what makes them work (or not)?

  • Csound, Faust, Max/MSP, Pure Data
  • Scratch

Open Source UI Components

Researching Potential Tools and Technical Challenges

  • Learn Angular.js because that’s what all the other DAW’s seem to be using
  • Google Blockly (similar to Scratch, a way to code using blocks)
  • What is the best approach for editable code in the browser?
  • How to handle the database?
  • Many more questions…

 

PERSONAL STATEMENT: Before coming to ITP, I directed an online music library inspired by the open software movement. It was inspiring to see communities form around shared music.

Now the web can be a platform for music creation, as well as distribution. I think this will excite a lot of people for a lot of different reasons, but most don’t realize it at this moment. I want to bring music creators and coders together to see what happens when these lines are blurred in a community forum where everything is open source / remixable.

I’m inspired by teaching after school workshops to young people with Scratch. Scratch uses code blocks and sprites to teach computational thinking, but for the millions of kids who use it, it’s just about fun making stories and games. Scratch was once just a desktop application, but now it’s online, and every project can be shared. There are millions of them. Every project has a “see inside” button and clicking that is similar to a github fork. I believe that remixing is a key part of the educational experience, and I want to bring the Scratch approach to music.

i started working with web audio in my first semester at ITP. I wanted to make a Web Audio Editor, and realized it couldn’t be done in Processing because any audio stuff I did would not be translatable by ‘javascript mode.’ This was a bummer because the whole point is that it would live in the browser, where people spend a lot of their time these days anyway, and where it could easily be shared. I studied Computer Music as an undergrad, and at that time it was hard to share interactive experiences—nobody outside of the department cared to check out my Max patches. But the Web Audio API is accessible to all—just open up your Javascript console and create a new AudioContext. This past summer I created p5.sound, an addon for p5.js that brings the Processing approach to Web Audio. In the process, I’ve developed a new appreciation for API specs.

I’ve been thinking about this idea for a few months, I’ve done some research and I’m excited to start prototyping so that I can trim down my goals to something that is actually feasible in 3 months.

Themes from the 1st WAC (outline)

This is just an outline off the top of my head of all the cool stuff that happened at the 1st WAC at IRCAM and Mozilla Paris.
  • Distributed performance
    • talk / demo
    • Lissijous –> distributed
    • Technical questions
    • Compositional questsion—maybe latency is something to be embraced by the composer.
      • how much control to give the audience, when they are performers, how much room to allow for improvisation?
    • Design questions:
      • phone as an interface – do you want people to interact with each other?
  • Port DSP to Web
    • 3D audio custom HRTF (IRCAM)
    • Meyda – audio feature extraction library – inspired by the Yaffa library (Python / C++)
      • using jsfft library to do fft themselves
    • various environments for running / optimizing C++ code
      • asm.js (Jari Kleimola DAW plugins for web browsers)
    • AudioWorker will make this all much more accessible and standardized.
  • Port frameworks to the web
    • Csound and Faust with emscripten and potable data client
    • Tone.JS inspired by Ableton, Max/MSP etc but built with the web in mind
    • WebPD – two separate projects
    • Recently converted to Web Audio:
      • Earsketch – more powerful than Reaper, transitioned 1 year ago
        • reference DAW paradigm while learning Python and JS
        • MOOC
        • Why is web better? No installation hurdles. Essential for education. Plus, accessible from anywhere!
      • Sonoport
    • Cool Hacks to get things to work:
      • Sonoport
      • Tone.js
    • Chris Lowis’ keynote on the original of computer music at Bell Labs half a century ago, really hasn’t evolved that much. Simple principles:
      • flexible building blocks
      • allow extension
      • separate orchestra from score
    • Q: What about the web makes it different?
      • challenges described by Paul Adenot
  • Modular resources
    • IRCAM’s WAVES – common.js
    • BRAID – Web Audio UI builder, uses canvas.
      • idea: share instruments via central database
      • modular with npm
    • Open Music – not talked about

Heating Gratetar

2014-09-23 21.38.28Heating grates sound great. I used to sit by them as a kid and strum metallic chords while basking in their warmth. It’s like campfire music.

At Home Depot, I found a heating grate. I played with it using my fingers, rubber, wood and metal drumsticks. They all sounded cool. I bought the heating grate.

Next I needed a box that could help shape the sound of the instrument. I bought a 1x48x5.5 plank of resonant oak wood, measured some sides, and cut with the chopsaw.

Continue reading →