Chanel’s Blog

I decided a couple of nights ago to see how well I would do in the Netflix Prize. This is a competition from the company Netflix, an online movie rental site, that gives two gigs worth of user ranking data to see if anyone can improve their own ranking algorithm, Cinematech, by at least 10%. Many have tried, but few have come close.

Recommendations are a fairly difficult problem. At Grooveshark, we have our own recommendation system using various statistical techniques that have been fine-tuned over the years. They are not perfect, but they do come close to what Pandora, and Last.fm have to offer in certain instances. I’m sure the techniques Grooveshark uses are no way near as sophisticated as Google or Amazon, but we try our best. Early Google has shown that simple algorithms using the right data can be more successful than advanced statistical tools. But even with Google, their algorithms and tools have grown more sophisticated over time. In my opinion, simple tools using the correct insights can be very powerful as proven by a psychologist who has jumped very high in the leaderboard (Just a guy in a garage).

Overall, this project gives a lot of goodwill to Netflix for being so open and providing a great competition for researchers and joe-schmoes alike. I really just want to apply some of the new techniques I have learned in an environment other than music (not surprising when you spend 60 hours a week thinking about music). Here’s some of the books I have read or currently reading:

On Intelligence by John Hawkins: hierarchical Markov Models FTW

Collective Intelligence by Toby Segaran: leveraging simple statistical tools to add intelligence to web applications

Predictably Irrational by Dan Ariely: more psychology than statistics/intelligence

Pattern Recognition by Theodoridis and Koutroumbas: never finished – a little over my head for right now

Probabilistic Reasoning in Intelligent Systems by Judea Pearl: not finished, but find the language more understandable than “Pattern Recognition”

Along with these books, I have kept up a large collection of recommendation and music information retrieval papers. I have read a lot of them, but most of them are on my to-read list. If you would like, check out my document subversion repository at: svn://cmunezero.com/docs.

Also, here’s a pretty good presentation by somebody at Netflix talking about the challenges and issues they face. Now for some muzik:

Grooveshark, Netflix, music, programming books, Grooveshark, grooveshark music, intelligence, mir, movie recommondations, movies, music, music information retrieval, music recommendations, Netflix, netflix challenge, netflix leaderboard, pattern, pattern recognition, probability, rec, recommendations, recs, statistics, stats, subversion

It has been a really long time since my last post. Lots of stuff have been happening at Grooveshark. We’ve created a widget site for sharing music across various social networks using Clearspring. Here’s an example below (you can also find it here):

I started part-time work with Conceptual Arts a local Gainesville web shop. The people are really cool and smart, so I’m really excited to work on lots of cool projects.

I’m also going to start tracking personal projects I work on in my Projects page. Hopefully by posting it online, it will force me to stop abandoning projects and create complete prototypes of each.

Grooveshark, web c-arts, conceptual arts, Grooveshark, grooveshark widgets, web, web design, widgets

Sigh….

After months of hardwork, its finally here: Grooveshark Lite. Everyone at Grooveshark has been busting their tails off trying to bring the best music, listening experience the internet has to offer. Now, you don’t have to download Sharkbyte to stream music, though it is still needed to download and purchase songs. This flash application allows users to easily find any song in our system. The best way to find out about Grooveshark Lite is to see it for yourself. Go to http://listen.grooveshark.com to see it in action.

Grooveshark, music

It’s 1.40AM on a Friday night and I’m still at work. Something is very wrong with that picture.

Over at Grooveshark, LOTS of changes are being that is going to make a pretty good music site an even better one. Better organized data, more online songs and a better user experience are all on the horizon. For all those impatient folks out there, which I am one of, just hold on and this major update will knock you out of your boots, or socks… or feet so make sure you wear some kind of footwear!

Grooveshark, life, music download, download music, download songs, Grooveshark, music, update

Over the weekend, I really got into music information retrieval (MIR). Its basically grabbing meta-information of an audio file by analyzing its waveform. This type of information is really valuable, especially for a music company (ie: Grooveshark). If I ever have time, this would be a really fun side project. A really good source of information about this topic is this bibiography page (too bad it hasn’t been updated since August, 2007). A list of up and running MIR systems can be found here.

What makes MIR systems so important is that for music sites, they can generate a lot of useful data without anyone having to enter it by hand. For iTunes, this is not a problem because labels give them all the information they need, but for sites where song files can come from anywhere and anyone, there’s no way you can handle the variability in data quality and availability. By having a system that could automatically fetch the required info, within certain bounds of error, you create a vast collection of information that you can use to generate recommendations, provide more accurate searches, and create better categorization of all that music.

The problem with MIR systems is that they require large amounts of storage space and processing power. The cost of both storage and processing are dropping everyday which is great for the future of MIR systems. Processing power is the largest inhibiting factor, especially when you try to analyze millions of songs. The only companies that could probably do a project like this on a large scale would be Google, Amazon and their ilk. Currently, I’m very hopeful that a startup with the right mix of programmers, hardware, and music can compete with the big boys

Grooveshark, music, programming Grooveshark, hardware, itunes, itunes music, mir, music, music information, music information retrieval, programming, songs

The key to web2.0 is customer service. Look at all of the major websites created after the bubble and the one thing they have in common is their community. Flickr is the best example: They actually started out as an online game but evolved into an image service from constant communication with their user base.

As a developer, it’s especially important that we constantly use the site and interact with our users. Not only do we get a feel for what the users are dealing with, it also gives us a firsthand account on what the users are thinking. I recently had an eye opening encounter with a user that really gave me a new perspective on Grooveshark.

The user just downloaded Sharkbyte, the Grooveshark helper application, and when he went on the site, he didn’t immediately see all of his songs. Of course the user would think something is broken. What they don’t realize is that we are constantly getting new files added to the system so it takes some time processing all of that data. After sending a couple of messages back and forth between the user and me, the user became frustrated and actually said, “Now Im lost maybe I should just go back to youtube…”

Now that is a scary moment when someone would rather listen to music on a low quality video site instead of a site specifically made for music. After explaining the situation and giving him a $5 dollar gift code (tactic I got from Jay). Now that user has been listening, rating and buying songs. It’s a really big turnaround from going back to Youtube to listen to music, to using Grooveshark and all of its features.

Grooveshark, music customer, customer service, Grooveshark, help, music, service, users

For most people, music is just something they listen to. Sure they enjoy it, and some even develop an attachment to certain songs and artists. In my case, music and life are eternally intertwined. Music is not a series of notes, rhythms and melodies. When I listen to a song, I hear memories and emotions conjured from the depths of the artist’s imagination.

Music has always been a big part of my life. My mom made me take piano lessons when I was a kid. She was tired of me doing nothing after-school except break windows and cause trouble. In the long run, she probably saved money. Learning how to play an instrument really helped me appreciate all types of music. I was lucky to have a teacher who didn’t just focus on the technical aspects of music but taught me the history of most musical genres and how to appreciate what the artist is trying to do. Most of all, she taught me how to listen.

Listening to music occurs on a multitude of levels. There is the immediate sounds you hear, which is how most people listen to music. Then there is concentrating on each instrument and they notes they play. If you know how to read music, this is really easy, especially if you have absolute pitch. Next is following the tempo, melodies and transitions of a song. This is how an artist tells a story (songs with lyrics are another story).

The iPod absolutely changed the way I listen to music. One of my favorite things between classes is listening to my library on shuffle and noticing how the music I’m listening to follow what’s going on in the outside world. Of course this is all by chance, but its those magical moments when Radiohead’s “Creep” starts playing as you walk out of the Engineering building, followed by, “Milkshake” by Kelis just as you are passing the food court and ice cream shop and it ends with “Momma, I’m Coming Home” by Black Sabbath as I dump my backpack on the floor of my dorm room (Disclaimer: I did skip some songs along the way). At this point, music is not just some accessory you enjoy, it is a colorful fabric in the tapestry of life.

Grooveshark, music Grooveshark, life, music

At Grooveshark, in order to play music you need our helper application we call Sharkbyte that handles all of the p2p functionality and music playback. Recently there was a post by a friend of mine Jay, also an employee of Grooveshark, describing some faults in how Sharkbyte handles new updates. His main caveat is that an installed software shouldn’t fail silently and that updates should be installed automatically and actively prompt users to restart.

Today I woke up and decided to play some music while I got ready. Well when I tried playing a song, a box pops up telling me that new updates were available for Sharkbyte. It seems that the Sharkbyte team did a good job fixing those usability problems very quickly. Good job Team!

Grooveshark, music Grooveshark, helping, music, programming, usability

On a daily basis for the last year, I use jquery at work (Grooveshark) to handle different DOM and animation methods. I started out using jquery because I was new to the browser world and Javascript in particular. Jquery made it really easy to come right in and start creating usable modules.

As I got more comfortable with javascript, I started moving away from jquery and building my own methods or extending jquery. One problem with browsers in general is accessing the DOM and any javascript heavy site will always go over this bump. At Grooveshark, because everything is so heavily based on accessing/modifying song/artist/album information, we already have a unique identifier from the MySQL database. By using a contextual name based system, we can get really fast DOM lookups for very cheap:

HTML:

<table class="songs">
	<tr id="song_123"><td>A Song</td></tr>
	<tr id="song_456"><td>My Song</td></tr>
		...	MORE SONGS ...
	<tr id="song_789><td>Your Song</td></tr>
</table>

Javascript:

function song_click() {
	$("table.songs tr").click(function() {
		var domid = $(this).attr("id");
		var songid = domid.split("_")[1];
		// possibilities endless...
	}
}

This is not exactly what we do (currently using inline event handler – I know, not very web2.0-ish of me), but this is the basic idea. The html elements themselves contain a reference to the data they represent so whenever a user interacts with that element, it’s very easy to identify what it is.

Grooveshark, jquery, mysql, programming dom, javascript, jquery, mysql, programming