Archive

Archive for the ‘music’ Category

Netflix Prize: Joining the Rat Race

November 19th, 2008

I decided a couple of nights ago to see how well I would do in the Netflix Prize. This is a competition from the company Netflix, an online movie rental site, that gives two gigs worth of user ranking data to see if anyone can improve their own ranking algorithm, Cinematech, by at least 10%. Many have tried, but few have come close.

Recommendations are a fairly difficult problem. At Grooveshark, we have our own recommendation system using various statistical techniques that have been fine-tuned over the years. They are not perfect, but they do come close to what Pandora, and Last.fm have to offer in certain instances. I’m sure the techniques Grooveshark uses are no way near as sophisticated as Google or Amazon, but we try our best. Early Google has shown that simple algorithms using the right data can be more successful than advanced statistical tools. But even with Google, their algorithms and tools have grown more sophisticated over time. In my opinion, simple tools using the correct insights can be very powerful as proven by a psychologist who has jumped very high in the leaderboard (Just a guy in a garage).

Overall, this project gives a lot of goodwill to Netflix for being so open and providing a great competition for researchers and joe-schmoes alike. I really just want to apply some of the new techniques I have learned in an environment other than music (not surprising when you spend 60 hours a week thinking about music). Here’s some of the books I have read or currently reading:

On Intelligence by John Hawkins: hierarchical Markov Models FTW

Collective Intelligence by Toby Segaran: leveraging simple statistical tools to add intelligence to web applications

Predictably Irrational by Dan Ariely: more psychology than statistics/intelligence

Pattern Recognition by Theodoridis and Koutroumbas: never finished – a little over my head for right now

Probabilistic Reasoning in Intelligent Systems by Judea Pearl: not finished, but find the language more understandable than “Pattern Recognition”

Along with these books, I have kept up a large collection of recommendation and music information retrieval papers. I have read a lot of them, but most of them are on my to-read list. If you would like, check out my document subversion repository at: svn://cmunezero.com/docs.

Also, here’s a pretty good presentation by somebody at Netflix talking about the challenges and issues they face. Now for some muzik:

Grooveshark, Netflix, music, programming , , , , , , , , , , , , , , , , , , , , ,

Grooveshark Lite: Listen to Any Song Anywhere

April 15th, 2008

Sigh….

After months of hardwork, its finally here: Grooveshark Lite. Everyone at Grooveshark has been busting their tails off trying to bring the best music, listening experience the internet has to offer. Now, you don’t have to download Sharkbyte to stream music, though it is still needed to download and purchase songs. This flash application allows users to easily find any song in our system. The best way to find out about Grooveshark Lite is to see it for yourself. Go to http://listen.grooveshark.com to see it in action.

Grooveshark, music

Working Hard for the Man

April 5th, 2008

It’s 1.40AM on a Friday night and I’m still at work. Something is very wrong with that picture.

Over at Grooveshark, LOTS of changes are being that is going to make a pretty good music site an even better one. Better organized data, more online songs and a better user experience are all on the horizon. For all those impatient folks out there, which I am one of, just hold on and this major update will knock you out of your boots, or socks… or feet so make sure you wear some kind of footwear!

Grooveshark, life, music , , , , ,

I Want MY Music: The History and Future of Music, Downloads and Mixtapes

March 26th, 2008

I want MY music. I want it fast, organized, high-quality and now. This is not too much to ask, and I, a consumer, have been begging for it for years.

Before BitTorrent, Napster or CDs, there was the mixtape. I would spend hours listening to the radio and when a song I liked came on, I recorded it. It was MY music. I choose to keep the notes, the voice, the rhythms with me for all time. I would mix and match songs according to my mood and the occasion. I would swap tapes with friends with similar tastes. And if they were lucky, they might have received one as a gift. Of course being a kid, I didn’t have a lot of money. But nothing says I care more than a well crafted mixtape (and no, I’m not ripping off Nick Hornby).

Then came the CD with its shiny new cover and it’s obvious similarity to the venerable vinyl record. But this was the 90’s. A new time, a new era, full of ones and zeros. Anything digital was all the rage. So I paid $20 a pop for a piece of plastic that did sound better than a tape, but was still inferior to a vinyl record. Granted, the CD came with a small booklet full of useful information: more pictures, sometimes lyrics, and even the name of the sound engineer from that song I never listened to. With a mixture of mass marketing and an era of economic prosperity, YOU, the labels, reaped record profits and I was left with piles of CDs containing just a few songs of enjoyable music.

In the end, I was left unfulfilled. I returned to the trusty mixtape to create the album I really wanted. Sure it was lesser quality and I couldn’t magically skip to the next song, but I at least had the songs I wanted in the order I wanted. It was my music my way. At the end of the 90’s, the party was over. The floodgates were open and a sea of unlimited free music was unleashed upon us. But this wasn’t a revolution. People didn’t storm the castle or burn down the walls. It’s the same evolutionary process that brought about the transistors, computers and the internet.

As the technology advanced with cheaper disk space and faster web connections, I wanted more. First there was Usenet, full of silly chatter and annoying trolls, but I could download free music. Then there was IRC with better file downloading but added viruses. Each successive iteration provided faster downloads and more content. The tide of progress couldn’t be stopped so technology finally reached a tipping point where the will, getting the music I want, found a way, Napster.

In truth, Napster changed everything for internet users. Before, advanced internet users had to download from large, clunky servers that might not have the data you wanted, or use a command-line interface to connect directly to files you wanted. What Napster did correctly was provide an adequate interface to content that was searchable and downloadable. What Napster neglected to do was filter out all the junk that people tossed in the system. Despite their drawbacks, what made Napster and other P2P networks even more worthwhile was recordable CDs. Now you had the best of both worlds: the crisp, clean digital sound of a CD along with the malleable, personal nature of a mixtape.

During this era of free music, a connection was broken. I was no longer a customer. I became a common theif and had to be dealt with immediately. You tried suing me and that did not work. You tried suing my little siblings and grandmother and you lost my respect. You did not ask me what I wanted or what it would take to change my ways. You could have created a new and better system with higher quality files and organized information. You could have created a faster system with special content like videos or live concerts. You could have done a lot of things, but you chose to ignore me. That was your decision, but now I have made mine: I WILL have my music.

There is Bittorrent which lets me download massive amount of music with a click of a button. There are decentralized P2P networks which let me find the exact song I want without being sued. There are social music sites where random strangers and I can share our individual tastes with each other and the world. There are programs out there that let me organize, tag and correct the gigabytes of music I have on my hard drive, and portable music players that can carry my entire library and fit in my pocket. I do this because I still love music and listen to it everyday. Now I download music and create playlists instead of buying CDs and burning mixed CDs. I do this because you drove me away and made me an outcast. Now a new era is upon us and you have to decide what your next step will be.

It’s starting out with a trickle: Radiohead, Trent Reznor and Barenaked Ladies. But that’s only the beginning of your troubles. When I finally build a system that let’s me listen to my music when, where and how I want it, I will need you less. When I can easily share and broadcast my music with friends or the world, you will see my influence. When I can discover the music I like or find music I never knew I would like, I will be empowered. And when I can finally pay the people directly who create, perform and produce the music I love, you will be nothing.

There are some who claim that they can do these things already. That they can provide a solution to my problems. I will be the judge of that. In the meantime, I refuse to wait for a solution because I am the solution. I choose to build the very system I pleaded for you to build. I have built Wikipedia, Flickr, Digg, Last.fm and Grooveshark. These are just the beginning and more is coming ahead. You can be part of this future if you really want. Just remember, I will have my music, but will YOU be part of the system?

life, music , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Music Information Retrieval

March 18th, 2008

Over the weekend, I really got into music information retrieval (MIR). Its basically grabbing meta-information of an audio file by analyzing its waveform. This type of information is really valuable, especially for a music company (ie: Grooveshark). If I ever have time, this would be a really fun side project. A really good source of information about this topic is this bibiography page (too bad it hasn’t been updated since August, 2007). A list of up and running MIR systems can be found here.

What makes MIR systems so important is that for music sites, they can generate a lot of useful data without anyone having to enter it by hand. For iTunes, this is not a problem because labels give them all the information they need, but for sites where song files can come from anywhere and anyone, there’s no way you can handle the variability in data quality and availability. By having a system that could automatically fetch the required info, within certain bounds of error, you create a vast collection of information that you can use to generate recommendations, provide more accurate searches, and create better categorization of all that music.

The problem with MIR systems is that they require large amounts of storage space and processing power. The cost of both storage and processing are dropping everyday which is great for the future of MIR systems. Processing power is the largest inhibiting factor, especially when you try to analyze millions of songs. The only companies that could probably do a project like this on a large scale would be Google, Amazon and their ilk. Currently, I’m very hopeful that a startup with the right mix of programmers, hardware, and music can compete with the big boys ;)

Grooveshark, music, programming , , , , , , , , ,

Customer Service

March 16th, 2008

The key to web2.0 is customer service. Look at all of the major websites created after the bubble and the one thing they have in common is their community. Flickr is the best example: They actually started out as an online game but evolved into an image service from constant communication with their user base.

As a developer, it’s especially important that we constantly use the site and interact with our users. Not only do we get a feel for what the users are dealing with, it also gives us a firsthand account on what the users are thinking. I recently had an eye opening encounter with a user that really gave me a new perspective on Grooveshark.

The user just downloaded Sharkbyte, the Grooveshark helper application, and when he went on the site, he didn’t immediately see all of his songs. Of course the user would think something is broken. What they don’t realize is that we are constantly getting new files added to the system so it takes some time processing all of that data. After sending a couple of messages back and forth between the user and me, the user became frustrated and actually said, “Now Im lost maybe I should just go back to youtube…”

Now that is a scary moment when someone would rather listen to music on a low quality video site instead of a site specifically made for music. After explaining the situation and giving him a $5 dollar gift code (tactic I got from Jay). Now that user has been listening, rating and buying songs. It’s a really big turnaround from going back to Youtube to listen to music, to using Grooveshark and all of its features.

Grooveshark, music , , , , , ,

Music is Life

March 13th, 2008

For most people, music is just something they listen to. Sure they enjoy it, and some even develop an attachment to certain songs and artists. In my case, music and life are eternally intertwined. Music is not a series of notes, rhythms and melodies. When I listen to a song, I hear memories and emotions conjured from the depths of the artist’s imagination.

Music has always been a big part of my life. My mom made me take piano lessons when I was a kid. She was tired of me doing nothing after-school except break windows and cause trouble. In the long run, she probably saved money. Learning how to play an instrument really helped me appreciate all types of music. I was lucky to have a teacher who didn’t just focus on the technical aspects of music but taught me the history of most musical genres and how to appreciate what the artist is trying to do. Most of all, she taught me how to listen.

Listening to music occurs on a multitude of levels. There is the immediate sounds you hear, which is how most people listen to music. Then there is concentrating on each instrument and they notes they play. If you know how to read music, this is really easy, especially if you have absolute pitch. Next is following the tempo, melodies and transitions of a song. This is how an artist tells a story (songs with lyrics are another story).

The iPod absolutely changed the way I listen to music. One of my favorite things between classes is listening to my library on shuffle and noticing how the music I’m listening to follow what’s going on in the outside world. Of course this is all by chance, but its those magical moments when Radiohead’s “Creep” starts playing as you walk out of the Engineering building, followed by, “Milkshake” by Kelis just as you are passing the food court and ice cream shop and it ends with “Momma, I’m Coming Home” by Black Sabbath as I dump my backpack on the floor of my dorm room (Disclaimer: I did skip some songs along the way). At this point, music is not just some accessory you enjoy, it is a colorful fabric in the tapestry of life.

Grooveshark, music , ,

Sharkbyte Team Hard at Work

March 5th, 2008

At Grooveshark, in order to play music you need our helper application we call Sharkbyte that handles all of the p2p functionality and music playback. Recently there was a post by a friend of mine Jay, also an employee of Grooveshark, describing some faults in how Sharkbyte handles new updates. His main caveat is that an installed software shouldn’t fail silently and that updates should be installed automatically and actively prompt users to restart.

Today I woke up and decided to play some music while I got ready. Well when I tried playing a song, a box pops up telling me that new updates were available for Sharkbyte. It seems that the Sharkbyte team did a good job fixing those usability problems very quickly. Good job Team!

Grooveshark, music , , , ,