So get this….I had an online chat with a Netflix rep last night regarding an issue that has been plaguing my queue for some time. It shows my predicted rating for almost every movie out there (I subscribe to streaming only) will be 5 stars, even for dogs like “One for the Money” and “Aeon Flux”. Yesterday, I logged in to my account on my desktop and everything looked fine. I was checking to see what the average rating was for “Hemlock Grove”, which I started watching a few days ago. I only watched one episode and was pretty unimpressed and wanted to see if I should continue. My predicted rating was coming up as 2.5, which is not a ringing endorsement. But as I went down the list, everything started becoming a five-star prediction. And when I returned to “Hemlock Grove”, same deal. I really though the problem had been temporary, but it appeared not to be the case.
At first I thought the person I was chatting with was high. When I told him of the issue, he would write stuff like, “Ha! How about that?” That told me he had never seen anything like it before. I waited patiently for him to let me know what was going on, not sure if he was really researching it or talking with his buddy in the chair next to him. He finally came back and told me that I “confused the rating system” by rating too many movies. I checked today and I have about 1,900 ratings. OK, so that’s a lot, but confuse the model? That hardly seemed likely. More data typically yields better results.
Ironically, that night I was reading from my book “Naked Statistics: Stripping the Dread from the Data” and he specifically mentioned the Netflix model when talking about correlation. A few years back, Netflix had a contest to pay one MILLION dollars to any individual or team that could improve the accuracy of the predicted ratings by 10% (or something like that). An international team of data scientists ended up winning. You can even read the winning paper here. I doubt very much that these guys would win for a model that breaks down with less than 2,000 inputs.
When I asked the customer service representative if this could be fixed, he wrote back something incomprehensible (a run-on sentence) that sounded as if to say that he didn’t think it was an important feature to be able to rate movies. Uh…..huh? But doesn’t that drives the predictions? And I am actually very interested in the predictions. Over the years, they have steered me to movies I would not ordinarily watch, and I would say that they have been right about whether or not I would like the movie more often than not.
We tried clearing the cookies, but that did not work. After awhile, I got tired of the online chat with this guy because either he was a very slow typist or preoccupied with something else. I will probably try calling someone else another time. Honestly, if it’s not getting fixed, there’s a good chance I might drop Netflix and give Amazon Prime a try. I really think the predicted rating is one of the best features on Netflix streaming, because the quality of the movies available certainly aren’t that great.
Broke the model. Ha!