Uber averages the last 500 ratings. If you've got a couple of low ratings in that 500, it might take a long time before they drop off that list of 500. Don't sweat it, 4.66 tells drivers that you've taken lots of rides and it's really not a bad number.