We look at footy in grey terms - a feeling we have about players, a perception, an image. The Brownlow isn't grey, it's black and white (pardon the pun). Each game is unique and self-contained; who was the best player that game, who was second best, and who was third best. Using this simple decision, a winner is announced at the end of the season.
The mere fact that only *three* players out of the 44 who can take the field can receive votes, makes winning Charlie dependent on at least a little bit of luck.
The one gripe at the moment I have is the fairest part. Hundreds of "unfair" acts/tactics are used each season which are within the rules. To me, that means you didn't play fairly - the notion that if you aren't suspended, you were fair, is ridiculous. And, the fact you *are* suspended means you weren't fair? Crap. With the minor things that happen these days, you can miss out on the award because of a total accident that's considered careless or reckless. Maybe the MRP needs to apply points for each case for "fairness" too.
I don't think there's much that can be done to the Brownlow Medal (apart from the MRP issue), but I think that's part of it's charm - it is what it is. Get in the top three players on the ground in the umpires eyes, and you'll do well. Personally, a 5 4 3 2 1 system creates a bit more accuracy and I'm unsure why it was changed back (can someone tell me?) but I don't know if I want it changed. The count is log enough as it is

Australian Rules Football is an art form - the game has beauty and creativity in my opinion higher than any other in the world.
So judging the "best" is never going to be accurate.