As you're going through the site, Rate each photo you look at with the "Rate Photo" boxes. One box being the lowest rating, and five boxes being the highest rating. Every time you rate a photo, that adds a "vote" to the offices total "Votes". You don't cast a vote for or against the office. You rate the photos of the office. Those ratings add up to create an overall score for the office, which determines where it ranks among all the offices.
For example, an office could have 100 votes, but if all the photos of that office were rated low (one box), that office would score fairly bad and rank low among all offices. Whereas an office with 50 votes, that has photos which were all rated highly (five boxes) would likely score higher and rank better against the other offices.
We do factor in how many "votes" an office has, as well as how each photo in that office's collection of photos were rated. It's a little complicated, but the goal was to prevent (or at least make it difficult) for someone to artificially move their office to the top spot.