Adamas is the main town of the island and staying in Adamas is the most comfortable choice you can make. The proximity to the port, the presence of many taverns and shops will allow you to spend a pleasant holiday. Obviously, the comforts do not really match with peace and landscape beauty.
Staying in Pollonia could be a bit uncomfortable for who wants to visit the whole island, as it is a village located in the north, and thus not the most convenient starting point for exploring the entire island. But it could be a good starting point to visit the north shore and Kimolos. If you have the opportunity to spend several days on Milos you could also decide to divide your stay and spend also a few days in Pollonia.
Plaka is the most charming village of Milos island thanks to its little streets among the white cycladic style buildings and its breathtaking view on the Milos gulf.