It really depends, as Al James posted before me, how long *you* think you'd want to spend there and how you plan on getting about. You can rent bikes, but the entire Angkor complex -- not just Angkor Wat -- is HUGE. You might prefer to work with a local travel agent and hire a car, driver & guide. It's a great value for the price, and you can get around the tourist scrum by taking an alternate route.
I visited in early 2008, and in retrospect, I would have liked to spend more time there than 3 days, but that's just me. We did see all of the key sites (Angkor Tom, the Bayon, Pre Rup, Ta Keo, etc) but I would have loved to have had more time to venture out further afield to some less visited archeological sites.
One more thing if you go, visit the new museum in Siem Reap. It's a great primer of Khmer history and puts many of the sites in historical perspective. Wish I'd done that first, not at the end of my visit.
