I don't blame you for thinking Swangard would be ideal. It certainly appears that way, sitting between two stations (about a five-minute walk), nice view of the mountains and trees, and a great location far enough away from BC Place Toilet, but just across the street from Vancouver in the city of Burnaby. However, it has one down side, which is the City of Burnaby. They won't let anyone touch that track, and they're not very footie-friendly. When there used to be World Cup Qualifiers there, Canada couldn't practise the there because the away team (rightfully so) would demand the opportunity to have a run out on the pitch, and the knuckle-scrapers at Burnaby City Hall would say, "That's enough. You can have a match and someone can run on it for an hour or so. Otherwise, get lost."
The PNE, where Empire Field was, was another great location. That temporary stadium was almost as nice as the original Empire Stadium, and with a much better pitch. The turf at Empire Stadium was 500 area rugs from Ikea of various shades of green, stapled together. I digress. You're right, that was another excellent location, other than public transport. Since tearing down the stadium, the PNE (similar to Toronto's CNE, I presume) has done a bunch of very careful landscaping and other crap to make the area look as if someone's done some very careful landscaping. You know, an obviously placed shrubbery here and there, and some fakey-looking bridge that doesn't really go over anything except the level ground underneath it. Nice job, if you like that sort of thing. I think they left one fake turf field, with no room for any stands.