ASCX transformations can contain (theoretically) any number of controls (webparts), however performance may take a hit, especially with nested repeaters.
You usually only nest web parts though if you need repeat content, and apply some control/webpart to each item (like add a "Buy now" web part to a repeater of purchasable items).
In terms of normal pages though, you usually have your Page Layout (HTML), and you add Webpart Zones to the Layout, then add web parts to the zones. Knowing a web part can have a content-before and after, you have to gauge the usage of zones and web parts. You usually put zones as Content pockets, like the Footer you put a zone, then add various footer webparts to the zone.
PagePlaceholders you usually only use 1, and that's where sub content will be put for templates that inherit the parent.