You have to create your own pool of agents and do not depend on anyone. We have an Azure VM with a pool of 4 agents (It can be your server on premise). You might want to build other solution (APIs)a the same time. You need to have a few agent in the pool. If you have one agent and build is long it means all other builds will wait for the agent to be free. For CI/CD solution
With a shared agent you not limited only with time, but I believe with the resources as well (CPU and memory). On top of that you might wait for a free agent. If you get your own build server it should be fairy fast. In our kenitco pipeline most of time is taken by javascript bundles, not the Kentico.