I dont think of it as annoying but as strange. Why couldnt any portal take you anywhere does make the whole portal operation seem inefficient.
But that said, with just 2 of us running these missions we made it to the next portal in almost no time so i dont think locking the missions to a single portal is necessary.
If its ever needed, an elevator from the portal room to outside would probably streamline the process enough. But even all that to save just a few seconds seems like overkill.