I don't think they had different times or anything, but they were held way apart from each other by the summon spirits. The seed of the tree you saw was giving them a little bit of life, so they had to switch off which one was getting it, and the other world would decline while that happened till another Chosen could complete their journey and switch it back.