Callings are not "up or down" but "forward" (if we act on them well)

 "we do not “step down” when we are released, and we do not “step up” when we are called. There is no “up or down” in the service of the Lord. There is only “forward or backward,” and that difference depends on how we accept and act upon our releases and our callings."

- "The Keys and Authority of the Priesthood," By Elder Dallin H. Oaks, Of the Quorum of the Twelve Apostles, General Conference, April 2014

https://www.churchofjesuschrist.org/study/general-conference/2014/04/the-keys-and-authority-of-the-priesthood?lang=eng