“Therefore, brethren, pick out from among you seven men of good repute, full of the Spirit and of wisdom, whom we may appoint to this duty. … These they set before the apostles, and they prayed and laid their hands upon them.”
Short answer
In the New Testament, office is handed on by the laying on of hands, and those who receive it are told to appoint others. Clement of Rome says the apostles arranged for approved men to succeed the ones they had appointed. Irenaeus and Tertullian point to named lines of bishops as the proof that a church's teaching is the apostles' teaching. The Church holds the two together: the succession is handed on by the sacrament of Holy Orders, and it exists to keep the apostles' teaching.
Built from the project document Apostolic Succession from the New Testament to the Year 300 AD. Every quotation was then checked against the linked source, and later witnesses were added to bring the timeline to the present.
Where this teaching appears on the Offshoots timelines: 1559, Calvin on leaving the communion of Rome; 10 September 1784, Wesley sets up a separate Methodist church in America.
18 quotations in date order
“And when they had appointed elders for them in every church, with prayer and fasting, they committed them to the Lord in whom they believed.”
“Do not neglect the gift you have, which was given you by prophetic utterance when the elders laid their hands upon you.”
“This is why I left you in Crete, that you might amend what was defective, and appoint elders in every town as I directed you”
“Hence I remind you to rekindle the gift of God that is within you through the laying on of my hands”
“and what you have heard from me before many witnesses entrust to faithful men who will be able to teach others also.”
“…not laying again a foundation of repentance from dead works and of faith toward God, with instruction about ablutions, the laying on of hands, the resurrection of the dead, and eternal judgment.”
“Christ therefore was sent forth by God, and the apostles by Christ. … And thus preaching through countries and cities, they appointed the first fruits [of their labours], having first proved them by the Spirit, to be bishops and deacons of those who should afterwards believe.”
“Our apostles also knew, through our Lord Jesus Christ, that there would be strife on account of the office of the episcopate. For this reason, therefore, inasmuch as they had obtained a perfect fore-knowledge of this, they appointed those [ministers] already mentioned, and afterwards gave instructions, that when these should fall asleep, other approved men should succeed them in their ministry.”
“…we are in a position to reckon up those who were by the apostles instituted bishops in the Churches, and [to demonstrate] the succession of these men to our own times … The blessed apostles, then, having founded and built up the Church, committed into the hands of Linus the office of the episcopate. … To him succeeded Anacletus; and after him, in the third place from the apostles, Clement was allotted the bishopric. … In this order, and by this succession, the ecclesiastical tradition from the apostles, and the preaching of the truth, have come down to us.”
“But Polycarp also was not only instructed by apostles, and conversed with many who had seen Christ, but was also, by apostles in Asia, appointed bishop of the Church in Smyrna, whom I also saw in my early youth”
“Let them produce the original records of their churches; let them unfold the roll of their bishops, running down in due succession from the beginning in such a manner that [that first bishop of theirs] bishop shall be able to show for his ordainer and predecessor some one of the apostles or of apostolic men … For this is the manner in which the apostolic churches transmit their registers: as the church of Smyrna, which records that Polycarp was placed therein by John; as also the church of Rome, which makes Clement to have been ordained in like manner by Peter.”
“…Novatian is not in the Church; nor can he be reckoned as a bishop, who, succeeding to no one, and despising the evangelical and apostolic tradition, sprang from himself. For he who has not been ordained in the Church can neither have nor hold to the Church in any way.”
“The succession of priests keeps me, beginning from the very seat of the Apostle Peter, to whom the Lord, after His resurrection, gave it in charge to feed His sheep, down to the present episcopate.”
“…bishops, who have succeeded to the place of the apostles, … are placed, as the same apostle says, by the Holy Ghost, to rule the Church of God; that they are superior to priests; administer the sacrament of Confirmation; ordain the ministers of the Church”
“That divine mission, entrusted by Christ to the apostles, will last until the end of the world, since the Gospel they are to teach is for all time the source of all life for the Church. And for this reason the apostles, appointed as rulers in this society, took care to appoint successors. … Therefore, the Sacred Council teaches that bishops by divine institution have succeeded to the place of the apostles”
“The Church is apostolic because she is founded on the apostles, in three ways: - she was and remains built on "the foundation of the Apostles," the witnesses chosen and sent on mission by Christ himself; - with the help of the Spirit dwelling in her, the Church keeps and hands on the teaching, the "good deposit," the salutary words she has heard from the apostles; - she continues to be taught, sanctified, and guided by the apostles until Christ's return, through their successors in pastoral office: the college of bishops”
“By the power of the same Holy Spirit they entrusted this power to their successors. This "apostolic succession" structures the whole liturgical life of the Church and is itself sacramental, handed on by the sacrament of Holy Orders.”