Here are examples of what early Christian writers had to say on the subject of the Peter’s Roman residency:
Ignatius of Antioch
“Not as Peter and Paul did, do I command you [Romans]. They were apostles, and I am a convict” (Letter to the Romans 4:3 [A.D. 110]).
Dionysius of Corinth
“You [Pope Soter] have also, by your very admonition, brought together the planting that was made by Peter and Paul at Rome and at Corinth; for both of them alike planted in our Corinth and taught us; and both alike, teaching similarly in Italy, suffered martyrdom at the same time” (Letter to Pope Soter [A.D. 170], in Eusebius, History of the Church 2:25:8).
Irenaeus
“Matthew also issued among the Hebrews a written Gospel in their own language, while Peter and Paul were evangelizing in Rome and laying the foundation of the Church” (Against Heresies, 3, 1:1 [A.D. 189]).
“But since it would be too long to enumerate in such a volume as this the succession of all the churches, we shall confound all those who, in whatever manner, whether through self-satisfaction or vainglory, or through blindness and wicked opinion, assemble other than where it is proper, by pointing out here the succession of the bishops of the greatest and most ancient church known to all, founded and organized at Rome by the two most glorious apostles, Peter and Paul, that church which has the tradition and the faith which comes down to us after having been announced to men by the apostles. With that church [of Rome], because of its superior origin, all the churches must agree, that is, all the faithful in the whole world, and it is in her that the faithful everywhere have maintained the apostolic tradition” (ibid., 3, 3, 2).
“The blessed apostles [Peter and Paul], having founded and built up the church [of Rome], they handed over the office of the episcopate to Linus. Paul makes mention of this Linus in the letter to Timothy [2 Tim. 4:21]. To him succeeded Anacletus, and after him, in the third place from the apostles, Clement was chosen for the episcopate. He had seen the blessed apostles and was acquainted with them. It might be said that he still heard the echoes of the preaching of the apostles and had their traditions before his eyes. And not only he, for there were many still remaining who had been instructed by the apostles. In the time of Clement, no small dissension having arisen among the brethren in Corinth, the church in Rome sent a very strong letter to the Corinthians, exhorting them to peace and renewing their faith. . . . To this Clement, Evaristus succeeded . . . and now, in the twelfth place after the apostles, the lot of the episcopate [of Rome] has fallen to Eleutherius. In this order, and by the teaching of the apostles handed down in the Church, the preaching of the truth has come down to us” (ibid., 3, 3, 3).
Gaius
“It is recorded that Paul was beheaded in Rome itself, and Peter, likewise, was crucified, during the reign [of the Emperor Nero]. The account is confirmed by the names of Peter and Paul over the cemeteries there, which remain to the present time. And it is confirmed also by a stalwart man of the Church, Gaius by name, who lived in the time of Zephyrinus, bishop of Rome. This Gaius, in a written disputation with Proclus, the leader of the sect of Cataphrygians, says this of the places in which the remains of the aforementioned apostles were deposited: ‘I can point out the trophies of the apostles. For if you are willing to go to the Vatican or to the Ostian Way, you will find the trophies of those who founded this Church’” (Disputation with Proclus [A.D. 198] in Eusebius, Church History 2:25:5).
Clement of Alexandria
“The circumstances which occasioned . . . [the writing] of Mark were these: When Peter preached the Word publicly at Rome and declared the gospel by the Spirit, many who were present requested that Mark, who had been a long time his follower and who remembered his sayings, should write down what had been proclaimed” (Sketches [A.D. 200], in a fragment from Eusebius, History of the Church, 6, 14:1).
Tertullian
“But if you are near Italy, you have Rome, where authority is at hand for us too. What a happy church that is, on which the apostles poured out their whole doctrine with their blood; where Peter had a passion like that of the Lord, where Paul was crowned with the death of John [the Baptist, by being beheaded]” (Demurrer Against the Heretics 36 [A.D. 200]).
“Let us see what milk the Corinthians drained from Paul; against what standard the Galatians were measured for correction; what the Philippians, Thessalonians, and Ephesians read; what even the nearby Romans sound forth, to whom both Peter and Paul bequeathed the gospel and even sealed it with their blood” (Against Marcion 4, 5:1 [A.D. 210]).
The Little Labyrinth
“Victor . . . was the thirteenth bishop of Rome from Peter” (The Little Labyrinth [A.D. 211], in Eusebius, Church History 5:28:3).
The Poem Against the Marcionites
“In this chair in which he himself had sat, Peter in mighty Rome commanded Linus, the first elected, to sit down. After him, Cletus too accepted the flock of the fold. As his successor, Anacletus was elected by lot. Clement follows him, well-known to apostolic men. After him Evaristus ruled the flock without crime. Alexander, sixth in succession, commends the fold to Sixtus” (Poem Against the Marcionites 276–284 [A.D. 267]).
Eusebius of Caesarea
“[In the second] year of the two hundredth and fifth Olympiad [A.D. 42]: The apostle Peter, after he has established the church in Antioch, is sent to Rome, where he remains as a bishop of that city, preaching the gospel for twenty-five years” (The Chronicle [A.D. 303]).
Peter of Alexandria
“Peter, the first chosen of the apostles, having been apprehended often and thrown into prison and treated with ignominy, at last was crucified in Rome” (Penance, canon 9 [A.D. 306]).
Lactantius
“When Nero was already reigning, Peter came to Rome, where, in virtue of the performance of certain miracles which he worked . . . he converted many to righteousness and established a firm and steadfast temple to God. When this fact was reported to Nero . . . Peter he fixed to a cross, and Paul he slew” (The Deaths of the Persecutors 2:5 [A.D. 318]).
Cyril of Jerusalem
“[Simon Magus] so deceived the city of Rome that Claudius erected a statue of him. . . .While the error was extending itself, Peter and Paul arrived, a noble pair and the rulers of the Church, and they set the error aright. . . . [T]hey launched the weapon of their like-mindedness in prayer against the Magus, and struck him down to earth. It was marvelous enough, and yet no marvel at all, for Peter was there—he that carries about the keys of heaven” (Catechetical Lectures 6:14 [A.D. 350]).
Optatus
“You cannot deny that you are aware that in the city of Rome the episcopal chair was given first to Peter; the chair in which Peter sat, the same who was head—that is why he is also called Cephas [‘Rock’]—of all the apostles; the one chair in which unity is maintained by all” (The Schism of the Donatists 2:2 [A.D. 367]).
Epiphanius of Salamis
“At Rome the first apostles and bishops were Peter and Paul, then Linus, then Cletus, then Clement, the contemporary of Peter and Paul” (Medicine Chest Against All Heresies 27:6 [A.D. 375]).
Damasus I
“Likewise it is decreed: . . . [W]e have considered that it ought to be announced that although all the Catholic churches spread abroad through the world comprise one bridal chamber of Christ, nevertheless, the holy Roman Church has been placed at the forefront not by the conciliar decisions of other churches, but has received the primacy by the evangelic voice of our Lord and Savior, who says: ‘You are Peter, and upon this rock I will build my Church, and the gates of hell will not prevail against it; and I will give to you the keys of the kingdom of heaven, and whatever you shall have bound on earth will be bound in heaven, and whatever you shall have loosed on earth shall be loosed in heaven’ [Matt. 16:18–19]. The first see, therefore, is that of Peter the apostle, that of the Roman Church, which has neither stain nor blemish nor anything like it.
“In addition to this, there is also the companionship of the vessel of election, the most blessed apostle Paul, who contended and was crowned with a glorious death along with Peter in the city of Rome in the time of Caesar Nero. . . . They equally consecrated the above-mentioned holy Roman Church to Christ the Lord; and by their own presence and by their venerable triumph they set it at the forefront over the others of all the cities of the whole world” (Decree of Damasus 3 [A.D. 382]).
Jerome
“Simon Peter, the son of John, from the village of Bethsaida in the province of Galilee, brother of Andrew the apostle, and himself chief of the apostles, after having been bishop of the church of Antioch and having preached to the Dispersion . . . pushed on to Rome in the second year of Claudius to overthrow Simon Magus, and held the sacerdotal chair there for twenty-five years until the last, that is the fourteenth, year of Nero. At his hands he received the crown of martyrdom being nailed to the cross with his head towards the ground and his feet raised on high, asserting that he was unworthy to be crucified in the same manner as his Lord” (Lives of Illustrious Men 1 [A.D. 396]).
Augustine
“If all men throughout the world were such as you most vainly accuse them of having been, what has the chair of the Roman church done to you, in which Peter sat, and in which Anastasius sits today?” (Against the Letters of Petilani 2:118 [A.D. 402]).