There are some (not many) non-gospel sources which refer to Jesus, and which Christians say prove the historical existence of Jesus. These are the writings of:
Flavius Josephus - c90CE
Suetonius - c120CE
Tactitus - 110CE
Pliny - c110CE
Thallus - cited in c300CE
Talmud - 200-500CE
'Acts of Pilate'
The Josephus passage is among the most celebrated as proving that Jesus existed:
"Now, there was about this time, Jesus, a wise man, if it be lawful to call him a man, for he was a doer of wonderful works; a teacher of such men as received the truth with pleasure. He drew over to him both many of the Jews and many of the Gentiles. He was the Christ; and when Pilate, at the suggestion of the principal men amongst us, had condemned him to the cross, those that loved him at the first did not forsake him; for he appeared to them alive again the third day, as the divine prophets had foretold these and ten thousand other wonderful things concerning him; and the tribe of Christians, so named from him, are not extinct at this day." (Antiquities XVIII 63f)
At first glance, this appears to be a very good source for the historical existence of Jesus. However, Josephus was a Jew, and remained a Jew. For him to say that Jesus was 'the Christ' and was resurrected from the dead, would have had him a Christian, not a Jew, and he would have been banned from the synagogues, as were all the others who said Jesus was the messiah. Futhermore, up until the 4th Century there are no mentions of Josephus having written about Jesus in this way. None of the Christian Church Fathers mentioned Josephus as having written about Jesus in this way, if he had done so, Justin Martyr and Origen among others would have been glad to use it as ammunition in their disputes with the Jews. They did not, however, no mention of it at all. Origen actually said that Josephus did not acknowledge Jesus. Most scholars do not believe Josephus wrote this passage, but that it is a later addition by Christian scribes - Bishop Warburton denounced it as "a rank forgery and a very stupid one, too."
Suetonius wrote:
"Since the Jews constantly made disturbances at the instigation of Chrestus, he expelled them from Rome". Life of Claudius (XXv.4)
Again there are a number of questions raised about this passage. Firstly, Suetonius is talking about "Chrestus" - but he spelt "Christians" correctly later in his book, which makes you wonder whether he was actually talking about Jesus at all, and not some other person, especially as 'Chrestus' is the correct Latin form of an actual Greek name. He also seems to imply that there was someone called Chrestus in Rome in 49CE when the expulsion occurred. This makes him a very dubious source indeed.
Tactitus wrote that:
"Consequently ... Nero fastened the guilt and inflicted the most exquisite tortures on a class hated for their abominations. Called Christians by the populace. Christus, from whom the name had its origin, suffered the extreme penalty during the reign of Tiberias at the hands of the Procurator Pontius Pilatus, and a deadly superstition, thus checked for a moment, again broke out not only in Judea, the first source of the evil, but also in the City." Annals (XV.44.2-8)
in his account of the burning of Rome. Again this piece of Tactitus' writing, besides not being by an eyewitness, has a big question mark hanging over its authenticity. This particular piece is not quoted before the 15th Century, and when it was quoted, there was supposed to be only one copy of the 'Annals' in the world, made in the eighth century (600 years after Tactitus' death). Also, Tactitus could not have been using Roman records of Jesus' death (if there were any) because he prefers to Pilate as a Procurator, when in fact he was a Prefect. Again we must discount this passage.
Pliny wrote a letter to the emperor Trajan saying:
"They also declared that the sum total of their guilt or error amounted to no more than this: that they had met regularly before dawn on a fixed day to chant verses alternately among themselves in honour of Christ as if to a god, and also to bind themselves by oath, not for any criminal purpose, but to abstain from theft, robbery, and adultery ..."
He was asking Trajan to advise him what action to take against Christians living in Asia Minor. He proves that there were Christians there, but not that Jesus ever existed.
Thallus is said to have written that Jesus' death was accompanied by earthquake and darkness. His original work has been lost and it was cited only in Julius Africanus' work in the third century. This is the only reference to unusual meterological events occuring after the death of Jesus outside the New Testament, which is strange as such things were routinely recorded. It is impossible to determine whether Thallus actually wrote this, when he wrote this or if the events actually happened as there is no other evidence.
The Talmud says Jesus was the illegitemate son of a Roman soldier called Pandera (or Pandira) who worked magic. However, most of that material derives from 200-500CE and is the Jewish reaction to the spread of Christianity. It is not a contemporary reference but a reaction to a movement.(For anti-Christian parts of the Talmud, including those referring to Jesus, refuted, please see
http://crnews.pastornet.net.au/jmm/aasi/aasi0151.htm)
Many Christians also make reference to the "Acts of Pilate" whicih Justin Martyr said was Pilate's report to Rome of the crucifixion of Jesus. Several other early church writers also referred to this, including Euseubius, who said there was a forged copy of that report circulating in his day. At the present time, the apocryphal Gospel of Nicodemus claims to have this report within it, and there is also another report. The second report, called " The letter of Pontius Pilate which he wrote to the Roman Emperor, concerning our Lord Jesus Christ. is thought by most historians to have been written in the fifth century. The Gospel of Nicodemus is thought to have been written c150-200 which leaves a small possibility that it has a copy of the report of Pilate in it, but the gospel is not accepted by most Christians as being authentic, and most historians doubt that it has the report of Pilate either.
There are some other historical sources, but these are the main (and earliest) ones, so I will not cover them. There is a possibility that Jesus did exist, as vouchsafed by the historical evidence, but the practise of the Christian church in destroying records of Jesus (at one time, anyone attempting to preserve writings which were hostile to him was subject to the death penalty) and of falsifying various others (such as Josephus) has paradoxically made it unlikely we will ever be able to say with certainty that Jesus existed.