My days usually begin between 3 and 4, not because I have to get up but because I wake up and can't go back to sleep. I stopped trying a long time ago and just enjoyed it for what it was... quiet, peaceful, a good time for thinking. And having nice cup of coffee.
Once in a great while I'll go back to bed around 5 or so and get another hour of sleep but that doesn't happen very often.
Maybe you could go back to sleep more easily if you didn't drink coffee.