Obviously these are my opinion, but your ideas about scripture are also your opinion so . . .
In the Garden yes. I'm not talking about the garden. The Garden is a tiny spot on earth. Why is getting kicked out of the garden such a bad thing if the entire earth is the same paradise?
No more access to the Tree of Life.
It's literally the reason God kicked them out, stated EXPLICITLY in scripture.
Then the Lord God said, “Behold, the man has become like one of Us, to know good and evil. And now, lest he put out his hand and take also of the tree of life, and eat, and live forever”—therefore the Lord God sent him out of the garden of Eden to till the ground from which he was taken.So He drove out the man; and He placed cherubim at the east of the garden of Eden, and a flaming sword which turned every way, to guard the way to the tree of life. - Genesis 3:22-24
http://www.biblegateway.com/passage/?search=Genesis3:22-24&version=NKJV
But oddly enough, none of them are mentioned in scripture until much later, they just "appear" out of nowhere.
So what?
God didn't prohibit marriage between close relatives until Moses' time (most likely due to the genetic harm it would bring to the offspring).
Cain married his sister, so did Seth, so did each of their descendants. All it would take is for each married couple to have more than two kids each and the population would grow very quickly.
In other words, there's a perfectly biblical explanation for where the people came from that DOES NOT require God to have createfd other people besides Adam and Eve.