You're the one asserting that "God just didn't choose them." so who else should be blamed according to your position?
What better excuse does the unbeliever have than "God didn't love me enough to choose me" according to your position?
By saying that, you call Paul's words into question, or worse, you call him a liar:
For the wrath of God is revealed from heaven against all ungodliness and unrighteousness of men, who suppress the truth in unrighteousness,because what may be known of God is manifest in them, for God has shown it to them.For since the creation of the world His invisible attributes are clearly seen, being understood by the things that are made, even His eternal power and Godhead,
so that they are without excuse,because, although they knew God, they did not glorify Him as God, nor were thankful, but became futile in their thoughts, and their foolish hearts were darkened. - Romans 1:18-21
http://www.biblegateway.com/passage/?search=Romans1:18-21&version=NKJV