You're the one asserting that "God just didn't choose them." so who else should be blamed according to your position?
What better excuse does the unbeliever have than "God didn't love me enough to choose me" according to your position?
By saying that, you call Paul's words into question, or...