The scholarly consensus of the estimates of the native indigenous populations of North America have been trending upward over the years and it's based on sound archaeological evidence. You might find the following informative.
The Native Population of the Americas in 1492
Second Revised Edition...
Some was to be sure. Much wasn't. The wealth in my neck of the woods, for example, was NOT built on slavery. In fact my great-great-grandfather risked his wealth and his life to end slavery.
Due primarily to disease.