We have this 11 year old orphan. He is in an orphanage during wartime. So, we can assume it is over crowded and under funded. He is the only magical child in the orphanage. So he is the weird kid. Children are cruel and violent, especcially when in cramped cndtions with little supervision. Most orphanages are run by churches, so it was likely he was targeted by the adults as well, they might have thought his magic was the devil, he was possessed or simply that he was a witch and therefore evil.

So, Dumbledore is at least indirectly responible for Tom becoming Voldemort. Who knows, if Dumbledore hadnt screwed up the letter, then Tom might have become Minister of Magic. After all, not all sociopaths are homicidal