Wiggins told Britain's Daily Telegraph newspaper on Wednesday that he moved his two children Ben and Isabella to another school last year because they were repeatedly asked whether their father had used drugs like Armstrong.
Speaking at a training camp on the Spanish island of Majorca, Wiggins said his victory in the 2012 Tour, when he became the first Briton to win cycling's showcase event, was a life-changing event.
''I left home pretty much unknown and came home the most famous man in the country for that week,'' Wiggins said. ''It was hard for me and the family. It affected them as well. The Lance Armstrong thing in January... my kids started getting harassed at school. 'Is your dad on drugs? He won the Tour. Is he the same as Lance Armstrong?' My son getting bullied at school. I had to move my kids from that school and move them to another school.''
Armstrong was stripped of his seven Tour victories and banned for life in 2012. After years of denials, Armstrong finally acknowledged drug use in a TV interview with Oprah Winfrey in January last year.
Wiggins watched Armstrong's confession with his son and explained to him that his dad had won the Tour clean. That did not prevent the bullying at school and the 33-year-old Olympic champion said he felt accountable for it.
''Horrendous stuff. Horrible,'' Wiggins said. ''I felt responsible for that and it all added to my unhappiness at the time.''
Wiggins, who has always taken an uncompromising stance on doping, said he has moved on from that unhappy period and feels ''much more comfortable in my own shoes now'' - despite a disappointing season.
After his Tour triumph, followed by a time trial gold medal at the London Olympics, Wiggins struggled to cope with his new celebrity status. He lost his focus during a year that saw him withdraw from the Giro with a chest infection and then pull out of the Tour de France because of a knee injury.
His working relationship with teammate Chris Froome also broke down as his former lieutenant became Team Sky leader and went on to win the Tour last summer.
However, Wiggins said he is now ready to ride in a supporting role after putting aside his disagreements with Froome during a three-day team bonding session in December.
''We're actually looking forward to racing together,'' Wiggins said. ''It's nice when it's like that. I want to do the team proud and do myself proud.''
Wiggins added he has yet to decide whether to stay with Sky in 2015 or switch back to track cycling with the aim of ending his career at the Rio Olympics in 2016.
''That was where it all started for me in Sydney (2000),'' said Wiggins, a former pursuit specialist. ''To finish there, and hopefully go out on a high, would be great.''