To be fair, the books do describe non-robe clothing - the Weasley jumpers, for instance.
It's plain that some wizards, at least, do wear only robes; there's the evidence of the one in an argument on arrival at the Quidditch Cup, if nothing else. But it also seems plain that robes aren't the only clothing that wizards wear, either.
Still, the notion that everyone at Hogwarts is wearing pants et cetera underneath their robes is an annoying - and annoyingly common - one.
It's plain that some wizards, at least, do wear only robes; there's the evidence of the one in an argument on arrival at the Quidditch Cup, if nothing else. But it also seems plain that robes aren't the only clothing that wizards wear, either.
Still, the notion that everyone at Hogwarts is wearing pants et cetera underneath their robes is an annoying - and annoyingly common - one.