From the Dictionary of Nautical, University, Gypsy and Other Vulgar Tongues first published in 1859.
Belcher: a kind of handkerchief. See Billy.
Billy: a silk pocket handkerchief. See Wipe.
Wipe: a pocket handkerchief. Old cant.
(It could have been called a Belcher because you covered your mouth with it when you belched. That would at least make sense. And a Wipe is pretty self-evident. But why was it called a Billy?
Perhaps Billy was a specific person. I did find these specifics of a Belcher – close striped pattern, yellow silk, and intermixed with white and a little black; named from the pugilist, Jim Belcher. How perfect is that!)