In vim, backspace does not delete the previous character, but inserts ^?. To delete a character backwards, I should use Control + h. It was a nuisance but using vim with rxvt on cygwin, I cannot expect many people to have this problem.
simple solution:
Add this line in ~/.profile
stty erase ^?
where you generate the ^? using backspace in vim.