Since VSLM is supported only by classless routing protocols, if VSLM was introduced in 1987(RFC 1009) and all classless protocol(RIPv2 developed in 1993 , OSPF standardized in 1989 RFC 1131 ) cited by my Computer Networks book (Tanenbaum 5th edition) are posterior to VSLM introduction, what were the classless routing protocols used with VLSM when it was first introduced?
Actually, VLSM is first mentioned in RFC 950 from 1985. It enabled subnetting the classful address ranges, so that you could actually split a classful range into a more complex network. That eventually evolved to CIDR and obsoleted classful networking.
The first routing protocol actually supporting VLSM seems to be OSPF v1 from RFC 1131 in 1989. Likely, most people used static routing back then - as far as I know static routes have always supported arbitrary subnet prefix lengths (or even non-contiguous masks).
Remember that back in 1985, the total number of hosts on the Internet was around 2,000. Any network with more than a dozen hosts would have been considered huge.