This depends entirely on the switch model.
Simple layer-2 switches, "not likely". The issue is the inability to enter an atomic set of commands. Each configuration change will be active the instant it's entered. So, when you enter
primary-vlan xx, the minute you hit enter your connection will fail.
With layer-3 switches, "Yes". But it takes a careful sequence. Be sure your moving of the mgmt interface and any associated route(s) will not break traffic forwarding/routing through the switch. Bring up IP on a different SVI (vlan). Then access the switch through that new interface, preferably from a local host (i.e. host in the same subnet) to avoid any routing issues. Then move the management vlan, if that model has any notion of "management vlan".
[Doing an in-place network / management redesign with a few nortel (avaya) switches was a real test. That ultimately took a bit of netmask juggling, proxy-arp, /32 routing *grin* (yes, a host can live on the wrong network), and leap-frog telnet's... and that was on a switch down the hall from me.]