Most basic example of neighbor ports would be as you mentioned: "ports on two different switches (or routers) that are connected by a cable"
You won't find a defined definition for this phrase because it will also depend on the context, for example when you are talking at a L3 from the OSI Layer, you might be going across several L2 ports, which each one along the way are directly connected, however from a protocol standpoint the neighbor port might be considered several L2 hops away.
Lets say you have a dedicated line with your service provider, your neighboring port from a LLDP connectivity standpoint might be many miles and port connections along the way but all being considered L2, hence your endpoint device would be considered the actual LLDP neighbor port (L3).
So although your statement is true, it still always good to have a clear context, as some people might be referring of a neighbor port on a different perspective.
Indeed engineers might also refer to: "Are they the ports physically neighboring each other on the same switch" , which could bring confusion if not stated properly, I would say a phrase like that in this specific scenario is/would be used for clarification purpose or to rephrase a statement for further understanding, i.e while troubleshooting a linecard, although I would preferably say something like - check the port next to it, number XYZ, instead of calling it "neighbor".