If I have a switch or laptop running a PPPOE server, could I go from ethernet-to-modem then connect the modem's WAN port to another modem's WAN port and have it connect via PPPOE?
No. A DSL modem is the CPE part of a DSL link. You can't connect two CPEs, only CPE to COE, e.g. a DSLAM. DSLAMs usually require substantial infrastructure, so a lab environment may be costly.
However, leaving out the problematic DSL link and focus on the higher level function (like PPPoE) you could use a simple Ethernet port as "WAN". Most business-grade services nowadays use some kind of Ethernet as handover protocol anyway.
If need be you could try simple VDSL media converters as COE along with a VDSL modem. This approach isn't easy to build but it's possible.