If someone plugs an Ethernet over power (EOP, Homeplug) into the switch and then connect elsewhere in the building, can I detect this device by sniffing the packets with tools like Wireshark or tcpdump? Or how can I distinguish the frame generated by homeplug and the original network?
Frames that have been bridged from PLC devices don't differ from normal Ethernet frames (they do before bridging only).
To detect rogue devices in your network, you should monitor the switch port for new/unknown MAC addresses. It doesn't really matter whether someone just plugs into your switch or extends the network using PLC, Wi-Fi or something else (nor is it reliably detectable).
Of course, MAC addresses can be spoofed easily but for actually taking part in a network a unique address is required, so an attacker should give themselves away unless they spoof a known MAC that is currently offline.
If you want proper network security on the physical port level you'll need to implement 802.1X or (preferably) MACsec.
Yes and no. The "packets" are transmitted using the Homeplug standard, but there are at least two standards existing. You need to plug a compatible adapter to hook the carrier wave.
Once you do this, the two adapters also need to be on the same circuit (same meter, as far as I know), and the power cable must be of good quality, the circuit segment between the two free of stray capacitances or inductances (e.g. electric motors).
At this point, if it works, then you don't need Wireshark. If you have a first generation adapter with no security, it will light up green and happily report it found a little brother somewhere nearby, even if the Ethernet attached (or not even attached) is not transmitting. You would be detecting, so to speak, the "virtual ethernet cables" ran through the walls by the Homeplug device.
You need Wireshark if you want to know what is transmitted.
All modern devices use AES encryption (AV is AES-128); you won't have the security key, so they will light up orange and ask you to "pair" them. Lighting up orange means that there is a Secure Homeplug device, and the one you plugged in did hear a pairing request which was properly transmitted and understood, so there definitely is a device out there. It's the equivalent of shouting "Is anyone there?" in a dark room and hearing "मुझे तुम्हारी बात समझ नहीं आती" as an answer. For detection purposes, that's a solid hit, since it still means someone is in there.