The problem is I've been reading about RADIUS and RADIUSFree server and stuff, but every implementation says you need an AP that supports RADIUS to use it.
You must be misinterpreting what you are reading or looking at bad sources of information.
RADIUS is entirely independent of 802.11 wireless. 802.11 wireless can make use of it for authentication, but RADIUS has been around far longer than 802.11 as it was used back when Internet was dialup for managing accounts and activity.
Is there a way that I can experiment with RADIUS in a "real" setup (not PacketTracer simulation) without the need of extra hardware?
Just because the most common use of RADIUS today is 802.11 authentications, doesn't make it the only use nor does it mean you must run a wireless environment to install, configure and run a RADIUS server.
All you need is a server (i.e. computer or VM) to run it on. You may even be able to run it on the same computer you are running Wireshark.
I don't know why it asks for the layers the protocol works on, and then it says "explain why does it work on that protocol" instead of layers. Don't know if it's an error or there's something I don't understand.
This is one of the exact reasons homework questions are off topic here. We could only guess at what your instructor actually wants from you and if our guess is wrong, you could fail to provide the answers your instructor is looking to see. Or it could be a question that is testing your understanding of the concepts. Either way, it would be best if you asked your instructor what is meant.