Let us assume Host A wants to contact Host B. Does it send the packet addressing (IP address of the Host B) and MAC address of router, or does it use ARP to send to the MAC address of the Host B?
It works the same way as it would if you were using ethernet on a switch. If a host wants to send something to another host on the same network, it will send it to the MAC address of the destination host. If the destination host is on a different network, the host will send it to the MAC address of its configured gateway.
Based on your comment, it seems you need to step back and understand the functions of the network devices.
Routers route packets between different networks, so traffic is only sent to a router if it needs to get to a different network. If the source and destination hosts are on different networks, the traffic will be routed between the networks by a router.
Bridges, including switches and WAPs, switch frames on the same network. If the source and destination hosts are on the same network, the traffic will not go through a router. The traffic may be switched by a bridge (switch, WAP, etc.).