Accessing the public IP address, source NATing internal clients, forwarding the public IP back to a private one (destination NAT) and everything in reverse for a reply is highly inefficient. This "hairpinning" is very often deactivated on a router by default, some routers even don't support it at all.
Instead, make sure your private IP clients access the server's private IP. The easiest way is to use split-brain DNS on your DNS server (details are off-topic here). Not only does this remove load from the router (hairpinning is often done in software) but it's also much faster, enabling your clients to utilize the server's full bandwidth over a switched connection.