As the title suggests. I find it mind breaking to think how network architecture can differ to allow/not allow using public IP address to access a server within internal network from within the internal network.
So lets assume you have a private network, which is connected via a NAT box to the public internet. On the nat box explicit rules (sometimes referred to as "port forwards") are configured to allow servers on the private network to be reached from the public network.
In this case whether or not the connection is successful comes down to the behavior of the NAT box.
Normally the NAT box modifies the source of outgoing connections* from the private network to the public network. Since explicit rules are configured it also modifies the destinations of incoming connections based on those rules. The NAT maintains a tracking table so that packets for a given connection are treated consistently.
Connections from a client behind a nat** to a public address used for incoming connections by the same NAT are something of a special case. The NAT must modify the destination, despite the fact that the connection came from the inside. Furthermore in most cases the NAT must also modify the source of the connection, despite the fact that the connection is not going to outside.
This is perfectly doable, but it often requires extra rules in the NAT engine, for example https://unix.stackexchange.com/questions/282086/how-does-nat-reflection-nat-loopback-work discusses the rules that are necessary to make "nat reflection" work in iptables NAT.
Whether allowing this is a good idea is also questionable. Traffic from internal hosts to the public IP will unnecessarily go through the NAT box, and logs on the server for such traffic will not show the true source of the traffic. For these reasons some network admins prefer not to allow such connections and instead to use "split DNS" to direct the traffic from internal hosts to the internal IP of the server.
* Connection here refers to a combination of packet metadata that can be used to match-up requests and responses. This is somewhat broader than the usual definition of connection.
** Sometimes machines on the same private network can be behind different NATs, especially if the private network is large, in this case the NATs would not have to do anything special.
Routing to WAN with source NAT and immediately routing back with destination NAT is called NAT hairpinning.
Some NAT routers don't support NAT hairpinning at all, some can be configured to do it but suck at performance, and some do OK.
Generally, routing packets through a NAT construct that is completely unnecessary is a bad idea. Also, forcing traffic through a gateway that isn't really required is a general waste of resources and a potential bottleneck. Additionally, hairpinning hides a client's LAN address from a server and may be a security problem.
You should make sure that LAN clients access a LAN service using its LAN address. One easy way to achieve that is to use DNS names and to configure your local DNS server so that it resolves those services to their LAN address (split-brain DNS) instead of relying on DNS resolution from the public domain.