It is well known that a server is a system that offers a service.
It is also well known that a server to offer a publicly available service needs a way to be reached from the outside network (the Internet).
In the modern netwrok scenario having a public IP has a cost. Techniques like NAT and PAT have been introduced to make less urgent the need of a public IP address allowing a server with a private IP to be accessed from a public network.
When dealing with security you are always facing with the usability. If increasing security means decreasing usability, probably, it will translate into decreasing security [reference here] (I'm refering to comments where you are proposing some kind of reverse tunneling).
So essentially NAT and PAT are not inherent secure or insecure. They are just means for a service to be accessed from a public net. If you are concerned with security focun on the security of the infrastructure (with a firewall for example).
Ironically you coul see a security issue even in electricity.