Connection between client and remote Raspberry Pi that's behind a router

Question

I want to build an application for Raspberry Pi (C++, Boost, OpenSSL) that handles file download/upload requests (and other custom actions). The RPi has an external HDD connected to it (through USB). Let's call it simply Storage (RPi + HDD).

I have a public server that I can configure for any need. The problem is that the Storage is behind a router that I don't have access to configure (port forwarding is excluded), so, the public server cannot connect to Stroage directly.

Programmatically, what is the optimal way to communicate between a client (web browser / mobile application) and Storage through the public server? A simple use case is this: I want to download a file from Storage to my phone over the internet.

I can think of two possible solutions:

1. Maintain a tunnel open between public server and Storage. Is this scalable for multiple Storages (assuming that the public server will serve many of them in the future)?
2. The Storage pings every let's say 3 seconds the public server (more like announces server that it is online) and if the public server has a request from a client, then a full connection will be established (client <-> public server <-> Storage) and the file download/upload request will be handled normally.

How can I implement this (third party tools like Weaved is excluded) and what other possibilities are there? Also, are the two alternative I described above even viable?

The overall sistem architecture looks like the one below.

I know that this is a more general question, but I don't know how to approach my particular case.

Answer 1

I'll assume your app works on port X.

Install ssh server on PS (public server).

Connect via ssh to PS, and create a reverse port forward, from PS:127.0.0.1:Y (port Y on PS) to S:x

Create a daemon on PS, to listen on port Z. When someone connects to port Z on PS and authenticates (or whatever you need to do) stream data between that new connection and PS:127.0.0.1:Y.

This removes the need of port forwards.

If you want to take a step further, you can use a hole-punching library to try to do a direct connection (but this is much harder than the above solution).

You need to ensure the ssh server on ps allows no commands except authentication and reverse port forward, to disallow S clients to do things on the server (if this is desired).