How to connect several standardized networks to a common network?

Question

Imagine, if you will, a series of custom built modules (building size). Each module is setup with standardized control/automation equipment (PLC's, VFDs, HMIs etc.) which communicate to each other, within the module, via an Ethernet based industrial protocol (e.g. EthernetIP, Modbus TCP). As each module uses a standardized configuration for the control/automation equipment, PLC X in Module A will have the same IP address configuration as PLC X in Module B.

Now we install several of these modules at a site and wish to connect them to a centralized monitoring station (control room) with site-wide SCADA HMI and Historian. Since each module has the same configuration (utilizes same IP addresses) simply connecting them to a common network is problematic due to conflicting IP addresses, also configuring unique IP addresses for each Ethernet connected component across all modules is undesirable due to the extra effort, loss of configuration standardization and risk of human error (for example, don't want the PLC in module A accidentally controlling one of the 20 or so VFDs in Module B).

This the part I want feedback on (below)…

It seems to me that a solution is to equip each module with a NAPT (Network Address Port Translation) capable device (e.g. router or NAT/PAT capable switch), we'll call it a router. The router is configured with a unique IP address on the interface attached to the common network and a standardized IP address configured on the interface attached the module network. NAT/PAT rules will be added to the router configuration to allow the common SCADA system to access module equipment (e.g. module PLC).

Can anyone suggest improvements (or point out flows) on this implementation. I'm interested to know if there is more appropriate network technology for this use case (bridges, gateways etc.).

Also, I want to implement VLANs which extend into the module to allow separation of controls network (described above), data network (for variable computing equipment within the module), and business network.

Answer 1

I would consider a network design that brings along overlapping addressing by design harmful. Avoid it like the plague. In new/fresh designs, avoid NAT. Imagine the pain when you discover that one of the applications does not cope well with NAT and/or port forwarding.

I would suggest (and now this is getting opinionated, hence possibly off-topic)

Use some features only IPv6 can offer, and go IPv6-only for a modern network.

• each module gets an IPv6 capable router as entry/exit point towards the common network.
• towards the module, the router exposes as many VLANs/subinterfaces/subnets as needed, to accomplish the inta-module split of business/data/control "zones". Some rough-hewn access-lists or a firewall (e.g. ZBFW, if it's a Cisco) ensure that traffic cannot be forwarded between zones, but only to/from the central network (or the respective "zones" thereof).
• the router acts as DHCPv6-PD (prefix delegation) client towards a DHCPv6 server in the common network, requesting the delegation of a suitably sized IPv6 prefix.
• using appropriate mechanisms (i.e. a dynamic routing protocol), the central network then ensures that routing information for the freshly delegated prefix is disseminated appropriately (i.e. all routers participating in the common network eventually know that prefix 2001:0db8:000a::/48 is now reachable via router "A" and they know how to reach router "A" .) There are many different ways to accomplish this, all depending on how the common network is built (a flat LAN? a somehow emulated/stretched broadcast domain instead of a LAN ? PPPoE Sessions to a number of concentrators? a bunch of serial lines with PPP? an MPLS L3-VPN? Some MetroEthernet L2-service by a carrier? ... the possibilites are endless).

The router and the given module's systems then ensure that every system within the module gets predictable IPv6 addresses.

1. Either by being a stateful DHCPv6 server, assigning leases as <DelegatedPrefix> + <ModuleInternalSubnetID> + <predefinedHostID> (although, that might be a bit of a challenge on its own there, attempting to set up a stateful DHCPv6 service based on a DHCPv6-PD delegated network prefix with just the HostID part being preconfigured/fixed)
2. Or by implementing something like Linux's ip token on the end systems, where the end system does SLAAC to learn the current subnet prefix, but then tries it's own manually predefined HostID, performs DAD with that and brings up the IPv6 interface.

So much for (somewhat) preditctable addresses, With IPv6, having predictable addresses does not offer too much added value - it's still a bunch of Hex digits no one can remember. Enter DNS.

Some DNS magic might be advisable. Once the central DHCPv6 server assigns prefix 2001:0db8:000a to router "A", it could proceed with a script that generates a set of pre-known AAAA records like <oneOfTheKnownEndSystemNames>.moduleA.internal.myscadanet.tld, with each a ressource record of 2001:0db8:000a:<ModuleInternalSubnetID>:<givenHostID>. That's probably best done with DDNS (research topic: Dynamic DNS updates, not the DynDNS style HTTP update thing). That script may reside on the DHCPv6 server itself, which would in turn be the only system allowed to dynamically update the neighboring DNS server.

Hm. A bit on the progressive side, all of this. But still - this is the 2010's. IPv6 is a thing. Talk to your vendors.

Answer 2

For those interested in the topic and use case of per machine/module NAT in an industrial environment. I am providing a link (and attachment) for a white paper jointly produced by Cisco and Rockwell Automation.

Deploying Network Address Translation within a Converged Plantwide Ethernet Architecture