Internet centralization and the original sin of NAT

In this comic, the concept of an ordinary person having a FTP server is quickly dismissed. And yes, it’s not common. To the average computer user, the idea that someone could just… connect to your computer feels exotic, or even dangerous — see the very common ironic fear of your IP address being known to other people on the internet.

If you take someone who’s “good with computers” but not a networking person, their mental model of The Internet probably involves a definition of “servers” or “the cloud” that distinguishes them from personal computers in some meaningful way. True peer‐to‐peer, if they ever think about it, is an endeavor: WebRTC, STUN, TURN, ICE, what have you. Given that we live in a world of NAT, CGNAT, and restrictive ISPs, this isn’t entirely wrong, but it breaks the elegant design of the original Internet.

Why you don’t have a FTP server

Network address translation (NAT) was first formally proposed in RFC 1631 in 1994. In its abstract, it says:

The two most compelling problems facing the IP Internet are IP address depletion and scaling in routing. Long‐term and short‐term solutions to these problems are being developed. The short‐term solution is CIDR (Classless InterDomain Routing). The long‐term solutions consist of various proposals for new internet protocols with larger addresses.

Classless interdomain routing is not the point of this post, but basically we started giving people more options for network sizes, and while complex in implementation, it was philosophically virtually uncontroversial.

RFC 1631 proposed a second short‐term solution to IP address depletion and scaling in routing: NAT. While it is not exactly the same type of NAT omnipresent on home routers today, the basic idea is the same: it allows multiple devices to share an IP address (from the perspective of a device on the other end of a routing device) by modifying the network address information in the IP packet headers while transferring the packet across a traffic routing device. We then later reserved certain addresses for private use, and these things are used in conjunction on most IP networks — private addresses within the network, NATing to one public address at the router. On your typical home router, here’s how you usually connect to an external server with NAT¹:

1. Your computer sends a packet like this:

   Source IP	10.11.70.21
   Source Port	50413
   Destination IP	67.215.249.229
   Destination Port	70

2. It hits your router, and it modifies it to this:

   Source IP	146.7.15.85
   Source Port	60612
   Destination IP	67.215.249.229
   Destination Port	70

3. The server replies:

   Destination IP	146.7.15.85
   Destination Port	60612

4. Your router rewrites it back:

   Destination IP	10.11.70.21
   Destination Port	50413

If you’ve thought this through, you might be asking: in the situation that an external server wants to talk to you first, how does that happen? It sends a packet to 146.7.15.85, and your router…

Oh no. It has no idea where to send it.

Working around it

Naturally, people noticed this was a problem almost immediately, because people have wanted to run game servers, FTP servers, and web servers from their bedrooms since roughly the beginning of time. So a whole ecosystem of workarounds grew up around NAT, none of which restore the fundamental intention of the internet, and none of which work for everything.

Port forwarding

The most direct fix is to just tell your router “hey, when a packet comes in on port 60612, send it to 10.11.70.21 on port 50413, no questions asked.” This is port forwarding, and it’s the workaround to NAT that the most people are aware of. One of the problems with port forwarding, conceptually, is that one public IP+port can still only map to one device at a time, which means that two devices can’t operate a service on the same public IP+port at the same time. This is more of a problem than it sounds like; on big enterprise or university networks that choke down to a small number or even just one private IP, this basically kills on‐prem hosting without doing even more complicated shit. And sometimes, your ISP has put your external IP behind NAT too — which is called carrier‐grade NAT (CGNAT) — and now you don’t control the device doing the translation, so you can’t forward a port. You’re getting a fraction of a fraction of an IP address.

Also, another problem with NAT is that nobody wants to bother with it, which is why we invented:

UPnP

UPnP, and its modern cousins NAT‐PMP and PCP, tried to solve the “nobody wants to bother with it” problem by letting software ask the router directly to forward ports. Like manual port forwarding, it’s a request to your router — if your ISP is screwing with you, you’re out of luck. It’s also frequently disabled because of misguided security thinking — partially because of a couple buggy early implementations, and partially because the idea that someone could just connect to your computer feels exotic or even dangerous to a lot of people. There are plenty of valid reasons to want a firewall, but if you do, intentionally implement one instead of relying on NAT just not knowing where to send packets.

STUN, TURN, and ICE

STUN

Session Traversal Utilities for NAT (STUN), instead of trying to get cooperation from the firewall, simply asks a server on the public internet “what does my packet look like by the time it gets to you?” The STUN server hands back the public IP and port your NAT assigned, say, 146.7.15.85:60612. Under a “cone NAT”, where the router uses an identical external port mapping for all outbound connections, this works great. You can tell this mapping to a peer, and then they can send packets directly to you. This technique is known as hole punching. However, under a “symmetric NAT” — common on CGNAT and institutional networks — you get a different public port for every distinct destination. In this case, the STUN mapping is useless for connecting to a peer, since they'll see you differently than the STUN server..

TURN: giving up

Traversal Using Relays around NAT (TURN) is simply just passing traffic through a relay server, with both sides speaking to it outbound. This works mostly everywhere, but since someone has to run a server that should be unnecessary and you have to eat the added latency of every packet detouring through a third party, this really sucks.

ICE: trying everything

Interactive Connectivity Establishment (ICE) accepts that no technique is reliable and tries all of them in order of preference. Consider everything: direct connect, STUN‐discovered external address, a TURN relay), exchange the list with the other side, and throw shit at the wall until something works. This is what WebRTC does, and it’s the best you’ll get on today’s internet. But we’ve replaced a simple direct connection with, mostly, external infrastructure.

The long‐term solution that wasn’t

The principal “long-term solution” in the works that RFC 1631 was referring to was IPv6, and it was supposed to fix this; give everyone a real globally unique address and obviate NAT. However, the sigmoid function of IPv6 adoption seems to be stalling out too early, and even where it is implemented, many ISPs and institutional networks keep doing NATy stuff out of inertia and even more misguided security thinking: firewalls that refuse inbound because that’s we’re used to NAT doing that, or completely unnecessarily applying actual NAT to IPv6 — often deploying Unique Local Addresses (fc00::/7) the way they use private RFC1918 space on IPv4 — which is baffling to me.

The consequences for the Internet

There’s lots of things you can blame for killing the open Internet, but I think NAT was one of the earliest. Running a server used to be trivial: run an executable, tell people your address, done. Now, if you’re lucky, you probably have to configure port forwarding, which you often can’t even do if you’re behind CGNAT or on an institutional network.

It also trained everyone to think client‐server is natural. “My device talks to The Cloud which talks to other devices” feels normal, when that feeling originated as an artifact of address scarcity. Even more ironic is that NAT got normalized as a security feature — “your devices are hidden!” — which is one of the things that made people resist the thing that would fix it.

NAT certainly isn’t the only reason why the modern internet is full of centralized walled gardens, but it was the first — it’s why it’s hard to send a file to someone, it’s why you don’t run your email on your own computer, and why running your own services at all is difficult and often expensive (if you can’t port forward from your own internet connection, you have to buy a VPS instead of using hardware you already have).