5. Operating the Designs
Four designs, three source-address behaviours, one set of rules. Part 5 puts the designs side by side and turns them into operating practice: change procedures that do not strand traffic, what to verify after every change, and who owns which setting.
Intro: Tenant Topology for Cloud Providers running VCF9.1 · 1. Architecture and Foundations · 2. Use-Case A Private-VPC Connectivity to External Networks (Translated) · 3. Use-Case B&C Private-TGW Connectivity to External Networks (Routed or Translated) · 4. Use-Case D One Tenant, Three Exits · 5. Operating the Designs
Parts 1 to 4 built four complete designs for the source address of outbound tenant traffic on a VCF 9.1 multi-connection Centralized Transit Gateway. Part 1 gave the object model and the four rules; Part 2 used Provider SNAT for a Private-VPC tenant; Part 3 advertised a Private-TGW tenant's block to its enterprise and then showed the Provider SNAT alternative; Part 4 combined all three behaviours on one gateway with three exits.
This part is about running them: the designs side by side, the constraints that apply to all of them, change procedures that do not strand traffic, a verification checklist, and the provider/tenant governance split.
The Four Designs Side by Side
| A — Private-VPC + Provider SNAT | B — Private-TGW + advertisement | C — Private-TGW + Provider SNAT | D — one tenant, three exits | |
|---|---|---|---|---|
| Enterprise connection settings | Provider SNAT on, SNAT block = enterprise-facing block | Provider SNAT off, Private-TGW blocks on, advertise rule on the gateway | Provider SNAT on, SNAT block = enterprise-facing block | As B on Riyadh-wan-conn; plus Shared-services-conn with Provider SNAT on, SNAT block = services pool |
| Internet connection settings | Provider SNAT off, authorizes the Internet-facing block | Same | Same | Same (authorizes the public block) |
| Profile | Default Outbound NAT on, Auto SNAT block = Internet-facing block | Same | Same | Same (Auto SNAT block = public block) |
| Address the enterprise sees | Provider-owned (10.100.10.x) |
Tenant private (172.16.103.x) |
Provider-owned (10.100.11.x) |
Private-TGW: tenant private (172.16.10.x); public: 20.0.1.x; Private-VPC: Auto SNAT (20.0.0.x) |
| Address the Internet sees | Auto SNAT block (10.100.20.x) |
Auto SNAT block (10.100.21.x) |
Auto SNAT block (10.100.21.x) |
Auto SNAT block (20.0.0.x); public workloads 20.0.1.x |
| Translations toward the enterprise | 2 (VPC gateway, then CTGW) | 0 | 1 (CTGW) | 0 for Private-TGW and public; 1 for Private-VPC (VPC gateway) |
| Translations toward the Internet | 1 (VPC gateway) | 1 (CTGW) | 1 (CTGW) | 1 for private subnets; 0 for public |
| NAT state on the enterprise path | VPC gateway and CTGW | None | CTGW | None for Private-TGW; services path: CTGW (plus VPC gateway for Private-VPC) |
| Choose when | Workloads on Private-VPC subnets and the enterprise requires provider addresses | Workloads on Private-TGW subnets and the enterprise can route the tenant's unique private block | Workloads on Private-TGW subnets and the enterprise requires provider addresses, or the private block overlaps | A tenant needs a private DC link, a provider services network and the Internet at once, with a different source-address rule on each |
| Built in | Part 2 | Part 3 | Part 3 | Part 4 |
Two Constraints That Apply to Every Design
Provider SNAT and Private-TGW advertisement cannot both be enabled on the same connection. Provider SNAT must be disabled before Private-TGW IP Blocks can be enabled. The rule is per connection, which is why one gateway can advertise on one connection and SNAT on another.
Default Outbound NAT stays on in the profile in every design. It is the Internet path's only source translation. Turning it off removes the only thing that gives private subnets an address the Internet connection authorizes, and for Private-VPC subnets it removes their ability to reach the CTGW at all.
Choosing a Design
Two questions, asked per enterprise-facing connection.
What must the far side see? If it accepts only provider-owned addresses, or must never learn tenant addressing (an enterprise firewall policy, or a shared-services network that serves many tenants), the connection needs Provider Outbound SNAT with a SNAT IP Block the connection authorizes. If it wants to identify workloads directly in firewall rules, logs and monitoring, the connection needs advertisement.
Which subnet type are the workloads on? Advertisement works only for Private-TGW subnets, because their private address is still intact when it reaches the CTGW. A Private-VPC subnet is translated at the VPC gateway before the CTGW sees it, so the only correction available on the enterprise connection is a second translation. Workloads an enterprise must see by their real addresses belong on a Private-TGW subnet, and an advertised private block must be unique and routable in the enterprise; if it overlaps with existing ranges, use Provider SNAT (use case C).
The Internet connection is the same in all four designs: default route, Internet-facing block authorized, Provider SNAT off, Private-TGW blocks off, and the profile's Auto SNAT block set to that connection's block.
Note: Choose the Auto SNAT block for the default-route connection, normally the Internet connection, and let each other connection correct the source on its own. A profile has one Auto SNAT block; it can only be tuned for one exit.
Lifecycle Procedures
Each of these is a change to a live environment. Order matters, because most of them pass through a state where one exit stops forwarding.
Attaching an additional External Connection to a Transit Gateway
- Provider: make sure the connection is assigned to the project (the Set External Connections dialog of the project form). A Transit Gateway can only attach connections assigned to its project.
- Tenant: Networking > Transit Gateways > ⋮ > Edit, pick the additional connection under External Connection, save.
- Wait for the gateway status to change from In Progress to Success before testing.
- Check the routing consequence: the new connection either carries explicit, non-overlapping remote networks or is the one connection with the default route. A second default route is an invalid configuration; the same prefix on two connections is not ECMP.
Attaching a connection changes nothing on the connections already attached. Provider SNAT and advertisement are per connection.
Changing the Auto SNAT block on a live profile
The External IP Block for Default Outbound NAT cannot be changed while Default Outbound NAT is enabled.
- Edit the profile (VPC > Profiles > VPC Connectivity Profile > ⋮ > Edit). Add the new block under External IP Blocks.
- Select it as the External IP Block for Default Outbound NAT and remove the old one.
- Switch Default Outbound NAT off and save.
- Edit the profile again, switch Default Outbound NAT on, and save.
Between steps 3 and 4 there is no default translation: Private-VPC subnets are confined to their VPC, and Private-TGW subnets are reachable only across the VPCs of the same Transit Gateway unless a user NAT rule, Provider SNAT or Private-TGW advertisement supplies a source address. In practice, Private-VPC subnets lose every exit, and Private-TGW subnets lose the Internet path while an enterprise path served by Provider SNAT or advertisement keeps working. Plan the window accordingly.
Moving a connection from Provider SNAT to advertisement (C to B)
- Provider: edit the connection and disable Provider Outbound SNAT; it must be disabled before the next toggle can be enabled. The Provider SNAT rule and its No-SNAT rule are removed with the toggle.
- Provider: enable Private-Transit Gateway IP Blocks on the connection and save. This grants permission and advertises nothing by itself.
- Tenant: Transit Gateways > expand the Default Transit Gateway > Routing and Forwarding > Advertise Rules, add the type Private-Transit Gateway for that connection.
- Confirm the Private-TGW block appears on the provider gateway as a
tgwsroute.
Between step 1 and step 3, enterprise-bound traffic from private subnets carries the Auto SNAT address, which that connection does not authorize: the enterprise is unreachable until the advertisement is complete.
Moving a connection from advertisement to Provider SNAT (B to C)
The two settings are mutually exclusive, so the target state is: Private-TGW IP Blocks off on the connection, no Private-Transit Gateway advertise rule for it on the gateway, Provider Outbound SNAT on with a SNAT IP Block drawn from the connection's authorized External IP Blocks. Once Provider SNAT is on, the enterprise sees the SNAT block instead of the tenant's private addresses, so update enterprise-side firewall rules and logging that referenced workload addresses at the same time.
Adding profiles for VPCs with different needs
A project can hold several VPC Connectivity Profiles, and many VPCs can share one. Use Add VPC Connectivity Profile for VPCs that need different blocks or services. Each profile carries up to five External IP Blocks, which must have been assigned to the project by the provider, and up to five project-owned Private-TGW blocks, selected or created in place.
Block hygiene
- Name blocks after the tenant and the exit they serve, with the CIDR in the name (
Cairo-wan-block_10.100.10.0/24); every later dropdown shows the name only. - Inside a block, CIDRs and ranges cannot overlap and must be of the same IP address type; a CIDR can allocate subnets or single IPs, a range single IPs only. Use Excluded IPs for addresses that must never be handed to tenants.
- Give every project a distinct Private-TGW block that is unique and routable on the enterprise side if it will ever be advertised. Private-VPC ranges may repeat across projects without harm, because Private-VPC subnets are never advertised.
- A SNAT IP Block must be one of the connection's own authorized External IP Blocks; the dropdown offers only those. Authorize the block on the connection first.
Verification Checklist
Run this after any change to a connection, a project, a Transit Gateway or a profile.
Object state
- Transit Gateway status is Success after every connection attach.
- Each connection's External IP Blocks list contains exactly the ranges that should be advertised over it, and nothing else.
- Exactly one connection on the gateway carries the default route; every other one lists explicit remote networks, and no prefix appears twice.
- On connections with Provider SNAT: the toggle is on and the SNAT IP Block is one of the authorized blocks. On connections with advertisement: the provider toggle is on and the tenant's advertise rule of type Private-Transit Gateway exists for that connection.
- The profile has Default Outbound NAT on and an Auto SNAT block that the default-route connection authorizes.
Routing state on the provider gateways
- The enterprise VRF holds prefixes from the authorized enterprise-facing block, plus the project's Private-TGW block as a
tgwsroute if advertisement is in use. - The Internet Tier-0 holds prefixes from the Internet-facing block only. The private block must not appear there.
- A services Tier-0 (Part 4) holds prefixes from the services SNAT pool; that advertisement is what brings replies to translated sources back.
Data path
From a test workload on each subnet type in use, reach every exit and confirm both success and the translation count the design predicts:
# Design A (Private-VPC workload)
ping <enterprise host> # success; translated twice (VPC gateway, then CTGW)
ping <internet host> # success; translated once (VPC gateway)
# Design B (Private-TGW workload)
ping <enterprise host> # success; not translated anywhere on the path
ping <internet host> # success; translated once (CTGW)
# Design C (Private-TGW workload)
ping <enterprise host> # success; translated once (CTGW, Provider SNAT)
ping <internet host> # success; translated once (CTGW, Default Outbound NAT)For Part 4's three-exit design, the nine-cell matrix is the expected result: check the source address the far side sees for each workload type at each exit.
What the NAT rules should look like
- With Default Outbound NAT on: one auto-plumbed SNAT rule per VPC, applied at the VPC gateway for Private-VPC subnets and at the CTGW for Private-TGW subnets.
- With Provider SNAT on a connection: a Provider SNAT rule and a No-SNAT rule on the CTGW, scoped to that connection only, created and removed together with the toggle. The No-SNAT rule ranks below any user-defined TGW SNAT rule, and user-defined NAT rules take priority over the default rule.
- With advertisement on a connection: no Provider SNAT or No-SNAT rules for it.
Governance: Who Owns Which Setting
The provider and the tenant each control one half of the design. Making the split explicit prevents the two-role failures (a toggle without a rule, a rule without a toggle) and keeps the provider in control of what leaves its network.
| Setting | Owner | Space | Effect on the other role |
|---|---|---|---|
| External IP Blocks (create, name, exclude IPs) | Provider | Default project | Defines every range a tenant can ever consume |
| External Connections: type, gateway, remote networks, authorized blocks | Provider | Default project | Defines the exits, the routing table and the advertisable ranges |
| Provider Outbound SNAT and SNAT IP Block | Provider | Default project, per connection | Translates every source outside the connection's authorized blocks on that exit; a provider-space setting the tenant does not edit |
| Private-Transit Gateway IP Blocks toggle | Provider | Default project, per connection | Permission only; advertises nothing until the tenant acts |
| Project: assigned connections and blocks, edge cluster, span | Provider | Default project | Bounds what the tenant can select |
| Transit Gateway attachments | Tenant | Project | Limited to connections assigned to the project |
| Advertise Rules (type Private-Transit Gateway) | Tenant | Project | Only effective where the provider enabled the toggle |
| VPC Connectivity Profile: blocks, Private-TGW block, Default Outbound NAT, Auto SNAT block | Tenant | Project | Limited to blocks assigned to the project |
| Subnet type of each workload (public, Private-TGW, Private-VPC) | Tenant | Project | Decides what each exit is able to see |
Three consequences follow.
Provider SNAT is the provider's guarantee. It is set on the connection in the Default project, so a tenant cannot expose its own addressing on an exit where the provider has decided it must not appear. Part 4's Shared-services-conn is the clearest example: every tenant reaches the services network only through the provider pool, and the setting that guarantees it lives in provider space.
Advertisement is a shared responsibility with an external dependency. The provider's toggle authorizes the tenant to inject a block into the enterprise routing domain through the VRF's BGP peering. The provider should enable it only where the block is known to be unique and routable on the enterprise side; the tenant's rule then decides what is announced (public blocks, Private-TGW blocks, or both).
Per-connection scoping is what makes change safe. Adding an exit, or changing the setting on one, does not alter the other exits. A provider can roll out a shared-services connection to many tenants without revisiting each tenant's enterprise link.
Key Takeaways
- The source address a tenant packet carries is decided in two places: first by the profile's Default Outbound NAT rule, applied at the VPC gateway for Private-VPC subnets and at the CTGW for Private-TGW subnets, then by the exit connection's own setting.
- A connection forwards only sources from the blocks it authorizes. One profile has one Auto SNAT block, so with two connections that authorize different blocks the second connection needs Provider SNAT or advertisement.
- Provider SNAT keeps the tenant's private addressing hidden and shows the enterprise a provider-owned range; advertisement shows the enterprise the real private addresses and removes NAT from that path. Advertisement is possible only for Private-TGW subnets, because their private address is still intact when it reaches the CTGW.
- The settings are per connection, so one gateway can combine them: advertise the private block to the enterprise DC, apply Provider SNAT toward a shared-services network, and rely on Default Outbound NAT toward the Internet, all at once.
Strategic Recommendations
Standardise the block layout per tenant. One enterprise-facing block and one Internet-facing block, named after the tenant and the exit they serve with the CIDR embedded, and a distinct Private-TGW block per project. When a block's name tells you which connection authorizes it, a good share of the misconfigurations Parts 2 to 4 warn about never happen.
Tune the profile for the default route and let the connections do the rest. The Auto SNAT block belongs to the Internet connection. Every other exit gets Provider SNAT or advertisement, decided per exit by what the far side must see.
Decide subnet type by exit visibility, not only by VPC scoping. A workload an enterprise must identify directly goes on a Private-TGW subnet. A workload that must never be seen by its own address anywhere can stay on a Private-VPC subnet and rely on the double translation.
Treat advertisement as a routing integration, not a NAT alternative. It injects a tenant block into the enterprise's routing domain over the VRF's BGP session. Confirm uniqueness and routability before the provider enables the toggle, and keep Provider SNAT (use case C) as the fallback for the day the enterprise's address plan changes.
Write the two-role workflows into your runbooks. Every advertisement needs a provider toggle and a tenant rule; every change of the Auto SNAT block needs an off/on cycle; moving a connection from Provider SNAT to advertisement passes through a state in which the enterprise is unreachable until the tenant's rule is in place. Write them down as sequences, with the maintenance window they imply.
Keep Default Outbound NAT on. It is the Internet path's only source translation in every design, and for Private-VPC subnets it is the only reason they reach the CTGW at all.
Closing the Series
A multi-connection Transit Gateway in VCF 9.1 is a router with a source-address policy per exit. The routing table comes from Remote Networks. The policy comes from one profile-level behaviour, Default Outbound NAT, and two per-connection behaviours, Provider Outbound SNAT and Private-TGW advertisement, applied in a fixed order. Everything in this series, from Cairo's double translation to Riyadh's nine-cell matrix, is the four rules of Part 1 applied to a specific requirement. Learn the rules, name your blocks, and the designs follow.
Intro: Tenant Topology for Cloud Providers running VCF9.1 · 1. Architecture and Foundations · 2. Use-Case A Private-VPC Connectivity to External Networks (Translated) · 3. Use-Case B&C Private-TGW Connectivity to External Networks (Routed or Translated) · 4. Use-Case D One Tenant, Three Exits · 5. Operating the Designs