Kubernetes (K8s) Integration with VMware NSX-T

December 4, 2018

Tweet This:
Share on LinkedIn:

By Cindy Jiang, Kovarus, Distinguished Solutions Architect

The trend of modernizing a data center with cloud-native applications and containers introduces a new problem managing these applications and containers. There are some well-known solutions for solving these challenges, such as Kubernetes (K8s), Red Hat OpenShift, Pivotal Cloud Foundry (PCF), etc. However, there are additional challenges on how to efficiently handle the networking and security aspects of the container environment, and how we can have a cohesive security policy across physical and virtual endpoints. VMware introduced the vision of a Virtual Cloud Network for connecting and protecting any workload across any environment with VMware NSX-T. In this article, we briefly cover how Kubernetes integration with NSX-T solves the networking and security challenges in a Kubernetes environment.

The key design goal of Kubernetes integrated with NSX-T is to:

  • Avoid manual mapping between Kubernetes constructs to networking constructs
  • Automatically secure containers, VMs, and any other endpoints with consistent firewall policies without user interaction
  • Provide visibility and troubleshooting tools to ease the adoption of containers
  • Give freedom to developers.

Traditional Kubernetes networking topology is not true multi-tenancy. NSX-T integration using NSX Container Plugin (NCP) provides native multi-tenancy and builds network and security objects related to Kubernetes namespace automatically. NCP runs as a container inside of a Kubernetes Pod. Developers don’t need to know the existence of NSX-T at all — when they run development tasks on Kubernetes, NCP will see these change requests and react to it by creating related NSX objects such as: logical switches, logical routers, firewall objects, etc.

Here is a sample developer workflow in Kubernetes:

  1. Create a Namespace — NCP automatically creates an NSX-T logical topology for a Kubernetes cluster, and creates a separate logical network for each Kubernetes namespace with its own SNAT IP (NAT mode). In this case, it will automatically create a T1 router connecting to T0 router, create a logical switch, and allocate a subnet for it.
    >kubectl create namespace bar
  2. Create a Pod — This Kubernetes pod will connect to a logical switch, and attach a DFW in front of the pod. NSX-T has a built-in IPAM that provides IP address management by supplying individual IPs and MACs to Pods.
    >kubectl run nginx-bar --image=nginx –n bar
  3. Create an Ingress Policy — NCP watches for ingress events in Kubernetes. NSX-T will automatically create one L7 load balancer like NGINX for all ingresses. NCP creates a new forwarding rule sending a specific HTTP/S hostname and path to a specific Server Pool.
    >kubectl create –f nsx-k8s-ingress.yaml

Sample YAML File:

kind: Ingress
  name: nsx-k8s-ingress
  - http:
      - path: /home
          serviceName: home
          servicePort: 80
  1. Create a Network Policy – NCP translates Kubernetes Network Policy objects to NSX security groups and firewall rules. NSX-T dynamically creates source and destination security groups and applies the correct policy. NSX-T enforces security policies between Pods by leveraging NSX DFW.
    >kubectl create –f nsx-k8s-network-policy.yaml

Sample YAML File:

kind: NetworkPolicy
 name: nsx-k8s-network-policy
   app: web
  - from:
   - namespaceSelector:
     ncp/project: db
   - Port: 80
   - Protocol: TCP

To see NSX-T integration with CaaS/PaaS in action, please contact us to visit our Kovarus Proven Solutions Center (KPSC) in San Ramon, CA. We created the KPSC to let you see what’s possible and learn how we can help you succeed.