MetalLB for External Connectivity
This post explains how we can use MetalLB to attract traffic from a gateway router to kubernetes cluster.
Prerequisites
- kubernetes cluster ( I have 1 master + 2 node cluster)
- Installed pod networking. Calico is used as CNI in my setup
- Gateway router with BGP.
Installation and configuration
- Install Metallb using below yaml file.
This will create one controller and two speaker pods. Controller will be responsible for assigning external IP address from the address pool defined in the config file (step 3).
Speaker nodes will announce the IP address to the gateway router
2. You can view the pods running successfully in namespace metallb-system
3. Create a config map with BGP configuration details and address pool from which externalIP for the services will be picked automatically as shown below
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
peers:
- peer-address: 172.16.0.11
peer-asn: 65001
my-asn: 64512
address-pools:
- name: default
protocol: bgp
addresses:
- 80.11.12.0/30
In the above configuration peer-address and peer-asn are the IP address and asn of my gateway router.
4. After creating config map file check whether BGP peer is established with the gateway router in the speaker logs.
root@master1:~/metalLB# kubectl logs -n metallb-system speaker-fx4d9
......
{"caller":"bgp.go:62","event":"sessionUp","localASN":64512,"msg":"BGP session established","peer":"172.16.0.11:179","peerASN":65001,"ts":"2018-07-06T00:19:51.282711062Z"}
Create a service and test
- Create a service to check whether metalLB assigns externalIP. If you dont want to assign a fixed externalIP to your service you can use loadBalancerIP. Note that the address must be in the range defined in the config map in step 3.
apiVersion: v1
kind: Service
metadata:
name: metalservice
namespace: default
spec:
ports:
- port: 5001
protocol: TCP
targetPort: 80
selector:
run: my-nginx
loadBalancerIP: 80.11.12.1
type: LoadBalancer
- Check whether the service got external-ip
root@master1:~/metalLB# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17h
metalservice LoadBalancer 10.111.117.248 80.11.12.1 5001:30879/TCP 2h
In the speaker logs you can see the external IP is advertised
{"caller":"bgp_controller.go:162","event":"updatedAdvertisements","ip":"80.11.12.1","msg":"making advertisements using BGP","numAds":1,"pool":"default","protocol":"bgp","service":"default/metalservice","ts":"2018-07-06T00:22:18.896880181Z"}
- Verification from router
In the router you can see the externalIP address and next hop as the worker nodes
vm-011 ~ # gobgp global rib
Network Next Hop AS_PATH Age Attrs
*> 80.11.12.1/32 172.16.0.10 64512 02:11:07 [{Origin: ?} {Med: 0} {LocalPref: }]
* 80.11.12.1/32 172.16.0.9 64512 02:11:07 [{Origin: ?} {Med: 0} {LocalPref:}]
*> 172.16.0.0/16 0.0.0.0 00:21:31 [{Origin: i} {Med: 0}]
Check whether you can access the service from the router
vm-011 ~ # wget http://80.11.12.1:5001
Connecting to 80.11.12.1:5001 (80.11.12.1:5001)
index.html 100% |**************************************************************************************************************************************************************| 612 0:00:00 ETA
vm-011 ~ #
