Compiling and running VPP in LXD container revisited

A while ago I wrote about Compiling and running VPP inside LXD container.

Since then this has become my primary workflow - one LXD container per work task, pleasant, modular, reproducible and easy to clean.

Needless to say, manually going through all the steps is quite boring. So I wrote a couple of scripts automating the creation of the logistics - so I can issue a command and attend to other things, and a few minutes later I have the container with all the dependencies installed, VPP tree checked out and all the things done up to and including "make build". This is quite handy for me, so I thought to document this for anyone interested.

First, create a directory called "shared" in your home directory. This is not VPP-specific, just very very handy for exchanging the data, and as a default persistent data store that survives the container being deleted.

Then ensure your LXD profiles are set up right. Run "lxc profile create hugepages", "lxc profile create tuntap" and then "lxc profile edit hugepages" and "lxc profile edit tuntap" and "lxc profile edit default", to edit the profiles so that you get the results of "lxc profile show ..." identical to below:

ayourtch@xenial:~$ lxc profile show hugepages
name: hugepages
config:
  raw.lxc: |
    lxc.mount.entry = hugetlbfs dev/hugepages hugetlbfs rw,relatime,create=dir 0 0
  security.privileged: "true"
description: ""
devices: {}


ayourtch@xenial:~$ lxc profile show tuntap
name: tuntap
config: {}
description: ""
devices:
  tun:
    path: /dev/net/tun
    type: unix-char


ayourtch@xenial:~$ lxc profile show default
name: default
config: {}
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  shared:
    path: /shared
    source: /home/ayourtch/shared
    type: disk
ayourtch@xenial:~$

NOTE: You still MUST ensure hugepages are enabled on the host, i.e. this step from the previous blog post is done:

ayourtch@xenial:~$ echo "vm.nr_hugepages=1024" | sudo tee -a /etc/sysctl.d/20-hugepages.conf
vm.nr_hugepages=1024
ayourtch@xenial:~$
ayourtch@xenial:~$ sudo sysctl --system
ayourtch@xenial:~$ sudo sysctl vm.nr_hugepages
vm.nr_hugepages = 1024
ayourtch@xenial:~$

Then, copy the "vpp-bootstrap" file, which you can find at this blogpost below, into ~/shared/vpp-bootstrap. You can call this file yourself from a newly created container, or use the "16-vpp-new-dev" script to create and adjust the container for you.

You will also notice that the scripts try to copy the files from /shared/dot-ssh/* into the new container - I use this to avoid having to re-add the ssh keys ont the server side each time I create the container. Since the containers stay on the same host, are the same user, I think it is a reasonable approach.

Contact me on twitter @ayourtch or via email with any feedback!

Index of /blog/2016-11-26-Compiling-and-running-VPP-in-LXD-container-revisited/