Fabio Rehm 9e0e0d50b5 [GH-54] Fix zypper cache path

2013-11-06 19:16:54 -02:00

9.9 KiB

Raw Blame History

vagrant-cachier

A Vagrant plugin that helps you reduce the amount of coffee you drink while waiting for boxes to be provisioned by sharing a common package cache among similiar VM instances. Kinda like vagrant-apt_cache or this magical snippet but targetting multiple package managers and Linux distros.

Installation

Make sure you have Vagrant 1.2+ and run:

vagrant plugin install vagrant-cachier

Usage

The easiest way to set things up is just to enable cache buckets auto detection from within your Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = 'your-box'
  config.cache.auto_detect = true
  # If you are using VirtualBox, you might want to enable NFS for shared folders
  # config.cache.enable_nfs  = true
end

For more information about available buckets, please see the configuration section below.

Compatible providers

Vagrant's built in VirtualBox provider
vagrant-lxc
VMware providers with NFS enabled (See GH-24 for more info)

How does it work?

Right now the plugin does not make any assumptions for you and you have to configure things properly from your Vagrantfile. Please have a look at the available cache buckets section below for more information.

Under the hood, the plugin will monkey patch Vagrant::Builtin::Provision and will set things up for each configured cache bucket before running each defined provisioner and after all provisioners are done. Before halting the machine, it will revert the changes required to set things up by hooking into calls to Vagrant::Builtin::GracefulHalt so that you can repackage the machine for others to use without requiring users to install the plugin as well.

Cache buckets will be available from /tmp/vagrant-cachier on your guest and the appropriate folders will get symlinked to the right path after the machine is up but right before it gets provisioned. We could potentially do it on one go and share bucket's folders directly to the right path if we were only using VirtualBox since it shares folders after booting the machine, but the LXC provider does that as part of the boot process (shared folders are actually lxc-start parameters) and as of now we are not able to get some information that this plugin requires about the guest machine before it is actually up and running.

Please keep in mind that this plugin won't do magic, if you are compiling things during provisioning or manually downloading packages that does not fit into a "cache bucket" you won't see that much of improvement.

Benchmarks / shameless plug

Please have a look at this blog post for the numbers I've got down here.

Configurations

Auto detect supported cache buckets

As described on the usage section above, you can enable automatic detection of supported cache "buckets" by adding the code below to your Vagrantfile:

Vagrant.configure("2") do |config|
  # ...
  config.cache.auto_detect = true
end

This will make vagrant-cachier do its best to find out what is supported on the guest machine and will set buckets accordingly.

Cache scope

By default downloaded packages will get stored on a folder scoped to base boxes under your $HOME/.vagrant.d/cache. The idea is to leverage the cache by allowing downloaded packages to be reused across projects. So, if your Vagrantfile has something like:

Vagrant.configure("2") do |config|
  config.vm.box = 'some-box'
end

The cached files will be stored under $HOME/.vagrant.d/cache/some-box.

If you are on a multi VM environment, there is a huge chance that you'll end up having issues by sharing the same bucket across different machines. For example, if you apt-get install from two machines at "almost the same time" you are probably going to hit a SystemError: Failed to lock /var/cache/apt/archives/lock. To work around that, you can set the scope to be based on machines:

Vagrant.configure("2") do |config|
  config.vm.box = 'some-box'
  config.cache.scope = :machine
end

This will tell vagrant-cachier to download packages to .vagrant/machines/<machine-name>/cache on your current project directory.

Available cache "buckets"

System package managers

APT

Vagrant.configure("2") do |config|
  config.vm.box = 'some-debian-box'
  config.cache.enable :apt
end

Used by Debian-like Linux distros, will get configured under guest's /var/cache/apt/archives.

Please note that to avoid re-downloading packages, you should avoid apt-get clean as much as possible in order to make a better use of the cache, even if you are packaging a box

Zypper

Vagrant.configure("2") do |config|
  config.vm.box = 'some-suse-box'
  config.cache.enable :zypper
end

Used by SuSE guests, will get configured under guest's /var/cache/zypp/packages. It will also make sure that keep-packages is enabled for all repositories.

Yum

Vagrant.configure("2") do |config|
  config.vm.box = 'some-centos-box'
  config.cache.enable :yum
end

Used by CentOS guests, will get configured under guest's /var/cache/yum. It will also make sure that keepcache is set to 1 on guest's /etc/yum.conf.

Pacman

Vagrant.configure("2") do |config|
  config.vm.box = 'some-arch-linux-box'
  config.cache.enable :pacman
end

Used by Arch Linux, will get configured under guest's /var/cache/pacman/pkg.

Chef

Vagrant.configure("2") do |config|
  config.vm.box = 'some-box-using-chef-provisioner'
  config.cache.enable :chef
end

When a Chef provisioner is detected, this bucket caches the default file_cache_path directory, /var/chef/cache. Requires Vagrant 1.2.4+.

RubyGems

Vagrant.configure("2") do |config|
  config.vm.box = 'some-box-with-ruby-installed'
  config.cache.enable :gem
end

Compatible with probably with any type of guest distro, will hook into the cache folder under the result of running gem env gemdir as the default SSH user (usualy vagrant) on your guest. If you use rbenv / rvm on the guest machine, make sure it is already installed before enabling the bucket, otherwise you won't benefit from this plugin.

RVM

Vagrant.configure("2") do |config|
  config.vm.box = 'some-box-with-rvm-installed'
  config.cache.enable :rvm
end

Compatible with probably with any type of linux guest distro, will hook into the cache folder under the result of running rvm info as the default SSH user (usualy vagrant) on your guest. If you use rvm on the guest machine, make sure it is already installed before enabling the bucket, otherwise you won't benefit from this plugin.

npm

Vagrant.configure("2") do |config|
  config.vm.box = 'some-box-with-nodejs-installed'
  config.cache.enable :npm
end

Compatible with probably any type of linux guest distro, will hook into npm's cache directory under the result of running npm config get cache as the default SSH user (usually vagrant) on your guest. If you use nvm / n on the guest machine, make sure it is already installed before enabling the bucket, otherwise you won't benefit from this plugin.

Composer

Vagrant.configure("2") do |config|
  config.vm.box = 'some-box-with-php-installed'
  config.cache.enable :composer
end

Compatible with probably any type of linux guest distro, will cache guests' $HOME/.composer if PHP is detected.

APT-CACHER

Vagrant.configure("2") do |config|
  config.vm.box = 'some-debian-box'
  config.cache.enable :apt_cacher
end

This is useful, if you are using containers inside your VMs, e.g VirtualBox -> LXC. This would allow you to reuse packages without sharing folder inside VirtualBox. Only works with NFS-shared folders (since vboxsf is enforcing vagrant-user and apt-cacher is running under apt-cacher-ng user)

# install apt-cacher on (Host)-VM
$ sudo apt-get install apt-cacher-ng

# get the IP for eth0 interface
$ ifconfig eth0 |grep "inet addr"|awk '{print $2}' |cut -c6-20

# configure mirror on for your docker/LXC instances:
$ echo 'Acquire::http { Proxy "http://X.X.X.X:3142"; };' > /etc/apt/apt.conf.d/10mirror

# check, if working by tailing log on (Host)-VM, while installing packages on (Guest)-VMs
$ tail -f /var/log/apt-cacher-ng/apt-cacher.log

Used by Debian-like Linux distros, will get configured under guest's /var/cache/apt-cacher-ng.

Finding out disk space used by buckets

TODO

$ vagrant cache stats

Cleaning up cache buckets