glenux/vagrant-cachier-ng
glenux/vagrant-cachier-ng
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Look ma, first public release and no specs!

Browse source
This commit is contained in:
Fabio Rehm 2013-05-22 19:38:26 -03:00
commit 35515ed243
23 changed files with 796 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
.gitignore vendored Normal file
View file

@ -0,0 +1,17 @@
*.gem
*.rbc
.bundle
.config
.yardoc
InstalledFiles
_yardoc
coverage
doc/
lib/bundler/man
pkg
rdoc
spec/reports
test/tmp
test/version_tmp
tmp
.vagrant

1
CHANGELOG.md Normal file
View file

@ -0,0 +1 @@
# 0.0.2 Initial release

10
Gemfile Normal file
View file

@ -0,0 +1,10 @@
source 'https://rubygems.org'
# Specify your gem's dependencies in vagrant-cachier.gemspec
gemspec
group :development do
gem 'vagrant', github: 'mitchellh/vagrant'
gem 'vagrant-lxc', github: 'fgrehm/vagrant-lxc'
gem 'rake'
end

47
Gemfile.lock Normal file
View file

@ -0,0 +1,47 @@
GIT
remote: git://github.com/fgrehm/vagrant-lxc.git
revision: 2f88a060c911c466d304768dd349708336f8af1c
specs:
vagrant-lxc (0.3.4)
GIT
remote: git://github.com/mitchellh/vagrant.git
revision: ccfd321ef98dc5c12b180cc3a26f12d870c0eff5
specs:
vagrant (1.2.3.dev)
childprocess (~> 0.3.7)
erubis (~> 2.7.0)
i18n (~> 0.6.0)
json (>= 1.5.1, < 1.8.0)
log4r (~> 1.1.9)
net-scp (~> 1.1.0)
net-ssh (~> 2.6.6)
PATH
remote: .
specs:
vagrant-cachier (0.0.6)
GEM
remote: https://rubygems.org/
specs:
childprocess (0.3.9)
ffi (~> 1.0, >= 1.0.11)
erubis (2.7.0)
ffi (1.8.1)
i18n (0.6.4)
json (1.7.7)
log4r (1.1.10)
net-scp (1.1.0)
net-ssh (>= 2.6.5)
net-ssh (2.6.7)
rake (10.0.4)
PLATFORMS
ruby
DEPENDENCIES
rake
vagrant!
vagrant-cachier!
vagrant-lxc!

22
LICENSE.txt Normal file
View file

@ -0,0 +1,22 @@
Copyright (c) 2013 Fabio Rehm
MIT License
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

203
README.md Normal file
View file

@ -0,0 +1,203 @@
# vagrant-cachier
A [Vagrant](http://www.vagrantup.com/) plugin that helps you reduce the amount of
coffee you drink while waiting for boxes to be provisioned by sharing a common
package cache among similiar VM instances. Kinda like [vagrant-apt_cache](https://github.com/avit/vagrant-apt_cache)
or [this magical snippet](http://gist.github.com/juanje/3797297) but targetting
multiple package managers and Linux distros.
## Installation
Make sure you have Vagrant 1.2+ and run:
```
vagrant plugin install vagrant-cachier
```
## Usage
The easiest way to set things up is just to enable [cache buckets auto detection](#auto-detect-supported-cache-buckets)
from within your `Vagrantfile`:
```ruby
Vagrant.configure("2") do |config|
config.vm.box = 'your-box'
config.cache.auto_detect = true
end
```
For more information about available buckets, please see the [configuration section](#configurations) below.
## Compatible providers
* Vagrant's built in VirtualBox provider
* [vagrant-lxc](https://github.com/fgrehm/vagrant-lxc)
_It is possibly compatible with the [VMware providers](http://www.vagrantup.com/vmware)
as well but I haven't tried yet._
## How does it work?
Right now the plugin does not make any assumptions for you and you have to
configure things properly from your `Vagrantfile`. Please have a look at
the [available cache buckets](#available-cache-buckets) section below for more
information.
Under the hood, the plugin will hook into calls to `Vagrant::Builtin::Provision`
during `vagrant up` / `vagrant reload` and will set things up for each configured
cache bucket. Before halting the machine, it will revert the changes required
to set things up by hooking into calls to `Vagrant::Builtin::GracefulHalt` so
that you can repackage the machine for others to use without requiring users to
install the plugin as well.
Cache buckets will be available from `/tmp/vagrant-cachier` on your guest and
the appropriate folders will get symlinked to the right path _after_ the machine is
up but _right before_ it gets provisioned. We _could_ potentially do it on one go
and share bucket's folders directly to the right path if we were only using VirtualBox
since it shares folders _after_ booting the machine, but the LXC provider does that
_as part of_ the boot process (shared folders are actually `lxc-start` parameters)
and as of now we are not able to get some information that this plugin requires
about the guest machine before it is actually up and running.
Please keep in mind that this plugin won't do magic, if you are compiling things
during provisioning or manually downloading packages that does not fit into a
"cache bucket" you won't see that much of improvement.
## Configurations
### Auto detect supported cache buckets
As described on the usage section above, you can enable automatic detection of
supported [cache "buckets"](#available-cache-buckets) by adding the code below to
your `Vagrantfile`:
```ruby
Vagrant.configure("2") do |config|
# ...
config.cache.auto_detect = true
end
```
This will make vagrant-cachier do its best to find out what is supported on the
guest machine and will set buckets accordingly.
### Cache scope
By default downloaded packages will get stored on a folder scoped to base boxes
under your `$HOME/.vagrant.d/cache`. The idea is to leverage the cache by allowing
downloaded packages to be reused across projects. So, if your `Vagrantfile` has
something like:
```ruby
Vagrant.configure("2") do |config|
config.vm.box = 'some-box'
end
```
The cached files will be stored under `$HOME/.vagrant.d/cache/some-box`.
If you are on a [multi VM environment](http://docs.vagrantup.com/v2/multi-machine/index.html),
there is a huge chance that you'll end up having issues by sharing the same bucket
across different machines. For example, if you `apt-get install` from two machines
at "almost the same time" you are probably going to hit a `SystemError: Failed to lock /var/cache/apt/archives/lock`.
To work around that, you can set the scope to be based on machines:
```ruby
Vagrant.configure("2") do |config|
config.vm.box = 'some-box'
config.cache.scope = :machine
end
```
This will tell vagrant-cachier to download packages to `.vagrant/machines/<machine-name>/<provider-name>/cache`
on your current project directory.
### Available cache "buckets"
#### System package managers
##### APT
```ruby
Vagrant.configure("2") do |config|
config.vm.box = 'some-debian-box'
config.cache.enable :apt
end
```
Used by Debian-like Linux distros, will get configured under guest's `/var/cache/apt/archives`.
_Please note that to avoid re-downloading packages, you should avoid `apt-get clean`
as much as possible in order to make a better use of the cache, even if you are
packaging a box_
##### Yum
```ruby
Vagrant.configure("2") do |config|
config.vm.box = 'some-centos-box'
config.cache.enable :yum
end
```
Used by CentOS guests, will get configured under guest's `/var/cache/yum`. It will
also [make sure](lib/vagrant-cachier/bucket/yum.rb#L20) that `keepcache` is set to
`1` on guest's `/etc/yum.conf`.
##### Pacman
```ruby
Vagrant.configure("2") do |config|
config.vm.box = 'some-arch-linux-box'
config.cache.enable :pacman
end
```
Used by Arch Linux, will get configured under guest's `/var/cache/pacman/pkg`.
#### RubyGems
```ruby
Vagrant.configure("2") do |config|
config.vm.box = 'some-box-with-ruby-installed'
config.cache.enable :gem
end
```
Compatible with probably with any type of guest distro, will hook into the `cache`
folder under the result of running `gem env gemdir` as the default SSH user (usualy
`vagrant`) on your guest. If you use rbenv / rvm on the guest machine, make sure
it is already installed before enabling the bucket, otherwise you won't benefit
from this plugin.
## Finding out disk space used by buckets
_TODO_
```shell
$ vagrant cache stats
```
## Cleaning up cache buckets
_TODO_
```shell
$ vagrant cache clean apt
```
## Contributing
1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request

5
Rakefile Normal file
View file

@ -0,0 +1,5 @@
Dir['./tasks/**/*.rake'].each { |f| load f }
require 'bundler/gem_tasks'
task :ci => ['spec:unit']

52
development/Vagrantfile vendored Normal file
View file

@ -0,0 +1,52 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.require_plugin 'vagrant-cachier'
Vagrant.require_plugin 'vagrant-lxc'
Vagrant.configure("2") do |config|
config.vm.synced_folder "../", "/vagrant", id: 'vagrant-root'#, nfs: true
config.cache.scope = :machine
config.cache.auto_detect = true
debian_like_configs = lambda do |debian|
debian.vm.provision :shell, inline: '
if ! (which bundle > /dev/null); then
sudo gem install bundler --no-ri --no-rdoc
sudo apt-get install -y build-essential libffi-dev ruby1.9.1-dev git
cd /vagrant && bundle
fi'
end
config.vm.define :ubuntu do |ubuntu|
ubuntu.vm.box = "quantal64"
debian_like_configs.call ubuntu
end
config.vm.define :debian do |debian|
debian.vm.box = "squeeze64"
debian.vm.box_url = 'http://f.willianfernandes.com.br/vagrant-boxes/DebianSqueeze64.box'
debian_like_configs.call debian
end
config.vm.define :centos do |centos|
centos.vm.box = 'centos6_64'
centos.vm.box_url = 'http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.4-x86_64-v20130309.box'
centos.vm.provision :shell, inline: '
if ! (which bundle > /dev/null); then
time sudo gem install bundler --no-ri --no-rdoc
time sudo yum install -y libffi-devel ruby-devel git
fi'
end
config.vm.define :arch do |arch|
arch.vm.box = 'arch64'
arch.vm.box_url = 'http://vagrant.pouss.in/archlinux_2012-07-02.box'
arch.vm.provision :shell, inline: '
if ! (which bundle > /dev/null); then
time sudo gem install bundler --no-ri --no-rdoc
time sudo pacman -Syu --noconfirm libffi git
fi'
end
end

4
lib/vagrant-cachier.rb Normal file
View file

@ -0,0 +1,4 @@
require 'vagrant'
require "vagrant-cachier/version"
require "vagrant-cachier/plugin"

89
lib/vagrant-cachier/action.rb Normal file
View file

@ -0,0 +1,89 @@
require_relative 'bucket'
module Vagrant
module Cachier
class Action
class Install
def initialize(app, env)
@app = app
@logger = Log4r::Logger.new("vagrant::cachier::action::install")
end
def call(env)
return @app.call(env) unless env[:machine].config.cache.enabled?
@env = env
FileUtils.mkdir_p(cache_root.to_s) unless cache_root.exist?
env[:machine].config.vm.synced_folder cache_root, '/tmp/vagrant-cache', id: "vagrant-cache"
@app.call env
env[:cache_dirs] = []
if env[:machine].config.cache.auto_detect
Bucket.auto_detect(env)
end
if env[:machine].config.cache.buckets.any?
env[:ui].info 'Configuring cache buckets...'
cache_config = env[:machine].config.cache
cache_config.buckets.each do |bucket_name, configs|
@logger.debug "Installing #{bucket_name} with configs #{configs.inspect}"
Bucket.install(bucket_name, env, configs)
end
data_file = env[:machine].data_dir.join('cache_dirs')
data_file.open('w') { |f| f.print env[:cache_dirs].join("\n") }
end
end
def cache_root
@cache_root ||= case @env[:machine].config.cache.scope
when :box
@env[:home_path].join('cache', @env[:machine].box.name)
when :machine
@env[:machine].data_dir.join('cache')
else
raise "Unknown cache scope: '#{@env[:machine].config.cache.scope}'"
end
end
end
class Clean
def initialize(app, env)
@app = app
@logger = Log4r::Logger.new("vagrant::cachier::action::clean")
end
def call(env)
@env = env
if env[:machine].state.id == :running && symlinks.any?
env[:ui].info 'Removing cache buckets symlinks...'
symlinks.each do |symlink|
remove_symlink symlink
end
File.delete env[:machine].data_dir.join('cache_dirs').to_s
end
@app.call env
end
def symlinks
# TODO: Check if file exists instead of a blank rescue
@symlinks ||= @env[:machine].data_dir.join('cache_dirs').read.split rescue []
end
def remove_symlink(symlink)
if @env[:machine].communicate.test("test -L #{symlink}")
@logger.debug "Removing symlink for '#{symlink}'"
@env[:machine].communicate.sudo("unlink #{symlink}")
end
end
end
end
end
end

39
lib/vagrant-cachier/bucket.rb Normal file
View file

@ -0,0 +1,39 @@
module Vagrant
module Cachier
class Bucket
def self.inherited(base)
@buckets ||= []
@buckets << base
end
def self.auto_detect(env)
@buckets.each do |bucket|
if env[:machine].guest.capability?(bucket.capability)
env[:machine].config.cache.enable bucket.bucket_name
end
end
end
def self.bucket_name
# TODO: Handle MultiWord bucket classes
self.name.split('::').last.downcase
end
def self.install(name, env, configs)
bucket = const_get(name.to_s.capitalize)
bucket.new(name, env, configs).install
end
def initialize(name, env, configs)
@name = name
@env = env
@configs = configs
end
end
end
end
require_relative "bucket/apt"
require_relative "bucket/gem"
require_relative "bucket/pacman"
require_relative "bucket/yum"

34
lib/vagrant-cachier/bucket/apt.rb Normal file
View file

@ -0,0 +1,34 @@
module Vagrant
module Cachier
class Bucket
class Apt < Bucket
def self.capability
:apt_cache_dir
end
def install
machine = @env[:machine]
guest = machine.guest
if guest.capability?(:apt_cache_dir)
guest_path = guest.capability(:apt_cache_dir)
@env[:cache_dirs] << guest_path
machine.communicate.tap do |comm|
comm.execute("mkdir -p /tmp/vagrant-cache/#{@name}")
unless comm.test("test -L #{guest_path}")
comm.sudo("rm -rf #{guest_path}")
comm.sudo("mkdir -p `dirname #{guest_path}`")
comm.sudo("ln -s /tmp/vagrant-cache/#{@name} #{guest_path}")
end
end
else
# TODO: Raise a better error
raise "You've configured an APT cache for a guest machine that does not support it!"
end
end
end
end
end
end

39
lib/vagrant-cachier/bucket/gem.rb Normal file
View file

@ -0,0 +1,39 @@
module Vagrant
module Cachier
class Bucket
class Gem < Bucket
def self.capability
:gemdir
end
def install
machine = @env[:machine]
guest = machine.guest
if guest.capability?(:gemdir)
if gemdir_path = guest.capability(:gemdir)
prefix = gemdir_path.split('/').last
bucket_path = "/tmp/vagrant-cache/#{@name}/#{prefix}"
machine.communicate.tap do |comm|
comm.execute("mkdir -p #{bucket_path}")
gem_cache_path = "#{gemdir_path}/cache"
@env[:cache_dirs] << gem_cache_path
unless comm.test("test -L #{gem_cache_path}")
comm.sudo("rm -rf #{gem_cache_path}")
comm.sudo("mkdir -p `dirname #{gem_cache_path}`")
comm.sudo("ln -s #{bucket_path} #{gem_cache_path}")
end
end
end
else
# TODO: Raise a better error
raise "You've configured a RubyGems cache for a guest machine that does not support it!"
end
end
end
end
end
end

34
lib/vagrant-cachier/bucket/pacman.rb Normal file
View file

@ -0,0 +1,34 @@
module Vagrant
module Cachier
class Bucket
class Pacman < Bucket
def self.capability
:pacman_cache_dir
end
def install
machine = @env[:machine]
guest = machine.guest
if guest.capability?(:pacman_cache_dir)
guest_path = guest.capability(:pacman_cache_dir)
@env[:cache_dirs] << guest_path
machine.communicate.tap do |comm|
comm.execute("mkdir -p /tmp/vagrant-cache/#{@name}")
unless comm.test("test -L #{guest_path}")
comm.sudo("rm -rf #{guest_path}")
comm.sudo("mkdir -p `dirname #{guest_path}`")
comm.sudo("ln -s /tmp/vagrant-cache/#{@name} #{guest_path}")
end
end
else
# TODO: Raise a better error
raise "You've configured a Pacman cache for a guest machine that does not support it!"
end
end
end
end
end
end

37
lib/vagrant-cachier/bucket/yum.rb Normal file
View file

@ -0,0 +1,37 @@
module Vagrant
module Cachier
class Bucket
class Yum < Bucket
def self.capability
:yum_cache_dir
end
def install
machine = @env[:machine]
guest = machine.guest
if guest.capability?(:yum_cache_dir)
guest_path = guest.capability(:yum_cache_dir)
@env[:cache_dirs] << guest_path
machine.communicate.tap do |comm|
# Ensure caching is enabled
comm.sudo("sed -i 's/keepcache=0/keepcache=1/g' /etc/yum.conf")
comm.execute("mkdir -p /tmp/vagrant-cache/#{@name}")
unless comm.test("test -L #{guest_path}")
comm.sudo("rm -rf #{guest_path}")
comm.sudo("mkdir -p `dirname #{guest_path}`")
comm.sudo("ln -s /tmp/vagrant-cache/#{@name} #{guest_path}")
end
end
else
# TODO: Raise a better error
raise "You've configured a Yum cache for a guest machine that does not support it!"
end
end
end
end
end
end

14
lib/vagrant-cachier/cap/arch/pacman_cache_dir.rb Normal file
View file

@ -0,0 +1,14 @@
module Vagrant
module Cachier
module Cap
module Arch
module PacmanCacheDir
def self.pacman_cache_dir(machine)
# TODO: Find out if there is a config file we can read from
'/var/cache/pacman/pkg'
end
end
end
end
end
end

14
lib/vagrant-cachier/cap/debian/apt_cache_dir.rb Normal file
View file

@ -0,0 +1,14 @@
module Vagrant
module Cachier
module Cap
module Debian
module AptCacheDir
def self.apt_cache_dir(machine)
# TODO: Find out if there is a config file we can read from
'/var/cache/apt/archives'
end
end
end
end
end
end

20
lib/vagrant-cachier/cap/linux/gemdir.rb Normal file
View file

@ -0,0 +1,20 @@
module Vagrant
module Cachier
module Cap
module Linux
module Gemdir
def self.gemdir(machine)
gemdir = nil
machine.communicate.tap do |comm|
return unless comm.test('which gem')
comm.execute 'gem env gemdir' do |buffer, output|
gemdir = output.chomp if buffer == :stdout
end
end
return gemdir
end
end
end
end
end
end

14
lib/vagrant-cachier/cap/redhat/yum_cache_dir.rb Normal file
View file

@ -0,0 +1,14 @@
module Vagrant
module Cachier
module Cap
module RedHat
module YumCacheDir
def self.yum_cache_dir(machine)
# TODO: Find out if there is a config file we can read from
'/var/cache/yum'
end
end
end
end
end
end

30
lib/vagrant-cachier/config.rb Normal file
View file

@ -0,0 +1,30 @@
module Vagrant
module Cachier
class Config < Vagrant.plugin(2, :config)
attr_accessor :scope, :auto_detect
attr_reader :buckets
def initialize
@scope = UNSET_VALUE
@auto_detect = UNSET_VALUE
end
def enable(bucket, opts = {})
(@buckets ||= {})[bucket] = opts
end
def finalize!
return unless enabled?
@scope = :box if @scope == UNSET_VALUE
@auto_detect = false if @auto_detect == UNSET_VALUE
@buckets = @buckets ? @buckets.dup : {}
end
def enabled?
@enabled ||= @auto_detect != UNSET_VALUE ||
@buckets != nil
end
end
end
end

46
lib/vagrant-cachier/plugin.rb Normal file
View file

@ -0,0 +1,46 @@
module Vagrant
module Cachier
class Plugin < Vagrant.plugin('2')
name 'vagrant-cachier'
config 'cache' do
require_relative "config"
Config
end
guest_capability 'linux', 'gemdir' do
require_relative 'cap/linux/gemdir'
Cap::Linux::Gemdir
end
guest_capability 'debian', 'apt_cache_dir' do
require_relative 'cap/debian/apt_cache_dir'
Cap::Debian::AptCacheDir
end
guest_capability 'redhat', 'yum_cache_dir' do
require_relative 'cap/redhat/yum_cache_dir'
Cap::RedHat::YumCacheDir
end
guest_capability 'arch', 'pacman_cache_dir' do
require_relative 'cap/arch/pacman_cache_dir'
Cap::Arch::PacmanCacheDir
end
install_action_hook = lambda do |hook|
require_relative 'action'
hook.after Vagrant::Action::Builtin::Provision, Vagrant::Cachier::Action::Install
end
action_hook 'set-shared-cache-on-machine-up', :machine_action_up, &install_action_hook
action_hook 'set-shared-cache-on-machine-reload', :machine_action_reload, &install_action_hook
clean_action_hook = lambda do |hook|
require_relative 'action'
hook.before Vagrant::Action::Builtin::GracefulHalt, Vagrant::Cachier::Action::Clean
end
action_hook 'remove-guest-symlinks-on-machine-halt', :machine_action_halt, &clean_action_hook
action_hook 'remove-guest-symlinks-on-machine-package', :machine_action_package, &clean_action_hook
end
end
end

5
lib/vagrant-cachier/version.rb Normal file
View file

@ -0,0 +1,5 @@
module Vagrant
module Cachier
VERSION = "0.0.6"
end
end

20
vagrant-cachier.gemspec Normal file
View file

@ -0,0 +1,20 @@
# coding: utf-8
lib = File.expand_path('../lib', __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
require 'vagrant-cachier/version'
Gem::Specification.new do |spec|
spec.name = "vagrant-cachier"
spec.version = Vagrant::Cachier::VERSION
spec.authors = ["Fabio Rehm"]
spec.email = ["fgrehm@gmail.com"]
spec.description = %q{Speed up vagrant boxes provisioning}
spec.summary = spec.description
spec.homepage = "https://github.com/fgrehm/vagrant-cachier"
spec.license = "MIT"
spec.files = `git ls-files`.split($/)
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
spec.require_paths = ["lib"]
end