{"id":1437,"date":"2012-12-12T01:37:39","date_gmt":"2012-12-12T08:37:39","guid":{"rendered":"http:\/\/xiehang.com\/blog\/?p=1437"},"modified":"2014-01-28T11:08:54","modified_gmt":"2014-01-28T18:08:54","slug":"how-to-chroot","status":"publish","type":"post","link":"https:\/\/xiehang.com\/blog\/2012\/12\/12\/how-to-chroot\/","title":{"rendered":"How to chroot"},"content":{"rendered":"

My colleague setup a jenkins to do nightly and on demand build, it works perfect till last week, whenever we decided to move some components to rpm based packaging so to ease the production deployment and to reproduce the bug.<\/p>\n

The problem is, whenever you have multiple jobs in jenkins, you cannot yum\/rpm install packages in different versions. A simple case for us is that we are running boost-1.44, while some team who replies on us requested a new version of boost, we cannot build two set of jobs at same time.<\/p>\n

So I came up with a idea of setting something that can islolate jenkins jobs. Actually I was thinking of OpenVZ at very beginning, but later on I figured out it may be too complicate to us at least at this stage. I’m still keeping an eye on OpenVZ, just in case we meet some chroot-cannot-solve problem I may give it a try.<\/p>\n

Back to original topic – I need to have a way to setup an isolated environment for jenkins, it should be easy to create\/destroy, while use all standard (CentOS) tools to manage packages. I guess what I did works in Debin\/Ubuntu as well – you just need to replace all yum with apt, and rpm with dpkg \ud83d\ude1b<\/p>\n

First let’s create a snapshot of CentOS image first, you don’t have to create the image if you don’t care of the speed and network traffic too much, what’s being done here can put into jenkins jobs but in my case the speed is about 10 times faster by using a snapshot, and definitely it reduce the network data flow (my image has minimum set of development packages, even it installed limited packages, it still need to pull ~300M data from CentOS repo, but sure if you have setup your own repo, that’s another story).<\/p>\n

You should have some common sense of replacing versions\/locations below to meet your need, and of course all commands need to be run as root (sudo please):<\/p>\n

    \n
  1. create a directory to be the chroot’s root, let’s assume it is in $ROOT_DIR\n
    mkdir -p $ROOT_DIR<\/pre>\n<\/li>\n
  2. bookstrap yum, you can choose other versions as you like (check 5on6 sections below)\n
    mkdir -vp $ROOT_DIR\/var\/lib\/rpm\r\nrpm -v --rebuilddb --root=$ROOT_DIR\r\nrpm -Uhv --root=$ROOT_DIR --nodeps http:\/\/mirror.centos.org\/centos\/6\/os\/x86_64\/Packages\/centos-release-6-3.el6.centos.9.x86_64.rpm\r\nyum --installroot=$ROOT_DIR install -y yum<\/pre>\n<\/li>\n
  3. do other necessary configurations – this is to avoid warning\/errors during yum in step #4\n
    cp -v \/var\/lib\/random-seed $ROOT_DIR\/var\/lib\/random-seed\r\ntouch $ROOT_DIR\/etc\/fstab\r\ncp -v \/etc\/resolv.conf $ROOT_DIR\/etc\/resolv.conf<\/pre>\n<\/li>\n
  4. this is the image for jenkins, so most likely you need all dev tools\n
    yum --installroot=$ROOT_DIR groupinstall -y \"Development tools\"<\/pre>\n<\/li>\n
  5. Most likely you need to copy your repo file to $ROOT_DIR\/etc\/yum.repo.d\/<\/li>\n
  6. Some file systems need to be mounted (check comment #1 to see a better solution for this<\/a><\/strong><\/em>)\n
    mount -o bind \/proc $ROOT_DIR\/proc\r\nmount -o bind \/dev $ROOT_DIR\/dev<\/pre>\n<\/li>\n
  7. Now you can step into the root and do some other stuffs you want, to me the most important thing is clean up yum cache so that everytime jenkins will pull the most recent package (for our own repo only) instead of using cache\n
    chroot $ROOT_DIR\r\n[in chroot now] yum clean all\r\n[in chroot now] exit<\/pre>\n<\/li>\n
  8. unmount what we mounted before\n
    umount $ROOT_DIR\/proc\r\numount $ROOT_DIR\/dev<\/pre>\n<\/li>\n
  9. take an snapshot\n
    cd $ROOT_DIR\r\ntar zcf \/tmp\/centos6-20121212-01.tgz *<\/pre>\n<\/li>\n
  10. now we have a snapshot under \/tmp, named centos6-20121212-01.tgz
    \nhey, it’s 12-12-12 now, nice<\/strong><\/li>\n<\/ol>\n
    Once you have the snapshot, it becomes pretty easy to use it in jenkins, you have to solve sudo problem first, which allows jenkins user (either jenkins, or apache, or nobody, or tomcat) sudo to run the build.sh without password (ping me if you don’t know how to), and here are some codes copied from my build.sh:<\/p>\n
    mkdir -vp $ROOT_DIR\r\ntar zxf \/tmp\/centos6-20121212-01.tgz -C $ROOT_DIR\r\nmount -vo bind \/proc $ROOT_DIR\/proc\r\nmount -vo bind \/dev $ROOT_DIR\/dev\r\nmount -vo bind \/dev\/pts $ROOT_DIR\/dev\/pts\r\nmount -vo bind \/sys $ROOT_DIR\/sys\r\nchroot $ROOT_DIR yum install -y home-grown-packages\r\ncat > $ROOT_DIR\/tmp\/build.sh << EOF\r\n\/bin\/mkdir -p \/tmp\/build\r\ncd \/tmp\/build\r\nmkdir -vp BUILD BUILDROOT RPMS SOURCES SPECS SRPMS\r\n\/usr\/bin\/svn co svn+ssh:\/\/blah-blah-blar\/foo\/bar\r\ncd \/tmp\/build\/BUILD\r\nrpmbuild --define '_topdir \/tmp\/build' -bb path\/to\/jenkins.spec\r\nEOF\r\nchmod +x $ROOT_DIR\/tmp\/build.sh\r\nchroot --userspec=nobody $ROOT_DIR \/tmp\/build.sh\r\n# some more operations here, copy the final rpm to central repository, copy the test result back to jenkins workspace, etc\r\numount -v $ROOT_DIR\/sys\r\numount -v $ROOT_DIR\/dev\/pts\r\numount -v $ROOT_DIR\/dev\r\numount -v $ROOT_DIR\/proc\r\nrm -fr $ROOT_DIR<\/pre>\n
    Pretty much the commands explained what they do \u2026<\/p>\n
    There are something more to come soon, people are asking if build CentOS5 packages over CentOS6 is possible or not – theortically it should work, but I haven’t tested and I’m pretty sure there will be some troubles here and there, I will give a shoot tomorrow.<\/p>\n
    Will post more here in case I find anything interesting.<\/p>\n","protected":false},"excerpt":{"rendered":"
    My colleague setup a jenkins to do nightly and on demand build, it works perfect till last week, whenever we decided to move some components to rpm based packaging so to ease the production deployment and to reproduce the bug. The problem is, whenever you have multiple jobs in jenkins, you cannot yum\/rpm install packages […]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[373,372,374],"_links":{"self":[{"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/posts\/1437"}],"collection":[{"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/comments?post=1437"}],"version-history":[{"count":9,"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/posts\/1437\/revisions"}],"predecessor-version":[{"id":1654,"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/posts\/1437\/revisions\/1654"}],"wp:attachment":[{"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/media?parent=1437"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/categories?post=1437"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xiehang.com\/blog\/wp-json\/wp\/v2\/tags?post=1437"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}