In a previous life, I had built and deployed a partial yum mirror to speed up, and simplify, kick starting and deployment of supporting packages for an application stack. since there were multiple upstream yum repos that needed to be consulted (centos base, couchbase, epel, pgsql, etc), it made sense to put everything in one place. the kick start script then reference the internal repo, and everything deployed with no need to go out on the internet.
in my current life, this came up again, but i could not find any of the reference documentation regarding how to set up a configurable repo mirror.
the key is how yum (and reposync) parse their configuration files, and how they react to includePkgs/excludePkgs.
if you define includepkgs, only those in the list will be mirrored. this is normally a wild card list on a single line, but if you start the value with a newline/tab, it will be treated as a pseudo-here-doc.
for now, we’ll setup two upstream yum repos (base and epel6) to sync into a single local mirror.
lets get started:
rpm -ihv http://ftp.osuosl.org/pub/fedora-epel/6/i386/epel-release-6-8.noarch.rpm
yum install -y yum-utils createrepo
mkdir repos; cd repos
touch reposync.conf
touch base-pkglist.conf
touch epel-pkglist.conf
yum utils includes reposync, which will do most of the heavy lifting.
the reposync.conf:
[main]
gpgcheck=1
reposdir=/dev/null
[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
includepkgs=
include=base-pkglist.conf
[epel6]
name=Extra Packages for Enterprise Linux 6 - $basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
includepkgs=
include=epel6-pkglist.conf
note the reposdir=/dev/null; this will default to /etc/yum.repos.d/ if not specified, and that is not what we want.
the base-pkglist.conf can be generated from the existing rpm install list:
rpm -qa --qf '\t%{NAME}.%{ARCH}\n' | sort | head > base-pkglist.conf
only using the first 10 entries for brevitiy. the leading tab is important.
# cat base-pkglist.conf
acl.x86_64
alsa-lib.x86_64
apr-util.x86_64
apr.x86_64
atk.x86_64
attr.x86_64
audit-libs.x86_64
audit.x86_64
authconfig.x86_64
autoconf.noarch
remainder of pkglists should be generated by hand; one package per line, again prefixed with a \t. for this run, we’re just going to bring in ansible.
# cat epel6-pkglist.conf
ansible
one the files have been configured, run reposync to update:
# reposync -c reposync.conf
[base: 1 of 10 ] Downloading Packages/acl-2.2.49-6.el6.x86_64.rpm
acl-2.2.49-6.el6.x86_64.rpm
[base: 2 of 10 ] Downloading Packages/alsa-lib-1.0.22-3.el6.x86_64.rpm
alsa-lib-1.0.22-3.el6.x86_64.rpm
[base: 3 of 10 ] Downloading Packages/apr-1.3.9-5.el6_2.x86_64.rpm
apr-1.3.9-5.el6_2.x86_64.rpm
[base: 4 of 10 ] Downloading Packages/apr-util-1.3.9-3.el6_0.1.x86_64.rpm
apr-util-1.3.9-3.el6_0.1.x86_64.rpm
[base: 5 of 10 ] Downloading Packages/atk-1.30.0-1.el6.x86_64.rpm
atk-1.30.0-1.el6.x86_64.rpm
[base: 6 of 10 ] Downloading Packages/attr-2.4.44-7.el6.x86_64.rpm
attr-2.4.44-7.el6.x86_64.rpm
[base: 7 of 10 ] Downloading Packages/audit-2.3.7-5.el6.x86_64.rpm
audit-2.3.7-5.el6.x86_64.rpm
[base: 8 of 10 ] Downloading Packages/audit-libs-2.3.7-5.el6.x86_64.rpm
audit-libs-2.3.7-5.el6.x86_64.rpm
[base: 9 of 10 ] Downloading Packages/authconfig-6.1.12-19.el6.x86_64.rpm
authconfig-6.1.12-19.el6.x86_64.rpm
[base: 10 of 10 ] Downloading Packages/autoconf-2.63-5.1.el6.noarch.rpm
autoconf-2.63-5.1.el6.noarch.rpm
[epel6: 1 of 1 ] Downloading ansible-1.7.2-2.el6.noarch.rpm
ansible-1.7.2-2.el6.noarch.rpm
and finally, createrepo to update the meta data.
# createrepo .
Spawning worker 0 with 11 pkgs
Workers Finished
Gathering worker results
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
serve up via your httpd of choice.