Lampros - Weird Bricks

Setting up a custom PKG repository

28 December, 2014 | FreeBSD

In my previous post I demonstrated how to compile Mongo from ports, build a package from the port and then copy that package to another server over SCP to install it.

That worked great and saved us tons of time but it'd be nicer if instead of doing the SCP we had our own custom repository to install packages from.

To do this we're going to need a web server - Apache and Nginx are kinda overkill for just serving some static file so I'm going to go ahead and give Mongoose a try.

1. Upgrade PKG

env ASSUME_ALWAYS_YES=YES pkg upgrade

2. Search for mongoose in the packages:

pkg search mongoose
mongoose-5.3

3. Install mongoose:

pkg install mongoose

Output:

Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
All repositories are up-to-date.
The following 1 packages will be affected (of 0 checked):

New packages to be INSTALLED:
        mongoose: 5.3

The process will require 111 KB more space.
38 KB to be downloaded.

Proceed with this action? [y/N]: y
Fetching mongoose-5.3.txz: 100%   38 KB  39.1k/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Installing mongoose-5.3...
[1/1] Extracting mongoose-5.3: 100%

38KB!!!! That's awesome and ridiculous!!

4. Create a directory for the packages:

mkdir /var/www

5. Edit /etc/rc.conf and set Mongoose to:

* Start on boot

* Listen on port 80

* Set the document root to /var/www

* Set the access log file to be in /var/log/mongoose.log

The lines you need are:

mongoose_enable="YES"
mongoose_flags="-document_root /var/www -listening_port 80 -access_log_file /var/log/mongoose.log"

6. Test if it works:

service mongoose start

Output:

Starting mongoose.

7. See if it listens on port 80:

sockstat -4

Output:

root     mongoose   2170  3  tcp4   *:80                  *:*
root     mongoose   2170  4  tcp4   127.0.0.1:31158       127.0.0.1:31887
root     mongoose   2170  5  tcp4   127.0.0.1:31887       127.0.0.1:31158

It sure is!

8. Following the steps from before copy the package you want to serve from the repository into the /var/www directory:

cp /root/mongodb-2.6.6.txz /var/www/

9. Now change directory into /var/www and create a package repository:

pkg repo .

Output:

Creating repository in .: 100%
Packing files for repository: 100%

If you now list the files of the /var/www directory you'll see that there are 3 new ones - those contain the repository information:

ls -al /var/www/

Output:

total 47572
drwxr-xr-x   2 root  wheel       512 Dec 28 14:03 .
drwxr-xr-x  26 root  wheel       512 Dec 28 13:54 ..
-rw-r--r--   1 root  wheel       276 Dec 28 14:03 digests.txz
-rw-r--r--   1 root  wheel       264 Dec 28 14:03 meta.txz
-rw-r--r--   1 root  wheel  48657740 Dec 26 03:19 mongodb-2.6.6.txz
-rw-r--r--   1 root  wheel       760 Dec 28 14:03 packagesite.txz

[h3]Now to test the repository - on a new server:[/h3]

1. Upgrade pkg:

env ASSUME_ALWAYS_YES=YES pkg upgrade

2. Edit /etc/pkg/FreeBSD.conf and add the repository configuration:

You'll of course have to change out 'WeirdBricks' with the name you want to give your repository and '104.245.33.109' with the IP address of your server

WeirdBricks: {
  url: "http://104.245.33.109/",
  mirror_type: "http",
  signature_type: "none",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

3. Run pkg update to update the FreeBSD repository catalogue:

pkg update

Output:

Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
Updating WeirdBricks repository catalogue...
Fetching meta.txz: 100%   264 B   0.3k/s    00:01
Fetching packagesite.txz: 100%   760 B   0.8k/s    00:01
Processing entries: 100%
WeirdBricks repository update completed. 1 packages processed

4. Search for mongodb:

pkg search mongodb

Output:

mongodb-2.6.6
p5-Mojolicious-Plugin-Mongodb-1.16_1
p5-MongoDB-0.702.2_1
py27-nagios-check_mongodb-0.1
mongodb-2.6.6

5. Install mongodb:

env ASSUME_ALWAYS_YES=YES pkg install mongodb

If you get this message:

Checking integrity...pkg: cannot load files from mongodb to check integrity

Don't worry about it, just run the pkg install mongodb command again:

[7/7] Installing mongodb-2.6.6...
===> Creating users and/or groups.
Creating group 'mongodb' with gid '922'.
Creating user 'mongodb' with uid '922'.
[7/7] Extracting mongodb-2.6.6: 100%
Message for python27-2.7.9:

Back on the pkg server, let's check the mongoose logs, I see multiple lines like these:

104.245.32.182 - - [28/Dec/2014:14:19:26 -0500] "GET /meta.txz HTTP/1.1" 304 0 - "pkg/1.4.1"
104.245.32.182 - - [28/Dec/2014:14:19:26 -0500] "GET / HTTP/1.1" 200 0 - "pkg/1.4.1"
104.245.32.182 - - [28/Dec/2014:14:19:26 -0500] "GET /packagesite.txz HTTP/1.1" 304 0 - "pkg/1.4.1"

And there you have it! We setup our own pkg repository in just a few quick steps.

Cesanta - Mongoose Configuration Options

FreeBSD Man Pages - pkg -- a utility for manipulating packages