Squid config with NCSA Authentication in Ubuntu 11.04

This article is deprecated. For the recent version check the following article: https://www.grosseosterhues.com/2011/12/squid-config-with-ncsa-authentication/

This article describes a quick and easy way to implement a web proxy with most basic user authentication. The authentication mechanism uses an NCSA-style htpasswd file, well known also for other basic authentication purposes. The proxy server in use will be Squid proxy.

To install the necessary software, this line will do:

aptitude install squid

Now, a new file containing usernames and password hashes needs to be created:

htpasswd -c /etc/squid/passwd USERNAME
chmod 400 /etc/squid/passwd && chown proxy /etc/squid/passwd

The file /etc/squid/squid.conf has to be modified. The following lines have to be uncommented while the first of them needs adjustment as to the whereabouts of the passwd file. By default it points to usr/etc/passwd but we want it to be /etc/squid/passwd:

auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/passwd
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic casesensitive off

In the ACL section two lines have to be inserted. The ACL section can be found by searching for “# ACCESS CONTROLS“.

acl ncsa_users proxy_auth REQUIRED
http_access allow ncsa_users

In case you want Squid to not reveal your client’s real IP address in HTTP requests it forwards, the follow option has to be disabled:

forwarded_for off 

A restart of Squid’s daemon will activate the recent changes:

service squid restart 

Exim 4 and ClamAV Malware Scanning in Ubuntu

Although virus scanners are installed on virtually all regular client systems nowadays, it’s nice to have the mailserver pre-scanning mails for malware. This article describes the configuration of ClamAV in conjunction with Exim 4 in Ubuntu Linux.

Assuming a working Exim 4 environment, the fist step is to install the necessary ClamAV package:

aptitude install clamav-daemon

To tell Exim which anti-virus scanner to use, the file /etc/exim4/exim4.conf.template needs to be customized. The default config already brings the necessary lines. They just need to be adjusted and uncommented:

av_scanner = clamd:/var/run/clamav/clamd.ctl
  malware = *
  message = This message was detected as possible malware ($malware_name).
  message = X-Virus-Scanned: (${readsocket{/var/run/clamav/clamd.ctl}{VERSION}{2s}{}{AV Not running}})

In order for ClamAV to access the mail spool it has to become a member of Exim’s group:

usermod -a -G Debian-exim clamav && service clamav-daemon restart

Now, sending a testmail like the one from EICAR should give you the appropriate malware rejection.

Greylisting in Exim 4 with Greylistd on Ubuntu Linux

In this article the approach of greylisting as a protection against spam in Exim 4 is being described. The concepts of greylisting temporarily rejects mails at SMTP time. A regular MTA without spamming intent will try to send mail again after a certain period of time. The average trojan or worm on the other hand usually sends spam in waves. It will try to send its load only once and won’t try again later. This concept facilitates spam reduction to a huge extent.

Setting up greylisting in Ubuntu Linux using Exim 4 is a breeze. To work, it simply needs the following lines.

Firstly, the installation of the package greylistd:

aptitude install greylistd

To activate the new package in Exim’s configuration, the following command will do some magic:

greylistd-setup-exim4 add

The default configuration of greylistd is a bit too paranoid from my point of view. Therefore I adjust /etc/greylistd/config as follows (usually the last two existing entries in that file):

singlecheck = true
singleupdate = true

This has the effect that a domain that has already been verified won’t become greylisted again if a different user sends from that particular domain. Also, it enables for manual manipulation of the whitelist by just adding IPs of “good” MTAs.

The whitelist file that is supposed to be edited by admins resides at /etc/greylistd/whitelist-hosts. The predifined whitelist can be found at /var/lib/greylistd/whitelist-hosts.

Configure Spamassassin for Exim 4 in Ubuntu

This article describes the configuration of the spam scanner called spamassassin in conjunction with the MTA called Exim 4 in Ubuntu Linux (as of writing Ubuntu 11.10 is the most recent). Assuming an otherwise working Exim 4 setup, this document describes just the essential steps to a working spamscanner.

First, the necessary packages need to be installed:

aptitude install exim4-daemon-heavy sa-exim spamassassin

In /etc/default/spamassassin set the following lines:


The first option lets the spamassassin daemon start on boot. The second one defines that spamassassin’s rules become updated each night using a cron job.

In order to check if the first steps went ok, it’s time to start spamassassin the very first time:

service spamassassin start

To check if spamassassin runs smoothly on it’s dedicated port, this command helps:

lsof -i :783

Now in /etc/exim4/exim4.conf.template, insert the following lines in MAIN CONFIGURATION SETTINGS:

local_scan_path = /usr/lib/exim4/local_scan/sa-exim.so

To enable Exim’s spamassassin connector, the file /etc/exim4/sa-exim.conf has to be modified. Comment the following line in that file:

# SAEximRunCond: 0

In default setups, debugging is enabled. That will lead to huge log files. So for production use, changing the following line to zero is advised:

SAEximDebug: 0

The updated file /etc/exim4/exim4.conf.template becomes active by running the following line:

update-exim4.conf && service exim4 restart

For a basic protection, the MTA is now good to go.

Disable Multiarch in Ubuntu Oneiric 11.10

Since Ubuntu 11.10 multiarch support becomes set by default in new systems as well as in upgraded ones. This sounds like a nice feature, but it breaks one of my favorite tools: aptitude. As of now, aptitude can’t handle multiple arches of the same package and therefore runs wild when told to do any task.

The solution to this is to disable multiarch for the time being. This is done by the following command:

mv /etc/dpkg/dpkg.cfg.d/multiarch /etc/dpkg/dpkg.cfg.d/multiarch.backup && apt-get update

In case aptitude learns how to handle multiarch some day, it can be re-enabled by running the command the other way round.

Move Cacti data from one platform to another one

In order to migrate Cacti’s rrd-tool data from one system to a different one, that data has to be converted to XML-format first. That is also true if you move from x86 to a x86_64.

To to so, first convert all rrd-data in /var/lib/cacti/rrd to XML using the following line in that folder:

for i in `find -name "*.rrd"`; do rrdtool dump $i > $i.xml; done

Now pack up the resulting XML-files:

tar czf cacti-xml.tar.gz /var/lib/cacti/rrd/*xml

Move the tar.gz-container to the new system and unpack its files in the same directory using:

tar xzf cacti-xml.tar.gz

The re-conversion of those XML-files into rrd format is done using this line:

for i in `find -name "*.xml"`; do rrdtool restore $i `echo $i |sed s/.xml//g`; done 

Resetting the MySQL root password the dirty way

It happens to me quite often that I forget the root password of a MySQL-setup. In that case, a dirty way to reset the root user’s password is to stop the database and run it with “–skip-grant-tables”. This enables anyone to connect without a password and with all privileges which is certainly not recommended for regular use. After changing the password, the database is supposed to be restarted with regular settings.

The following lines document the actual doing. Remember to replace NEW_PASSWORD in the last line with your new one:

service mysql stop
mysqld --skip-grant-tables &
mysql -u root mysql
ps ax|grep mysqld | awk '{print $1}'|xargs kill -9
service mysql start

Adding Google Contacts to Roundcube Webmail


This article describes the necessary steps to integrate one’s Google-based address book to the Roundcube web-based IMAP client.

For people who tend to keep their personal information like contacts data in one self-managed place, it must seem rather lame to store those data at Google’s. Yet, there are virtually no free open source address books out there that could do what Google offers: full bidirectional sync with basically all mobile devices and support for major protocols. Therefore, using Google as storage for contacts data seems appropriate for the time being. Also, the integration with Google’s Calendar works like a charm. But that’s for another article.

Download of Google Contacts plugin

The necessary plugin can be retrieved from the following website: http://sourceforge.net/projects/roundcubegoogle/files/. It is supposed to be stored in Roundcube’s plugins folder, where it is unpacked by:

tar xzf google_contacts-<version_numnber>.tar.gz

There is a config file alongside with the archive. It’s options do not need customization for general needs.

Creating the database table

In the plugin’s SQL-subfolder there is a file called mysql.initial.sql. After creating a new table called google_contacts in your Roundcube’s database, import the file mysql.initial.sql into the new table.

Installation of Zend package

The Google Contacts plugin needs the Zend GData APIs for it to work. Download the most recent version from http://framework.zend.com/download/webservices and place it into Roundcube’s program/libs folder. After unpacking the according archive, a folder called ZendGdata-<version_number> should be there. In order to have Roundcube finding the relevant libs, create a symbolic link in program/libs as follows:

ln -s ZendGdata-1.11.10/library/Zend .

Configure Google Contacts access

If all went right there should be a new option in a user’s Roundcube settings. Under the section “Address Book” it is now possible to enter one’s Google account name and password. The next access to the user’s address book in Roundcube will trigger the sync with Google’s address book so that all existing contacts will be available.

Transmission daemon: remove finished torrents

In transmission-daemon, downloaded torrents stay listed as long as they are not manually removed. In order to get them flushed automatically, the following command run by cron can help:

transmission-remote -l | grep 100% | grep Done | \
awk '{print $1}' | xargs -n 1 -I % transmission-remote -t % -r

In order to add this to cron, the “%”-symbols have to be escaped with a leading slash like this:

*/5 * * * * /usr/bin/transmission-remote -l | grep 100\% | grep Done | awk '{print $1}' | xargs -n 1 -I \% /usr/bin/transmission-remote -t \% -r

FlexGet Installation in Ubuntu

This post is deprecated. Please go to http://www.grosseosterhues.com/2012/02/flexget-installation-and-upgrade/ for more recent information.

This posts documents the installation of FlexGet in Ubuntu. It uses FlexGet v1.0r2301 and by the time of writing it is tested on Ubuntu 11.04.

There is no apt-package for FlexGet available yet. It has to be installed from PyPI. In order to prepare the necessary build environment the following packages have to be installed:

aptitude install python2.6 python-setuptools

The next line downloads and builds FlexGet itself:

easy_install flexget

The following step is optional. In case an interface to transmission-daemon is of use, the according FlexGet-plugin is installed like this:

easy_install transmissionrpc