Dovecot with vpopmail and lua authentication

I'm running the Dovecot imap server with users and mailboxes managed by vpopmail and for various reasons I want to gradually move away from vpopmail but without losing the maildir++ storage structure for messages.

The first issue is that vpopmail authentication in dovecot is not compiled in by default on the Debian packages so every time I upgrade I have to recompile the packages to support the vpopmail driver.

This post explains how to use the lua authentication with the vpopmail vpasswd files so we continue to use the same users and passwords without relying on the special dovecot vpopmail driver. If assumes your vpopmail installation is in /home/vpopmail , if it's somewhere else just change the location in the auth.lua file.

Disable the vpopmail authentication module in conf.d/10-auth.conf

Add /etc/dovecot/conf.d/auth-lua.conf.ext with this content:

passdb {
 driver = lua
 args = file=/etc/dovecot/auth.lua blocking=yes # default is yes
}
userdb {
 driver = lua
 args = file=/etc/dovecot/auth.lua blocking=yes # default is yes
}

Create /etc/dovecot/auth.lua with the following content:

local function isempty(s)
  return s == nil or s == ''
end

function auth_password_verify(req, pass)
 local str=req.user
 local lastAt = str:find("[^%@]+$")
 local localPart = str:sub(1, (lastAt - 2)) 
 local domainPart = str:sub(lastAt, #str)
 local file = "/home/vpopmail/domains/"..domainPart.."/vpasswd"

 for line in io.lines(file) do
  local u,epass=string.match(line,"([^%:]+):([^%:]+)");
  if not isempty(epass) and localPart==u and req:password_verify("{MD5-CRYPT}"..epass,pass)>0 then
   return dovecot.auth.PASSDB_RESULT_OK, {}
  end
 end
end

function auth_userdb_lookup(req)
 local str=req.user
 local lastAt = str:find("[^%@]+$")
 local localPart = str:sub(1, (lastAt - 2)) 
 local domainPart = str:sub(lastAt, #str)
 local file = "/home/vpopmail/domains/"..domainPart.."/vpasswd"
 for line in io.lines(file) do
  local u,epass=string.match(line,"([^%:]+):([^%:]*):");
  if localPart==u then
   return dovecot.auth.USERDB_RESULT_OK, "uid=vpopmail gid=vchkpw"
  end
 end
 return dovecot.auth.USERDB_RESULT_USER_UNKNOWN, "no such user"
end

Create Windows 10 install USB from macOS Mojave

If you want to install windows on your mac and you need s bootable usb stick with windows, here's how to do it. Please note, this is not for installing windows on other devices.

There is a solution using unetbootin , one with bootcamp assistant and one using mounting and manually copying files but none of them worked for me.

This solution worked for me in macOS High Sierra and Mojave and it will probably work on other versions.

First you have to convert the windows iso file to an img:

hdiutil convert -format UDRW -o windows.img windows.iso

The result of this would most likely be a file named widows.img.dmg. The next step is to unmount the usb disk ( diskutil list to see which one it is , in my case this was /dev/disk2 )

sudo diskutil unmountDisk /dev/disk2

Now just use dd to copy the image to the usb device:

sudo dd if=windows10.img.dmg of=/dev/disk2 bs=1m

DIY dynamic dns and ssl certificates

Use SFTdyn to host your own dynamic dns server. The github page describes the whole setup process so there's no point in going over it again.

This article will show you how you can create a free letsencrypt certificate for these dynamic domains without running a http server on them. ( in case you want to use them for something like a smtp service or a http server that's not supported by the certbot plugins ).

When setting up SFTdyn you configured bind to accept local updates. We'll use the same key for the certbot dns plugin so it can do dns challenges for creating and renewing certificates.

The certbot dns plugin needs a configuration file ( ~/.secrets/certbot/rfc2136.ini ) that looks like this:

# Target DNS server ( always use an ip address here )
dns_rfc2136_server = 12.34.56.78
# Target DNS port
dns_rfc2136_port = 53
# TSIG key name ( this has to be set to local-ddns ) 
dns_rfc2136_name = local-ddns
# TSIG key secret ( get this from the ddns server's /var/run/bind/session.key  ) 
dns_rfc2136_secret = splfUqpteoHQ43DDzGKvMS6L3oCjzv20WpKfhxhgWo0=
# TSIG key algorithm
dns_rfc2136_algorithm = HMAC-SHA256

The secret will probably change when you restart bind, so make sure to update the rfc2136.ini file after you do it. Or you can add the contents of the /var/run/bind/session.key file to the named.conf.local file and then change the key's name to something like "remote-key" and then also change the ~/.secrets/certbot/rfc2136.ini file to reflect the name of the new key so you don't have to bother to copy the key every time you restart bind.

When all this is set up just run :

certbot certonly --dns-rfc2136 --dns-rfc2136-credentials ~/.secrets/certbot/rfc2136.ini -d <my-domain-name>

Installing cordova fails

I've just set up a new computer (macOS) and when I tried to install cordova it failed. It looked like it was doing somthing for a suspiciouly short time and then it exit whitout any error.

It seems the problem was I was trying to install it from an app folder that contained a folder named cordova and although I was running "sudo npm install -g cordova" which is supposed to install it globally it didn't work. All it did was to link my app folder to /usr/local/lib/node_modules . After I removed the link and ran "npm install -g cordova" it installed correctly.

Note to self: next time you install cordova on a new computer make sure you don't do it from a place where there's already a "cordova" folder.

Tevo Little Monster delta grid compensation

Before you start the calibration check if you have the grid calibration enabled in the smoothieware config file:
My config has this:

#Example for the delta grid leveling strategy
leveling-strategy.delta-grid.enable true # Enable grid leveling
leveling-strategy.delta-grid.radius 140 # Grid radius in millimeters
leveling-strategy.delta-grid.size 29 # Grid size, must be an odd number

The bigger the grid size, the more points it will test and the longer will take.

Heat the printbed to the temperature you normally print with.

Do the grid calibration:

; home
G28

;bltouch alarm release
M280 PS10.6

; probe pin down
M280 S3

; set bed temp to 80C
M190 S80

; do delta calibration
G32

; save trim to sdcard
M500

; probe the grid and enable grid compensation
G31

; save the grid to sd card
M374

;To set the zero for the print head
;Make sure there is no left over material on the nozzle ( maybe heat and wipe it )
;go home
G28

;go close to Z=0
G0 Z1

; put a sheet of thin paper under the nozzle
; Then move the head ( using the printer controls ) until it presses on the paper sheet, until ; you feel the friction when you pull on the paper. This is the Z=0 point
; Use this point as the homing offest
M306 Z0

;save to SD card
M500

But even after all this it seems the prints are still not all right, so another option is to do some manual calibration of the hotbed using some springs. 

Recompile Dovecot with Vpopmail on Debian

Here's a script I use to quickly recompile dovecot with vpopmail support on Debian 6.0. I'm using the backports repository to get the 2.1 version of dovecot.

Now I'm using the following script every time I update and I get a new version of the dovecot packages.

  1. span style="color: #ff0000;">'s/with-sqlite \\/with-sqlite \\\n\t\t--with-vpopmail \\/'"--with-vpopmail""*.deb"'dbg'

The last line also installs all the dovecot packages except for the debug one. You might want to change it if you don't want all.

grub2 notes

Grub2 configuration is in /etc/defaut/grub ( on Debian at least ).

Set GRUB_DEFAULT=saved in /etc/default/grub if you want to start with the last good boot.

update-grub - detects kernels automatically and writes boot lines to /boot/grub/grub.conf

grub-set-default - set the default entry ( First boot entry is number 0 )

grub-reboot - set the boot entry for the next reboot only ( for testing new kernels or other boot stuff especially when you're working remotely )

Howto check MySQL replication consistency

If you want to be sure the data on the slave is the same as the data on the master ( yes sometimes is can happen to bbe diffeent ) you can use pt-table-checksum ( part of percona-toolkit ) to compute checksums for the data in the tables and then compare the checksums from the master with the ones on the slave(s).

Some options

Use with replication

pt-table-checksum can be used to compare any two databases/tables but if you want to compare everything on master and it's slaves you can use the
--replicate option to connect only on the master and compute checksums. The checksums will then be computed on slaves too by replicating the checksum statements.

Detecting slaves

If you have slave hosts running on non standard ports use option --recursion-method=hosts to tell pt-table-checksum how to detect the slaves. Tell the slaves to report their hostname and and port with report-host and report-port in my.cnf . This will make them show in "show slave hosts" issued on the master.

If the "hosts" method doesn't work, try with --recursion-method=dsn=t=dbname.dsns_table . Create a table with the name "dsns_table" and the following structure in the "dbname" database:

CREATE TABLE `dsns` (
 `id` int(11) NOT NULL AUTO_INCREMENT,
 `dsn` varchar(255) NOT NULL,
 PRIMARY KEY (`id`)
)

And put the dsns for accessing the slaves in the dsn field.
Example:

insert into dsns values ('','h=1.1.1.2,u=root,P=3306,p=slavepassword');

For non innodb plugin

--lock-wait-time is required if using a version of mysql without innodb plugin.

Database for storing checksums

pt-table-checksum stores the checksums in mysql so you have to create a database where to store the checksums table. I named mine "mk" since the tool was called mk-table-checksum before it became part of percona toolkit.

Use the --create-replicate-table option to create the checksums table if it doesn't already exist.

Example usage

pt-table-checksum --recursion-method=hosts --lock-wait-time=50 --defaults-file=/home/mihai/mysql.pass -u root --create-replicate-table --replicate-check --replicate mk.checksums 127.0.0.1

Example output

            TS ERRORS  DIFFS     ROWS  CHUNKS SKIPPED    TIME TABLE
09-13T18:25:13      0      0      361       1       0   0.019 mydb.accounts
09-13T18:25:13      0      0       91       1       0   0.079 mydb.announcements

XtreemFS server on MacOSX

Some time ago I tried to get the XtreemFS server to work on MacOSX (Lion).

I had to patch it a bit to make it compile and run. So here's the patch if you want to give it a try:

[download id="33"]

I wanted to use it to have a synchroneous replicated filesystem over a WAN but in the end I gave up this idea and switched to unison.

Btw. I also tried to get gluster fs to work on MacOSX lion and partially succeeded. You can see my changes on github

Which email client for Linux?

I've been a thunderbird user since firefox was named firebird. I was happy with it, I like the way you can easily search over all accounts, how you can archive messages in folders by year by simply hitting the "a" key, I like the threads and even the "gmail" conversations extension even if I don't use it.

One problem with thunderbird is it became unusable when I added an old account which was accumulating mail up to over 170k messages. I know 170k messages is not realistic and no one should have so many messages in inbox but still this made me look for something better.

First let's see what I'm looking for in an email client:

  1. Easy global search like thunderbird
  2. Archive folders and 1 key hit like thudnerbird 🙂
  3. Easy to mark messages as Spam/Junk with a single key hit, would be great to be able to mark multiple messages with a single key ( thunderbird doesn't have this )
  4. Message threads
  5. And last but not least, be able to handle a lot of messages in a folder, don't have to be super fast but at least it shouldn't lock

Here's what I tested so far:

Evolution

This was a nice surprise.
Pro: Evolution downloaded 87k message headers in just a few seconds and had no problems with selecting all and moving around.
Cons:

  • No archives
  • It has global search but it's a bit harder to access
  • Harder junk marking ( shift+ctrl+j instead of just j )
  • Always asks if I want to accept a certificate that's not issued for the exact domain of my email server ( at startup )
  • I could accept all the cons except for the lack of archiving.
    Anyone knows of a plugin that does archiving like thunderbird ? Please let me know!

    Kmail

    Becomes almost unusable with a big inbox ( 87k), hard to select all messages

    Pro: message list groups messages by year ( but this still doesn't beat yearly archives )
    Cons:

    • very slow with a lot of messages, takes a lot of time to fetch message headers
    • adding an account doesn't show it immediately in the accounts list, unless you restart kmail
    • no archives
    • no keyboard shortcut for junk/spam marking ( seriously ???)
    • global search harder to access

    sylpheed

    Pro: It's fast an lightweight but that also means it lacks what I want
    Cons:

    • No archiving
    • No keybboard shortcut for marking junk mail
    • No global search

    Conclusion: there's no client that meets all my needs so I'm going to have to stay with Thunderbird and just use evolution when I want to open a big folder.

    What email client are you using ?